Yea, I'm blown away someone actually said that, talk about being immensely retarded. "Penis ain't big enough, better be a cuckold." Absolutely pathetic response.
~Either way, it's extremely important to talk to a sexual therapist about this, most people who don't wallow in sadness until the end, and that's not a road you want to go down.
It's quite possible there's surgical procedures, medicinal aid, or maybe even physical aid like regular penis pumping that could gain you back some/all of your confidence. That with proper sexual education and things may end up being right as rain, but you have to be brave and get help.
Remember, people are born with defects all the time, some small, some major. But that's all they are, defects. Modern medicine can help many people regain part or all of the life they should've had back. Don't sit wallowing in depression while drowning yourself in pornography when you can make steps to improving your situation in the real world today.
Also, don't bank on finding a fairy tail partner someday who'll love you no matter what. That shit is movies/media, not real life. Hope is great, but depressed people use hope as a crutch and stop making choices/moving forward. Take charge, get on the phone, set up an appointment. By this time next year your entire life may be different and you'll owe it to yourself for making the first steps.