Simply Put, I Have A Genetic Defect In My Testicle

Simply put, I have a genetic defect in my testicle which cause me a lot of pain. I'm 34 and had this issue since i was 18. I'm not married and don't plan to be nor do I have/want kids. Every time I go to the urologist to have them removed they refuse. Other than for kids (which I don't want) I can't see any reason for the Dr. to not remove them. Is there any major reason I'm not being told why doctors don't like to remove testicles? Anything I can say so that they will do it?