I Have Used Nuvaring Birth Control For About 4 Years

I have used nuvaring birth control for about 4 years now. My husband is bigger then average and he complains about it rubbing him. I took it out last month due to the fact that we are trying to have another baby but he is still complaining that something is rubbing him?? Is there any chance I could of gotten rubbed also and it's still sore?? I'm really confused and worried that it's going to affect our sex life.