What Causes Nipple Pain?

I am a male, 38 years of age, when something comes in contact with my right nipple it hurts for a second or so then goes away. It can be someone hugging me, my shirt hitting it or just my arm rubbing it when I move. I do not feel lumps and when I push to feel lumps the pain goes away for an hour or so. Then it is right back.