What Causes Hole And Bump In The Leg?

I have a small hole in my leg accompanied by a dark bump that surrounds it.I have had is for years and my doc told me not to worry about it.
Well I noticed that if I do not clean it out it fills with dry, dead skin and starts to smell. The last time I did this was about 1.5 weeks ago and I caused it to bleed a bit, so I let it scar over, put neosporin on it and a bandaid. I was happy because it looked like it was going to go away.
Then I shaved and took the scab off... if had a foul smelling yellowish puss that came from it and now it is a smaller hole with the red bump around it and it is producing a white solid form of what I believe was the puss (when I push on it; like a zit).
Do you have any idea what this could be?What caused it?