How To Treat Penis Shaft Lumps?

About three-4 months ago i found a couple little bumps mid right shaft on my penis. they both gave off a little bit of white puss when squeezed. One of them dissapperared a little while ago but the bigger one thats kinda more under the skin seems to have gotten larger (a little bigger then a pea) it gave of some more yellow white puss turning to clear after. it seems to get irratted/swollen after messing with it. it also moves around with the skin. Basically i want to know if its anything dangerous, i would rather not go to the doctor if its not anything life threatning (or penis threatning). Is there anything i can do about it myself or do i just have to wait it out? Oh im 22 and only been with one girl and am still with her. Help with this will be greatly appreaciated :)