What Could Recurring, Painful Lumps On Labia Mean?

For the past 4 days I have had painful lumps that sometimes swell on my labia. I am not sexually active, I went to see a doctor and they suggested I may have herpes however the lumps aren't blisters and they don't become open sores. I have had this once before and the lumps go away after a week.