Suggest Treatment For Swollen Labia

I have a swollen labia. It seems to almost be an extra labia even though that's not possible. It's part of the skin attached from the vagina to anus. It's puffy, red, and hurts to touch. I noticed some small bumps around the area too. I'm not currently sexually active. I've been working out a lot and wearing tight pants so maybe it's a yeast infection?