Could Something Serious Be Wrong When Headaches Worsen Without Contacts And Sometimes Even With Them?

I’ve been having headaches since March 3rd , at first they were bad but then they got better & I wear contacts . But when I take them out my head hurts more and sometimes my head even hurts with my contacts in... what could be wrong? I’ve been scaring myself into thinking it’s something very serious