I Have Irregular Cycle Periods. Last Month, I Stopped Taking

I have irregular cycle periods. Last month, I stopped taking birth control pills after 2 years and four months of taking it. Now, my partner and I uses the withdrawal method. Just this month, I noticed that I was already 3 days late on my period from my usual 28 to 29 cycles. Now my question is, is there something to worry about my case?