What Causes Delayed Periods While Being On Birth Control Pills?

I've been on birth control for 4 months now and haven't missed a pill but I don't always take it the same time every day. About a month ago I let my boyfriend finish inside me with no condom and now my period is late. I should have gotten it 3 days ago but it's still not here, should I be worried I'm pregnant?