I Have Been On The Pill For Over A Year

I have been on the pill for over a year now. My periods lasted about 3-5 days.  This last month, on the last day of my period (monday July 6th) I had sex and started my new pack of pills the next day as I was supposed to.  For some reason I am nervous about getting pregnant because he ejaculated on a day I had not taken the pill. It was the last day of the "sugar pills".  I guess my question is should I be worried? I havnt done anything different than usual. Just hoping to put my mind anything and heart to rest.