Yesterday I Had Unprotected Sex With My Boyfriend During My

Yesterday I had unprotected sex with my boyfriend during my 7 day break from birth control. Last month I took the morning after pill, had a very light period and took a pregnancy test at around 4 weeks after and it was negative. This break from the pill I did not have my period. Should I be worried? He also did not ejaculate yesterday so should I be concerned about yesterday?