What Causes Reduced Foetal Head Diameter?

I am 35 and having my first child. I am at 36 weeks and had my third weekly checkup yesterday. Last week I was told that the baby is strong and all of her measurements are where they should be at this point. I was told yesterday that my baby (due in two weeks) may be on the smaller size (which tracks with me and my siblings). I received a call from my Dr a few minutes ago indicating that my baby's head appears to be small. He wants to send me to a specialist for a "second opinion." I am now very frightened, should I be? What could this mean and why is it a concern now, but never detected/mentioned before?

Please respond with my sincere thanks.

Worried Mom-to-Be