What Are The Risks When HCG Isn't High And Fetal Growth Is Slow?

I'm 6weeks 4days into my first pregnancy and had an u\s today. A yolk sac and fetal heartbeat was seen, but the doctor is concerned because the baby isn't as big as it should be? My 2 previous hcg tests were not very high. I went for another hcg test today. Do you think there's cause to be worried about miscarriage?