My Six Year Old Hardly Ever Gets Sick, But Has

My six year old hardly ever gets sick, but has had three fevers in a month. We had a bad cold or something, which he got the first fever during. After that was gone he got another fever 10 days later for one day. Then a couple weeks later he got a third fever. It lasted two days and on the third day only in evening, fourth day only fever at night. I'm so worried. The doctor says it probably viral, to just wait and see, because he's otherwise healthy. However my son is homeschooled and isn't exposed to many viruses.