What Causes Pepper Like Stools In 5 Month Baby?

My 5 month old baby is exclusively breastfed, has had a few diapers now that look like pepper has been sprinkled throughout. The first time was on Monday, LOTS of pepper, thought it was something I ate, but it happened again today, not as much but still quite a bit. Between now and then he hasn t had a proper poo, he sometimes goes everyday, other times goes once or twice a week. I also cut out dairy over 6 weeks ago because of his eczema. I plan to take him to his doctor on Monday, but should I be rushing him before that?