What Is The Dirt Like Thing Coming Out Of My Skin?

I've always had meticulous bathing/hygiene habits and take 2 baths/showers daily. Recently, I've noticed that when I bath/shower with a white washclothe, it seems like tons of --- what looks like dirt --- rubs off my skin. I'm wondering if this is an indication of some skin disease. Could the pigment be literally rubbing off?
Thanks...