I Have A Habit Of Rubbing Dead Skin Off My

I have a habit of rubbing dead skin off my face when I'm sitting around (watching tv, on the computer, etc). Basically I'm exfoliating my skin but I use my fingers and I don't make my skin damp first, it's dry.

Will this increase wrinkles on my face?

Thank you!