What Causes Itchy Burning Eyes With Redness, Swelling, Hardness And Skin Peeling?

I am a 34 year old male—healthy, active, non-smoker. In the last couple of months, every time I go swimming I develop very itchy, burning eyes within 24 hours. Over the following four days, My eyelids gradually become redder and swollen; the swelling is usually worst in the morning, particularly under the left eye. I do have a cat, who we've had since October, and a mild cat allergy, but the first few months she was here this did not happen despite my swimming regularly. My goggles (which I wear always while in the pool) are non-latex, so a chlorine-latex allergy probably isn't it. I always get at least a little water in my eyes during a swim, but have taken more care to rinse them out after swimming. After about 5 to seven days, the parts of the eyelids that were red and swollen harden, then peel, after which they are more or less normal again. This doesn't affect my upper eyelid, but just the upper socket and lower eyelid. No redness of the actual eye or discharge. Not sure what I should be doing, but I really don't want to give up swimming! Thank you!