Hello,
Your situation is difficult & I can understand your dellima. But you have to move on. Everyone has a past in one or the other way..but what is important is how he takes care of his present. How does he behaves with you, how is he emotionally with you? If these things are good and he dosent give you any reason to complaint, then you must forgive him.
I know that you cannot forget all what you saw and neither im telling you to forget it...it is waste to try such a thing...but at the same time tell yourself that it was his past , when you were not there in his life. He had emotions and did few things under emotions...and so do many people..

Let this thing stay there somewhere in your heart and try to move ahead with him.
Donot do injustice to him like this, he dosent deserve to be handled in this way ..( provided he is good with you now).
The best way to get out of misery is to make fun of that issue...may be laugh out loud about that thing with him..or may be tease him ..and just try to accept and let it go. This is what is right and this needs to be done now.

I hope you understand
May God give you courage to have a big heart.
Dr. Manisha Gopal
MD Psychiatry