What Causes Abnormal Cells In The Left Milk Duct?

I m at higher risk for breast cancer because I have abnormal cells in my left milk duct. Had mammos, biopsies, even a surgical biopsy. Surgeon said no cancer but is referring me to a medical oncologist to get his opinion. She said he might rx tamoxifen. Do I have to take it? What happens if I dont?