Does Withdrawal From SSRI's Cause Extreme Stress?

Three years ago my Primary Care prescribed Clonazapan I think it was and Xanax for emotional and physical exhaustion. I had to quit my position of twenty years and lost my insurance and Insurance so new Dr at Community clinic suggested Citalopram. This was a night mare and really caused me to have problems with memory and extreme confusion. Two months ago I finally weaned myself off all SSRIs but I am emotionally unstable and it is very bad in some ways but my memory is better and the fog and confusion has lifted. I am changed for the worse mentally and emotionally. I was recently prescribed Prednisone and antibiotics for bronchitis etc and had bad effect from that which years ago never gave me a problem. I am over sensitive it seems and very afraid of all medications. I am struggling daily but recently paid to go back to my original Primary Care that I trust and respect. I told him I did NOT want to walk out of his office with a RX and what did he recommend I do......his recommendation was good multi vitamin along with the B vitamins and other vitamins I take and continue blood pressure meds and concentrate on lots of walking and sunshine and Yoga or Tai Chi and try to limit high stress work like I previously had. My problem is that everything is stressful to the extreme... It is true that when you stop the SSRIs the side effects are really bad...I am like a different person...... Do you have any additional ideas or suggestions?