I Am Having Fluttering In My Chest. Heart Rate Fine,

I am having fluttering in my chest. Heart rate fine, bp fine. Every once in a while I feel like my breath is taken away and I have to take a deep breath to catch it again. I have an inhaler for allergy relayed asthma but it doesn't seem to help. This has been happening since yesterday but seems to be happening more after. It went completely away last night and started again this morning. I am wondering if I should be worried. I am still able to do things but am tired.