Brief Answer:
Might be due to wax problem or dryness of ear skin

Detailed Answer:
Hi,thank you for posting your question.
I have gone through your query.

There can be following possibilities:

1.)You might be having wax in the ear.
When you itch with ear buds,if any brownish sticky stain comes ,then it is possibly wax.
For which you can use any wax solvent ear drops,like Soliwax or Dewax ear drops, 3 drops three times a day for 7 days.
Generally,wax will get removed after getting liquified using ear drops.
If not, then you might need to personally consult an ENT specialist for ear examination and wax removal.

2.) You might be suffering from dry skin in ear.
For this you can use any liquid mositurizer like coconut oil in the outer aspect of ear.

If still itching remains,then you can use mild steroid cream like quadriderm cream in the outer aspect of ear for around 2 weeks.

Don't itch you ear with ear bud,as it may harm your ear skin and cause ear pain and infection.

Hope this helps.
If you have any query , I will surely try to answer.
Thank you.

---