Here's why:
* Adverbs modify verbs, adjectives, or other adverbs. They tell us how, when, where, or to what extent something happens.
* Verbs express actions or states of being.
For example, in the sentence "He only ate a sandwich," "only" modifies the verb "ate" and tells us the extent of his eating.
There are some rare cases where "only" might function as an adjective, but it's uncommon.