Here's why:
* It describes an action: It indicates a choice or a liking for something over something else.
* It can be used with a direct object: "I prefer coffee to tea." ("coffee" is the direct object)
* It can be conjugated: "I prefer, you prefer, he/she prefers"
While there are related words like "preference" (noun) and "preferable" (adjective), the word "prefer" itself is always a verb.