Here's why:
* It's a combination of a verb ("sweep") and a preposition ("off").
* The preposition changes the meaning of the verb. "Sweep" on its own means to move something with a sweeping motion. But "sweep off" specifically means to remove something by sweeping.
* It acts as a single unit. You can't separate "sweep" and "off" and still have the same meaning.
Let me know if you'd like to explore more about phrasal verbs!