If "linger" means "to stay in a place longer than necessary":
* Leave
* Depart
* Go
* Hurry
* Rush
If "linger" means "to remain in one's memory":
* Fade
* Disappear
* Vanish
* Forget
If "linger" means "to persist":
* End
* Cease
* Stop
* Finish
If "linger" means "to move slowly":
* Rush
* Hasten
* Speed up
* Accelerate
It's important to consider the specific situation to determine the most fitting opposite word.