Here's why:
* The sentence is about Jane whispering something.
* The pronoun "ME" is the object of the preposition "to" implied in the phrase "told ME".
* Since Jane is the one whispering and the one being told about something, "ME" refers back to her.