Here's why:
* Passive Voice: The sentence is in the passive voice, meaning the action is being performed on the subject ("he") by the object ("you").
* Prepositional Phrase: "By yourself" is a prepositional phrase, but it doesn't fit the grammatical structure of the passive voice in this case.
"He was seen by you" clearly conveys that you are the one who saw him.