Here's why:
* Prefixes are added to the beginning of a word to change its meaning. In "manifesto", "man" doesn't change the core meaning of the word.
* "Manifesto" comes from the Latin "manifestum", meaning "clear" or "obvious". The word "man" is not part of its etymological origin.
Therefore, "man" is simply part of the word "manifesto" and not a prefix.