"Payo" is a term used in some Romani (Gypsy) cultures to refer to a non-Romani person. It can be used in a neutral way, simply meaning "non-Romani," but it can also be used with a derogatory connotation, implying someone is untrustworthy or deceitful.
Therefore, "la palabra de payo" can mean:
* "The word of a non-Romani person" (neutral)
* "The word of a non-Romani person, who is untrustworthy" (derogatory)
The exact meaning depends on the context and the speaker's intent. It's best to avoid using this phrase unless you are familiar with the nuances of Romani culture and understand the speaker's intention.