The confusion might come from the fact that "per" is sometimes used as a prefix in French words, and in French, "per" can take the acute accent (é) or the grave accent (è). However, in English, "per se" is always written as per se and does not require any accent marks.