Here are some examples:
* Un problème difficile (a difficult problem - masculine noun, masculine adjective ending)
* Une tâche difficile (a difficult task - feminine noun, feminine adjective ending)
So, "difficile" is neither masculine nor feminine; it takes the appropriate ending depending on the noun it modifies.