It's a neutral term, simply describing the fact that someone or something is known for a specific reason. It doesn't imply anything about whether that reason is good or bad, but it usually refers to something negative or undesirable.
Here are some examples of how "notorious" can be used denotatively:
* "Al Capone was notorious for his criminal activities."
* "The city is notorious for its high crime rate."
* "That restaurant is notorious for its terrible service."