is a paper bag noun?

Yes, "paper bag" is a noun.

It specifically is a compound noun, meaning it's made up of two or more words that act as a single unit. In this case, it describes a type of bag made from paper.