Here's why:
* It modifies a verb: It describes how something is done. For example: "Considering the circumstances, it was a good day all and all." The phrase "all and all" modifies the verb "was" by explaining the nature of the day.
* It expresses a general assessment: It means "overall," "taking everything into account," or "in general."
While "all" can be an adjective or pronoun on its own, when used in the phrase "all and all," it takes on an adverbial function.