Here's why:
* Common nouns refer to general, everyday things. They are not capitalized unless they start a sentence.
* Proper nouns refer to specific people, places, or things (like "John" or "New York City"). They are always capitalized.
Since "finger" is a general term for a part of the human hand, it is a common noun.