What is the study of surnames called?

The study of surnames is called onomastics.

While onomastics encompasses the study of all names, including personal names, place names, and other types of names, a sub-field specifically focused on surnames is surnames.