It's called an exclamation mark or exclamation point and is used to show strong emotion, emphasis, or surprise in writing.