Here's why:
* Proper nouns are specific names of people, places, or things.
* Common nouns are general names for people, places, or things.
Since the Orange River is a specific river with a unique name, it's a proper noun. It's capitalized to distinguish it from other rivers.