Specifically, it comes from the Min Nan dialect, where it is pronounced "tê". This word referred to the tea plant itself and later became used for the beverage made from it.
The word traveled to English through Portuguese traders who interacted with Chinese merchants in the 16th century.