It's derived from the Greek words:
* akron (ἄκρον), meaning "peak" or "summit"
* stichos (στῖχος), meaning "line" or "verse"
Thus, "acrostic" literally translates to "peak-line" or "summit-line," referring to the way the first letters of each line in an acrostic poem form a word or phrase vertically.