It's easy to be misled by the "vers" part, which appears in words like "convert" and "verse," but in "controversy," "vers" is part of the root word, not a separate prefix.
Here's the breakdown:
* con-: means "with," "together," or "against"
* -versy: derived from the Latin word "vertere," meaning "to turn"
Therefore, "controversy" literally means "a turning against" or "a turning with" depending on the context.