While it might seem like "ed-" is the prefix, the "e-" is actually a separate prefix meaning "out" or "from." It is combined with the root word "dible," which means "able to be eaten."
So, "edible" literally means "able to be eaten from" or "able to be eaten out."