The Latin word you're likely thinking of is "gramma," which means "letter" or "character." This is the root of the English word "grammar," which refers to the rules of language.
However, the word "gram" in the context of weight is derived from the Greek word "gramma," which also means "letter." This is because the gram was originally defined as the weight of a specific volume of water, and the volume was marked with a small letter "g" on the measuring instrument.
So, while "gram" itself isn't a Latin word, its origin can be traced back to the Latin "gramma" and the related Greek word.