Similar meaning, but less formal:
* Celebrate
* Exult
* Be happy
* Be delighted
* Be thrilled
* Be overjoyed
More formal synonyms:
* Revel
* Jubilate
* Exult
* Triumph
* Glorify
Synonyms that emphasize specific emotions:
* Be ecstatic (intense joy)
* Be exhilarated (joyful excitement)
* Be jubilant (exuberant joy)
The best synonym for "rejoice" will depend on the specific context of your writing.