As a geographical feature:
* Cove: A small, sheltered inlet of a sea, lake, or river.
* Inlet: A narrow body of water extending into the land from a larger body of water.
* Creek: A small, narrow stream or inlet.
* Estuary: The tidal mouth of a river, where freshwater meets saltwater.
* Sound: A large body of water that is partly enclosed by land.
* Gulf: A large body of water that is partly enclosed by land.
* Harbor: A sheltered body of water used for anchoring ships.
* Marina: A harbor or basin designed for small boats.
As a part of a building:
* Window: A clear opening in a wall that allows light to enter.
* Panel: A flat section of a wall, often used to divide it into sections.
Other synonyms:
* Recess: A hollow or indentation in a surface.
* Curve: A smooth, continuous bend.
* Bend: A sharp turn or angle.
The best synonym for "bay" will depend on the specific context in which it is used.