Is the s in umbrellas pronounced with a or z sound?

This is because the "s" is followed by a voiced sound (the "m" sound). When a "s" is followed by a voiced sound, it becomes voiced itself, resulting in a "z" sound.