The boy, who witnesses say was around 3 years old, was with his parents and brother on the promenade near Emmons and Ocean Avenues.
The boy was looking at the swans through the metal railing and somehow fell into the Bay.
His father, who didn't know how to swim, instinctively jumped in after him.
Several Good Samaritans also came to his aid.
The owner of a clam bar even grabbed a dog leash and held it over the railing for the boy.
The Coast Guard eventually made it to the scene and helped everyone out of the water.
The boy suffered a cut on his head when he hit the concrete promenade on his way into the water.
Everyone is expected to be OK.