Witnesses said the dog had fallen through the ice and was in the Raritan River in Bridgewater for 30 minutes before help arrived.
Officials say two firefighters were tethered together and turned a ladder on its side to reach the Labrador/Rottweiler mix.
The dog was taken to a veterinary hospital, where it was warmed up.
The dog didn't have a collar, but a microchip helped officials identify its owner.
Get Eyewitness News Delivered