The NYPD Harbor boat came to the rescue at about 4:10 p.m.
When police arrived, a 2-year-old male Airedale Terrier named Bodhi was in the water under the West Village pier and the dog's owner was standing on a piling.
He had climbed down to from the side of the pier in an attempt to rescue his dog.
Police moved in and were able to use a canine noose to pull Bodhi from the water and onto the boat.
The dog was then wrapped in towels and brought into the warm cabin of the boat to prevent hypothermia.
Bohdi's owner says he had been throwing a Frisbee to the dog when the disc skipped off the ground and into the water, and the dog jumped in after it.
There were no reported injuries as a result of the rescue.