They were rescued by two Suffolk County Police Marine Bureau Officers and two Smithtown Harbor Master Officers.
They found the men treading water after one of them managed to place a 911 call around 5:30 p.m.
One man was wearing a life jacket, and about a mile from shore, the other was closer but was not wearing a life jacket.
They were brought back to the dock at Kings Park Bluff and checked by first responders, but declined further medical treatment.
* Get Eyewitness News Delivered
* Follow us on YouTube
* More local news
* Send us a news tip
* Download the abc7NY app for breaking news alerts