Nassau County detectives say the thefts happened in 2011 in Kings Point, a village whose waterfront mansions partly inspired F. Scott Fitzgerald's novel, "The Great Gatsby."
Police say Anatoliy Maryuk took the buckets from a home where he worked, then initially tried to sell them through the Sotheby's auction house.
Two buckets were later sold on eBay to a man in Maryland.
The FBI began investigating after Sotheby's notified the rightful owner.
Authorities say all four buckets have now been recovered.
Maryuk was scheduled to be arraigned Saturday in Hempstead.
It wasn't clear if he had a lawyer.
Nassau County police wouldn't reveal who owned the buckets.