City of Newburgh police say emergency crews found heavy black smoke pouring from the two-story home when they arrived just after 2 p.m. Friday.
They say several residents, including at least two children, had managed to escape, but a 1-year-old girl died inside the house.
The three-alarm blaze drew firefighters from volunteer companies in surrounding towns to help.
Police have joined firefighters investigating the cause of the fire.
Newburgh is about 80 miles south of Albany.