Detectives said all the children left alone were under the age of 11, and they were told to watch over each other.
While the parents were away, paramedics were called to the house on a report of a child having trouble breathing.
They discovered 11-month-old Major Gonzalez with a bruise on his head.
The infant was rushed to Brookdale University Hospital, where he was pronounced dead.
The parents were taken into custody when they arrived home. They told detectives they had left the older children in charge of the younger ones.
Following extensive questioning, the parents were charged with endangering the welfare of a child and will be arraigned in Brooklyn Criminal Court.
An autopsy on the dead infant was inconclusive, and further tests will be conducted to determine the cause of death. That could determine if further charges are warranted.
Gonzalez had 10 children, and the family had been visited by investigators from the Administration for Children's Services a handful of times in the past.
However, officials said all of the visits were for routine matters -- such as like failure of a child to show up at school or failure to take the child to the doctor when sick. None prompted further investigation.
In a statement, ACS said they are investigating the child's death:
"ACS is investigating the fatality of Major Gonzalez. We investigate all fatalities of children in NYC when the fatality is reported to the State Central Register for Child Abuse and Maltreatment."