The incident happened at 9th Street and Fifth Avenue in the Park Slope section around 12:30 p.m.
Authorities identified the young victims as a 4-year-old girl, Abbigail Blumenstein and a 1-year-old boy, Josh Lew.
The little girl's 34-year-old mother was also struck by the vehicle. Witnesses say she is pregnant and was pushing the little boy in a stroller and carrying her daughter at the time.
She was taken to Methodist Hospital in critical condition, but is now stable and expected to survive.
Officials say the boy's 33-year-old mother was also struck and is in serious but stable condition.
"We do not exactly know what happened yet, there is a full investigation underway. But I will state the obvious, this loss of life is tragic, painful for all of us, particularly those of us who are parents," NYC Mayor Bill de Blasio said.
After striking the victims, the 44-year-old driver crossed into an oncoming lane and crashed into a parked vehicle, officials say. She was taken into custody. The woman was questioned by police on the scene and taken to a nearby hospital for evaluation.
It is unclear if the driver was under the influence of anything or had a medical episode.
Kemberly Richardson tweeted video of the incident:
#Exclusive watch the white car at the light. For some reason the driver goes through it, hits people in the cross walk. 2 kids are dead & a woman, who is pregnant and 1 of the kid’s mom, is in critical. The female driver is with police . More soon @ABC7NY #Parkslope #Brooklyn pic.twitter.com/svukC2lKja— Kemberly Richardson (@kemrichardson7) March 5, 2018
"Everybody is like stop stop," said witness Jennifer Muniz. "Everybody yelling, people running crossing the street screaming, crying, call 911. People trying to cover the baby to stop her from bleeding."
Jessica Sierra immediately ran to help the little girl.
"I ran to her, a bunch of people ran to the mom and then I didn't even realize there was another baby down the block," said Sierra.
Meanwhile others tended to the mother laying helpless in the middle of the street.
"All she kept saying was, my baby my baby and she kept on trying to get up," said witness De'sir'ee Williams.
"She did get to her knees but then she dropped back down because she had a leg injury, because when the ambulance finally got here her head was bleeding," said witness June Clark Smith.