The children, just 2 and 5 years old, were shot while playing outside on East 181st Street in the West Farms section of the Bronx on Monday night
One of the bullets hit Patience Boyd, 2, in the head. Another 5-year-old girl was wounded in the neck.
Police said the gunman started firing into a crowd of children and adults.
The shooter had apparently been aiming for a 20 year old man, and shot him in the back.
The suspect was identified as Luis Moore, 23.
Officers canvassed the neighborhood and posted signs in hopes of finding Moore.
He surrendered to authorities late in the day.
Charges were pending.