Wakir Bryant, 27, was arrested Friday evening on aggravated manslaughter charges in the death of Mikhy Robinson, who suffered a gunshot wound to the mouth earlier in the day, according to police and the Essex County prosecutor's office.
They called the shooting a "senseless crime."
Bail was set at $1 million, according to a statement issued by police and prosecutors.
It was not immediately known whether Bryant had a lawyer.
An official familiar with the investigation said Mikhy's mother was asleep Friday morning and awoke to see her boyfriend holding the bleeding toddler and saying repeatedly: "I'm so sorry."
The official spoke on condition of anonymity because that person was not authorized to release details about the case.
Authorities said the boyfriend drove Mikhy and his mother to Newark Beth Israel Medical Center, where the boy was pronounced dead.
The official said the boyfriend has a lengthy criminal history, including prior weapons offenses.
Bryant was most recently released from prison on June 1, The Star-Ledger newspaper reported, citing state and county records.
Mikhy's death was the 90th homicide of the year in New Jersey's largest city, which has seen an 8 percent jump in homicides in 2011, police said.
Police Director Samuel DeMaio said the figures have improved since he took the helm of the police department in June, when homicides were up 60 percent over the year before, and ordered an immediate reorganization.
Since then, crime has gone down slightly and the number of weapons retrieved has gone up markedly, from 537 in 2010 to 703 guns in 2011, according to department statistics.
Get Eyewitness News Delivered