Dainius Zubrus scored twice, and Scott Clemmensen made 17 saves to help New Jersey win for the fourth time in five games and the seventh in 10.
Nick Foligno, Brendan Bell and Chris Kelly scored for Ottawa, 3-8-2 in its last 13 games. The Senators are 0-5-1 on their eight-game trip while the world junior tournament is in Ottawa.
Rolston's winner came on a slap shot from about 40 feet with Senators defenseman Brian Lee and Devils forward Brian Gionta standing in front of Auld.
The Senators took one-goal leads three time only to see the Devils tie it.
Langenbrunner tied it for the last time after the Senators seemed to clear the puck out of their zone with New Jersey on a power play.
Langenbrunner took a pass from Oduya and beat Alex Auld with a hard shot on which Ottawa defenseman Jason Smith might have either screened his goalie or touched the puck. The defenseman slammed his stick to the ice after the play.
Kelly had given Ottawa a 3-2 at 7:14 of the third period with a shot in close after being set up by some nice work behind the net by Jesse Winchester and Antoine Vermette, who also set up Foligno's opening goal.
The Senators, who came into the game with a league-low 84 goals, twice took one-goal leads on power plays, only to see Zubrus tie the game with goals in close.
Zubrus even had a hand in the Ottawa's opener at 2:10. He was sent off 15 seconds after the opening faceoff for hooking and Foligno tallied his sixth goal of the season from in close with 5 seconds left on the penalty.
Zubrus tied it in the final minute of the first period putting the rebound of a Gionta shot past Auld. Patrik Elias made a nice cross-ice pass to set up Gionta's shot although it looked like the Devils forward was close to be offside.
Bell put the Senators ahead 2-1 at 3:50 of the second period with the help of a lucky bounce. Jason Spezza sent a pass from just behind the goal line that deflected off defenseman Bryce Salvador, forcing Clemmensen to make a save. Bell put the rebound into an open net.
Zubrus got his 10th goal of the season a little more than seven minutes later after play seemed to pause for a second with the puck under Gionta in front of the Ottawa net. The Devils got control and Gionta sent a cross-ice pass to Elias who found Zubrus alone in front for an easy redirection into an empty net.
