Midfielder Kaku notched an assist and Luis Robles finished with four saves for New York (10-8-4, 34 points), which has won two of its last three matches overall and improved to 3-5-3 on the road.
The victory was Red Bulls' first in their last four road games versus Orlando City. They had recorded a draw and two losses against the host club since posting a 2-0 victory in 2015.
Orlando City's Carlos Ascues and Tesho Akindele each hit the crossbar in the 60th and 70th minute, respectively, and former Red Bulls midfielder Sacha Kljestan's blast caromed off the far post in the 77th minute.
Brian Rowe made four saves for Orlando City (7-10-5, 26 points), which failed to convert any of its nine corner kicks in the second half.
New York opened the scoring in the 32nd minute as Kaku's through ball connected with White, who had gained a step on Orlando City's Robin Jansson.
White quickly unleashed a sliding shot that caromed off Rowe and just over the goal line for his seventh goal of the season, tying him with midfielder Daniel Royer for the team lead.
Orlando City pressed for the equalizer just four minutes later, but Nani's blast from well outside the box sailed wide right of the goal.
The host club was not done, as Akindele settled the ball at the top of the box in the 51st minute and unleashed a shot that required a two-handed save from Robles.
The Red Bulls earned the first quality scoring opportunity of the match, but former Orlando City defender Amro Tarek's header from the center of the box was denied by Rowe.
Tarek had another point-blank opportunity in the 25th minute, but his header in the center of the box off a free kick was thwarted by the extended right leg of Rowe.
Orlando City hit woodwork 3 times in loss to Red Bulls
New York Red Bulls escaped Exploria Stadium with a 1-0 win vs. Orlando City, thanks in part to the woodwork around Luis Robles' goal.