The Yankees said Whitley was being examined by a team physician and would undergo an MRI on Friday.
"We won't know until tomorrow, but there's obviously concern about this," manager Joe Girardi said. "It's in that area you don't want to talk about. He's been trying to tough it out, and we had no inkling that he had an issue. So we'll just have to wait until tomorrow, and then we'll know."
Whitley left after his 43rd pitch of the game, a slider to Joey Butler that resulted in his second consecutive walk. Whitley pointed to the inside of his right elbow as Girardi and a team trainer hurried to the mound. After a brief discussion, Whitley was removed from the game and replaced by Esmil Rogers.
"It's been in there for a little while, but adrenaline would take over during a game," Whitley said. "Tonight, it just wasn't taking over and I felt it pretty much the whole time."
The Yankees trailed 1-0 at the time, but Rogers' first pitch was hit over the left-field fence by Rays catcher Rene Rivera for a three-run home run. Two of those runs were charged to Whitley, who was given the loss in a 6-1 Rays win.
Whitley had been recalled from Triple-A Scranton on April 28 to take the rotation spot of Masahiro Tanaka, who went on the disabled list with a forearm strain and will throw his second bullpen session Friday.
"It's frustrating, but I've been dealing with [tightness] for a while," Whitley said. "Of course there's a concern, but I'll know more about it tomorrow."
Girardi saidChris Capuanowill take Whitley's spot in the rotation. Capuano is coming off the disabled list, where he began the season after suffering a quad strain in spring training. He pitched six innings for Scranton on Tuesday night.
The 25-year-old Whitley is 1-2 with a 4.19 ERA in four starts this season.
The Associated Press contributed to this report.