Coming off a 3-0 defeat Sunday night against the Boston Red Sox, New York failed to provide any run support for ace Gerrit Cole, whose only run allowed followed a misplay in center field by Aaron Hicks.
"Obviously we gotta play better, we gotta do better than that," Yankees manager Aaron Boone said.
The AL East leaders were blanked for the fourth time in nine games and dropped to 8-16 since the All-Star break after a 64-28 start.
Yarbrough (1-7) pitched four innings as a bulk reliever for his first win in nearly a year, striking out six without a walk. He previously won Sept. 24, 2021 and had lost eight straight decisions over 14 appearances, matching the franchise record.
"It was nice," Rays manager Kevin Cash said. "I think Yarbs has pitched really well here as of late. He probably deserves a little bit better in the win-loss column, but he's contributing in a big way."
The Yankees hadn't been held without a run in back-to-back games since Sept. 22-24, 2016, when they were blanked once by the Rays and twice by the Toronto Blue Jays.
"Right now we gotta do better and it's hard right now," Boone said. "It's hard and we gotta find a way. We gotta keep digging ourselves out of this."
Hicks got turned around on the warning track and couldn't field David Peralta's flyball in the fourth inning, resulting in a triple. Isaac Paredes then singled off Cole (9-5) for a 1-0 lead, enough to get the Rays on their way to a third straight victory.
"First off, I turned the wrong way," Hicks said. "I tried to recover. I ended up thinking it was still right on top of me and it ended up being behind me."
Tampa Bay added three runs in the ninth on Jose Siri's RBI single and Yandy Díaz's two-run single.
Cole allowed one run and five hits in six innings. He struck out six, walked two and allowed one run or less for the 11th time this season.
Hicks batted with the bases loaded in the fourth but hit a soft comebacker to Yarbrough, who started a 1-2-3 double play
After Yarbrough's 58-pitch outing, Ryan Thompson got the next five outs, getting Aaron Judge on a double play in the eighth. Brooks Raley fanned Anthony Rizzo before the Rays added on, and Jason Adam pitched the ninth.
Rizzo thought he was grazed on the right leg in the third, but plate umpire D.J. Reyburn ruled he leaned into the pitch. Rizzo and manager Aaron Boone vehemently argued to no avail.
"It shouldn't have been called tonight," Rizzo said. "I have been hit by a lot of pitches and it's never been called."
After striking out on the next pitch, Rizzo slammed his helmet in the dugout five times.
"Whenever you feel like you got screwed over on a call anytime it's annoying because we're out here competing at the highest level and I'm up here talking to you guys (media)," Rizzo said. "And I feel like the umpires have zero accountability when they miss something like that. I know they're not trying to miss, but it's just frustrating.
Rizzo has been hit by a pitch 17 times, tied for the AL lead with Seattle's Ty France.
Rays: SS Wander Franco (right hamate bone) will join Triple-A Durham to start a rehab assignment Tuesday or Wednesday.
Yankees: OF/DH Giancarlo Stanton (Achilles tendon) took batting practice in the indoor batting cage and also did fielding drills in the outfield. He said he is not quite ready to start a rehab assignment. ... RHP Luis Severino (strained right lat) threw 25 pitches in his first bullpen session since going on the IL July 14. Severino said he feels "the best I've felt all year." ... An MRI on INF DJ LeMahieu showed inflammation in his right big toe and he is day-to-day.