The Yankees' edge fell to one game over Boston and is two games over Toronto and Seattle.
"Just couldn't break through enough and love the fight there at the end and the quality of the at-bats there at the end -- just couldn't break through," New York manager Aaron Boone said.
New York needed a win plus a loss by either Boston or Seattle to clinch its fifth straight playoff appearance. Instead, the Yankees fell short against the AL East champion Rays in the opener of a season-ending three-game series.
"We want to clinch, that'd be nice, but we got two more games to do that," New York left fielder Joey Gallo said. "We control our own destiny but that's up to us."
The Yankees lost for only the second time in 10 games.
Rookie Wander Franco gave the Rays a 4-1 lead by chopping a grounder up the middle for a two-run single in the ninth. His hit came a few pitches after he missed a three-run homer by lining a ball a few feet foul in left.
"We got the playoff juices flowing and this is great for guys of all ages," Rays center fielder Kevin Kiermaier said.
Franco's hit also came a night after he went 0 for 4 to snap a 43-game on-base streak, which tied Frank Robinson in 1956 for the longest such string in MLB history among players 20 or younger.
After Franco gave the Rays a three-run lead, the Yankees nearly pulled off a comeback.
Rays reliever Andrew Kittredge allowed Giancarlo Stanton's one-out double and a bunt RBI single by Gallo. Brett Gardner followed by singling home a run before Kittredge fanned Sánchez and Odor to end the game and get his eighth save.
Gallo's bunt hit occurred after Stanton took third on defensive indifference when the Rays shifted third baseman Yandy Diaz to left field.
"We needed all of it as the score ended but just a really well-played game both clubs," Rays manager Kevin Cash said. "The pitching was outstanding on both sides."
Kiermaier put the Rays ahead with an RBI single in the second, hitting a soft fly just beyond the reach of shortstop Gio Urshela. Kiermaier reached base in each of his four plate appearances.
"This was a big win and this is the atmosphere to prepare us for what's to come next week and hopefully many more weeks after that," Kiermaier said.
Nelson Cruz hit his 32nd homer, connecting in the first against Nestor Cortes Jr. (2-3). The Rays improved to 10-7 in the season series and 6-2 at Yankee Stadium this year. The Rays are 11-3 in the Bronx in the last two seasons and picked up their latest win after arriving only 90 minutes before first pitch following a late flight from Houston.
Rookie Shane McClanhan tuned up for the postseason by allowing one run on three hits in three innings. Pete Fairbanks tossed a scoreless fourth and Louis Head (2-0) pitched 2 1/3 innings.
Josh Fleming pitched two perfect innings before Kittredge nearly stumbled.
Cortes allowed two runs and five hits in 4 2/3 innings.
The Yankees honored the 60th anniversary of Roger Maris hitting his 61st homer to break Babe Ruth's single-season record by having his son, Kevin, virtually throw out the ceremonial first pitch.
CRUZ CRACKS TOP 40
Cruz's homer was his 449th, tying him for 40th on the all-time list with Vladimir Guerrero Sr. and Jeff Bagwell.
"I think one of the most memorable homers of my career," said Cruz, who also beat the Yankees with a two-run, game-ending homer off Aroldis Chapman June 10 in Minnesota.
Cruz has 13 homers with the Rays and 292 since the 2014 season, the most in the majors.
Before connecting, Cruz had not homered in his previous 12 games to match a season high.
Yankees: 3B DJ LeMahieu was unavailable because of right hip soreness and was to meet with a doctor. The Yankees did not have any further update LeMahieu left Thursday's game at Toronto in the sixth inning. ... RHP Jameson Taillon (right ankle) threw a bullpen that Boone said went well. Boone said Taillon is a candidate to start Sunday.
Rays RHP Shane Baz (2-0, 1.69 ERA) opposes LHP Jordan Montgomery (6-6, 3.49).
Rays take down Yankees behind Nelson Cruz's HR
Nelson Cruz blasts his 32nd home run of the season to help the Rays beat the Yankees.