Mariano Rivera pitched the ninth for his fifth save in five opportunities.
The Yankees open a two-game home series with Boston Wednesday.
Bobby Abreu walked, went to third on Alex Rodriguez's 400th career double and scored on Matsui's run-scoring grounder to tie it at 2 in the fourth. A-Rod then scored on Edwin Jackson's wild pitch to make it 3-2.
Derek Jeter and Abreu extended the Yankees' advantage to 5-2 with run-scoring singles in the fifth. Jeter's hit came on a nine-pitch at-bat, while Abreu had been hitless in his previous 12 at-bats against Jackson.
Jackson (2-1) gave up five runs and four hits in five innings. He struck out four and walked four.
Jason Bartlett hit an RBI infield single in the sixth to get Tampa Bay to 5-3.
Matsui put the Yankees ahead 1-0 with a homer in the second, his third of the year. But Jonny Gomes and Mike DiFelice hit two-out, run-scoring singles to give the Rays a 2-1 lead in the third.
The Yankees failed to score after loading the bases with one out in the seventh. Abreu flew out and Rodriguez struck out against J.P. Howell. Rodriguez is 1-for-14 with runners in scoring position this season.
New York also left the bases filled in the eighth when Johnny Damon flew out with two outs.
Tampa Bay loaded the bases with one out in the bottom of the second, but Pettitte worked out of the jam by inducing a double-play grounder from Bartlett.
