Phelps finished in 1 minute, 56.68 seconds -- the fastest in the morning -- and was 0.22 of a second ahead of Pace Clark.
"My third turn was bad, awful," said Phelps, who is seeking his fifth Olympic appearance. "Killed all the momentum. I felt like I didn't really use much legs, dragging them through the way. But it's something to build off of. We'll see."
Also advancing was Chase Kalisz, Phelps' protege who already made the team by winning the 400 individual medley Sunday. Tyler Clary, who was second to Phelps at the 2012 trials, moved on with the sixth-fastest time.
In each of his four previous appearances at Olympic trials, Phelps competed in the 400 IM on the meet's first day. However, he has not competed in that grueling event since finishing fourth at the London Olympics in 2012.
Phelps had originally been entered in Monday's 200-meter freestyle but was later scratched to focus on the 200 fly, where he has the top four performances in the race's history.
"I just wanted to swim," Phelps said. "That was the most annoying part: having to wait this long. But I'm happy to get the first one out of the way and just keep going."
Last year at the U.S. nationals in San Antonio, Phelps turned in the year's fastest time in the 200 fly at 1:52.94. He'll look to edge closer to that mark in Tuesday night's semifinals.
In other events, Katie Ledecky, who already qualified for Rio in the 400 freestyle, easily advanced to the semifinalsof the 200-meter freestyle with a time of 1:55.60.Leah Smith was second in 1:56.47.
Missy Franklin tied for seventh with Hali Flickinger in 1:58.61 and will face a busy night. She will swim the semifinals shortly before 8 p.m. ET and return about 15 minutes later for the 100 backstroke final in a bid to make her second straight Olympic team.
"I think I do my best when I'm given a big challenge, and tonight I'm really going to have a big challenge ahead of me," said Franklin, who won four golds and a bronze four years ago at the London Games. "My warm-ups have been feeling really great. I think I'm trying a little bit too hard and thinking a little bit too much. It's just about trusting myself and just letting it happen."
Defending Olympic champion Allison Schmitt qualified fourth-fastest for the 200 free semis in 1:57.77.
In the 200 individual medley, Maya DiRado advanced to the semifinals with the fastest time of 2:11.66. Caitlin Leverenz was second at 2:11.89, followed by Melanie Margalis at 2:12.36. Margalis also advanced as third-fastest in the 200 free.
The Associated Press contributed to this report.