The Jets also fired vice president of player personnel Brian Heimerdinger, sources told ESPN's Adam Schefter and Jeff Darlington.
Johnson said he came to the decision after conducting a "deep dive" into the inner workings of the organization after last season's 4-12 finish.
"There's never a good time to do this," Johnson said during a conference call with reporters. "This is about more than one draft or one free agency, even two. As I educated myself about this process, it became more clear to me, the deeper I got in this building, that it was time for a change and a time to move on."
Just months after NFL Network draft analyst Mike Mayock took the Oakland Raiders' GM job, his replacement at the network, Daniel Jeremiah, is under serious consideration for a prominent front-office position with the Jets, sources told Schefter.
Philadelphia Eagles executive Joe Douglas, who has ties to Gase and is a close friend of Jeremiah, could be a leading candidate for the GM job.
The shake-up comes amid reports of internal discord in the Jets organization. ESPN reported during the draft that friction had developed between Maccagnan and Gase because of disagreements over personnel during free agency.
Both men denied that a rift had developed in their relationship, but sources said Johnson was so concerned that he began spending more time at the Jets' facility to monitor the Gase-Maccagnan relationship.
Johnson insisted the relationship had no bearing on his decision, although he didn't deny there was strife.
"No, this had nothing to do with Adam," Johnson said Wednesday. "Look, I want a good give-and-take between the coach and GM. This was not one person or another winning a power struggle. This was completely my decision.
"It was a decision that evolved and it took a while. Maybe I was slow at that decision, but I finally came to it in the last few days."
Sources said that part of the tension between Gase and Maccagnan involved running back Le'Veon Bell and the amount of money it cost to sign him as a free agent. Bell was signed to a four-year, $52.5 million contract that included $35 million guaranteed. The average salary per year ($13.1 million) ranks second among NFL running backs, and Gase felt it was too much, sources said.
Gase, however, has spoken highly of Bell, saying, "It's hard to find guys who are that size with his ability. He's a rare player."
Bell took to Twitter to say he wasn't jumping to conclusions based on rumors.
Johnson defended the team's performance in free agency, claiming he doesn't believe the Jets overpaid for any players.
"No, no, I'm totally good with that," he said. "No remorse at all. I approved them all."
The Jets went 10-6 in their first season under Maccagnan and former coach Todd Bowles in 2015, but the team has struggled through three consecutive losing seasons since. Bowles was fired after last season, and Maccagnan had significant input into the hiring of Gase.
Wednesday marked a stunning about-face for Johnson, who praised Maccagnan and his working relationship with Gase at the owners meetings in late March.
"I think there was pretty good synergy, but it was not everything I was hoping for," Johnson said, when reminded of those March comments. "I made the decision that I want to find a better fit for this building. ... It's more than a talent-evaluation guy. I want a great strategic thinker, a great manager, a communicator, someone who can collaborate well with the building. I'm convinced we're going to find that person."
Johnson said Gase will assist in the GM search, but there will be no changes to the organizational power structure. Both Gase and the new GM will report directly to Johnson, as it was with Maccagnan-Gase and Maccagnan-Bowles.
Johnson empowered Maccagnan to spend more than $125 million in guaranteed money in free agency, then let him use the No. 3 pick in the draft on defensive tackle Quinnen Williams.
The team also has constructed a promising core of young players, including quarterback Sam Darnold, safety Jamal Adams and defensive lineman Leonard Williams.
The Jets are one of eight teams to draft multiple Pro Bowlers in the first round since 2015 (Adams and Williams), but they also missed with second-round picks Christian Hackenberg (2016) and Devin Smith (2015) under Maccagnan. Another first-rounder, linebacker Darron Lee (20th overall in 2016), was traded to the Kansas City Chiefs on Wednesday night, a league source told ESPN's Adam Schefter.
Before joining the Jets, Maccagnan spent 14 seasons with the Houston Texans.
Johnson, running the franchise while older brother Woody Johnson serves as the U.S. ambassador to the United Kingdom, admitted he is learning on the job.
"This is a learning process for me too," he said. "I'm going to get some things right. I'm going to get some things wrong. But I think I'm getting this right."
Jets fire GM Maccagnan, Gase to step in as interim
Adam Schefter reports that Eagles VP of player personnel Joe Douglas is a likely candidate to take over for Mike Maccagnan.
Stephen A. on Jets: 'You just can't make this up'
Stephen A. Smith goes off on the Jets after the team fired GM Mike Maccagnan and replaced him with Adam Gase on an interim basis.