Asked by ESPN's Andy Katz why he is resigning, Brown did not offer details, saying he first wanted to discuss his decision with his players and their parents.
"I've got nothing else I can say right now," Brown said.
Brown resigned because he wanted a long-term deal, according to ESPN's Jeff Goodman and first reported by CBS Sports.
SMU athletic director Rick Hart said the school had offered Brown a contract that ran through 2020, but sources told Goodman that Brown wasn't satisfied with the terms of the deal and wanted a longer commitment.
Brown had one season left on his contract.
"Coach Brown was able to accomplish so much in his four years on the Hilltop, including leading us to our first conference title since 1993," Hart said in a statement. "He has left his mark on SMU basketball, and we are thankful for what he's done."
Tim Jankovich, who was hired as the coach-in-waiting under Brown in 2012, will take over as SMU coach. Jankovich has previously been the head coach at Illinois State and North Texas.
"As Director of Athletics, I'm also thankful to the search committee that brought us Coach Brown and for their plan that will allow us such a smooth transition to Tim Jankovich," Hart said. "I have the utmost confidence in Tim based on both his coaching résumé as well as my time working with him here on the Hilltop. I know he will keep our program among the nation's elite, and I look forward to the seasons ahead."
Brown, 75, returned to the college ranks four seasons ago after making numerous stops in the NBA. His Mustangs program underwent an NCAA investigation last year centered on academic improprieties involving former player Keith Frazier, resulting in a self-imposed postseason ban.
The Mustangs were 25-5 last season and 13-5 in the American Athletic Conference. They were the final unbeaten team in the country, winning their first 18 games before falling at Temple in late January.
Jankovich was the acting head coach during Brown's nine-game suspension to open last season.
Brown was 94-39 at SMU. His .707 winning percentage was the best for any Mustangs coach who spent multiple seasons at the school, according to ESPN Stats & Information.
Brown has coached the Denver Nuggets, New Jersey Nets, San Antonio Spurs, Los Angeles Clippers, Indiana Pacers, Philadelphia 76ers, Detroit Pistons, New York Knicks and Charlotte Bobcats. He also coached the Nuggets and Carolina Cougars in the ABA and won the 2004 NBA title with the Pistons.
He also coached at Kansas and UCLA. He led the Jayhawks to the 1988 NCAA title, but the program was banned from postseason play the following season and placed on probation for three years because of recruiting violations. UCLA also incurred a postseason ban and two years of probation for using ineligible players while Brown was coach.
The Associated Press contributed to this report.