Sifford, who recently had suffered a stroke, was 92. Details of his death and funeral arrangements were not immediately available.
"His love of golf, despite many barriers in his path, strengthened him as he became a beacon for diversity in our game," PGA of America president Derek Sprague said. "By his courage, Dr. Sifford inspired others to follow their dreams. Golf was fortunate to have had this exceptional American in our midst."
A proud man who endured racial taunts and threats, Sifford set modest goals and achieved more than he had imagined.
Sifford challenged the Caucasian-only clause, and the PGA rescinded it in 1961. He won the Greater Hartford Open in 1967 and the Los Angeles Open in 1969. He also won the 1975 Senior PGA Championship, five years before the Champions Tour was created.
His career was fully recognized in 2004 when he became the first African-American inducted into the World Golf Hall of Fame. This past November, President Barack Obama presented him with the Presidential Medal of Freedom. Jack Nicklaus and Arnold Palmer are the only other golfers who received that honor.
"Charlie won tournaments, but more important, he broke a barrier," Nicklaus once said. "I think what Charlie Sifford has brought to his game has been monumental."
The one goal that eluded him was a chance to play in the Masters, which did not invite its first black player until Lee Elder in 1975. Sifford remained bitter, although the pain was eased when Tiger Woods won the first of his four green jackets in 1997.
Woods often has said he would not have played golf if not for Sifford and other black pioneers.
"It's not an exaggeration to say that without Charlie, and the other pioneers who fought to play, I may not be playing golf," Woods said in an email to The Associated Press late last year. "My pop likely wouldn't have picked up the sport, and maybe I wouldn't have either."
Woods took to Twitter on Wednesday to mourn the loss.
The road was never easy.
Sifford was born on June 22, 1922, in Charlotte, North Carolina. He worked as a caddie and dominated the all-black United Golfers Association, winning five straight national titles. He longed to play against the best players, only to run into the same barrier that Teddy Rhodes and Bill Spiller faced -- the Caucasian-only clause.
In his autobiography, "Just Let Me Play," Sifford told of meeting Jackie Robinson in California about the time Robinson was trying to break the color barrier in baseball.
"He asked me if I was a quitter," Sifford wrote. "I told him no. He said, 'If you're not a quitter, you're probably going to experience some things that will make you want to quit.'"
During the 1952 Phoenix Open, one of the few events in which black golfers could play, Sifford found human feces in the cup when he got to the first green. He received death threats over the phone at the 1961 Greater Greensboro Open and heard racial slurs as he walked the fairways. He finished fourth, and didn't quit.
He was beloved by some of golf's biggest stars, including Nicklaus and Palmer.
During his Hall of Fame induction ceremony, Sifford told of his first meeting with Palmer. They were playing in the 1955 Canadian Open, and Sifford opened with a 63 to lead Palmer by one shot. He recalled Palmer standing in front of the scoreboard saying, "Charlie Sifford? How the hell did he shoot 63?"
"I'm standing right behind him," Sifford said. "I said, 'The same damn way you shot 64.' That's how we met."
Sifford also received an honorary doctorate degree from the University of St. Andrews in Scotland for his career as a pioneer.
He often attended the Bridgestone Invitational at Firestone, not far from his home in Ohio. During an interview with the AP in 2000, Sifford said he was proud of the role he played in making the PGA Tour accessible to African-Americans.
"If I hadn't acted like a professional when they sent me out, if I did something crazy, there would never be any blacks playing," he said. "I toughed it out. I'm proud of it. All those people were against me, and I'm looking down on them now."
The White House released a statement from President Obama on Wednesday.
"Charlie was the first African-American to earn a PGA tour card -- often facing indignity and injustice even as he faced the competition. Though his best golf was already behind him, he proved that he belonged, winning twice on tour and blazing a trail for future generations of athletes in America. I was honored to award Charlie the Presidential Medal of Freedom last year -- for altering the course of the sport and the country he loved. Our thoughts and prayers are with his family, his friends, and his fans."
Information from The Associated Press was used in this report.
Charlie Sifford Dies At 92
Charlie Sifford, the first African-American to earn what is now called a PGA Tour card, died at the age of 92.