The former Indiana governor flew in so he could watch Peyton Manning's jersey retirement ceremony on Sunday. But Pence didn't stick around long.
Pence said on Twitter: "I left today's Colts game because POTUS and I will not dignify any event that disrespects our soldiers, our Flag, or our National Anthem."
The White House also issued a statement from Pence, in which he says Americans should rally around the flag. Pence said: "I don't think it's too much to ask NFL players to respect the Flag and our National Anthem."
Pence is a noted sports fan and it's second major event he's attended in his home state since taking office in January. He also attended May's Indianapolis 500, a family tradition.
Manning will become the first Indianapolis-era player in Colts history to have his number retired and will also be inducted into the team's Ring of Honor.
Pence flew in on Saturday after a statue of Manning was unveiled, an event attended by a number of luminaries including NFL Commissioner Roger Goodell.
Instead, Pence spent most of Saturday honoring victims of the Las Vegas shooting before returning to his home state.