The suspect was spotted inside St. John's Church on Main Street on May 15.
New video was released over the weekend that police say shows the suspect walking around and stealing various items before leaving.
Detectives believe he has stolen from the church at least five times since last fall -- and it mostly happens when people are there.
The suspect is believed to have stolen multiple cellphones and more than $500 on several locations.
"It's very brazen, it's scary because you don't know what he could do if confronted," Rev. Dr. James Wheeler said.
His larcenous behavior since he first struck and stole Father Wheeler's phone has some people wondering if nothing is sacred anymore.
"He came up here and must have heard we play Bingo and thought that we had money up here," executive director Betty McOsker said.
McOsker said she came face to face with the suspect on the second floor. It is believed he later pried open a file cabinet to steal cash and checks.
The suspect's brazen nature has forced church officials to take extra precautions like adding security cameras and locking most doors.
Authorities hope someone is able to identify the suspect.
* More Connecticut news
* Send us a news tip
* Download the abc7NY app for breaking news alerts
*Follow us on YouTube