With 35 minutes to go it seemed that Colchester would be going back to Essex with three points courtesy of a 3-1 lead.
However, Cheltenham's new look team stormed back with three goals to win the match and leave new boss Allen delighted.
"Three points is very important and to hear the fans singing at the end was great," he said.
"I want to get this place buzzing - let's have some more of it. I took all the staff into the dressing room at the end and let all the players who deserve the credit receive the applause.
"It's exciting to be scoring goals. I've never known anything like it. But we do have people right through the team and the squad who can do a job for us."