Hayes came off the bench to make his debut for the Bees, but his penalty - from his first touch of the ball - was saved by visiting keeper Marek Stech and the home side were made to suffer by the counter-attacking flair of the Glovers.
Rosler had brought Hayes and Harry Forrester off the bench to go for the win, in a tactic he admitted backfired badly.
“It was too offensive too early and I take the blame for that. It was too much too soon and we paid the price, he said.
“I take responsibility but we will regroup and have a chance against Crewe on Saturday to put things right."
Two goals form James Hayter, coupled with a Tony Craig own goal, sealed a deserved three points for the Somerset side.
It means they will be full of confidence ahead of Saturday's trip to Glanford Park.
“I think the penalty save was the turning point but we showed great resilience throughout and real composure at the end to pick our counter attacks," said Gary Johnson, the Yeovil manager.
“We knew pace would be the key and we have it in abundance. We didn’t fill the bench with subs but it’s quality not quantity.
“It’s a good win here because Brentford are looking to be one of the sides up there at the end of the season so to come and finish so clinically was superb – that’s what you get from James Hayter.”