And with Ali Gibb red carded shortly after the restart, it appeared the Iron were on course for round two.
"I'm very disappointed, I'm absolutely seething," said Adkins.
"We allowed them three attempts at goal in the second-half and they scored from two of them.
"If we're going to stay in the Championship we can't allow that to happen.
"We wanted the result but we're very disappointed. We didn't exploit them when they lost a man.
"Credit to Hartlepool because they denied us space, but I hate losing. We're out of the cup and we could have got a good side in the next round."
Pools boss Danny Wilson hailed debutant keeper Jan Budtz, after his string of first-half saves kept his side in the tie.
"We were under pressure a bit but Budtz made some fantastic saves to keep us in it," he beamed.
"I thought some of our decisions were poor and that is how they got their goal, through a poor decision that we made.
"We were lucky to reach half time on level terms but we turned around and we showed the great character that we need. I thought the sending off seemed to gee the boys up a bit.
"To come here with the success that Scunthorpe have had was a great test of character."
