Adkins and his assistant Andy Crosby agreed terms to replace Alan Pardew at St Mary's, but the two clubs did not agree compensation for the Iron to release the pair from their contracts.
Wharton said that Adkins was told by Southampton on Wednesday he should resign - presumably in an attempt to avoid paying a fee.
"Nigel was not prepared to do that. He wanted to walk out of Glanford Park with his head held high and everything agreed," he told the Scunthorpe Telegraph.
"I was not prepared to roll over and let Nigel leave for nothing. I wanted some compensation for what I felt he was worth. It was only a modest amount.
"The Southampton people have been very difficult to deal with. They are probably not the right sort of people for Nigel to be working with."
Events moved on quickly on Thursday morning, with the Saints withdrawing their offer after failing to agree on a fee for their services.
"After failure to agree a compensation package, it appears Southampton's offer to manager Nigel Adkins and assistant Andy Crosby has been withdrawn," read a statement from Scunthorpe United.
Southampton denied the claim they did not want to pay compensation, and said their search for a manager was ongoing. They claimed to be speaking to several "potential candidates".
"The club has made no ultimatums, has not made any approaches without appropriate permissions, and is fully aware that appropriate compensation payments become due where the candidate managers are currently under contract," they said.
Adkins took training as normal on Thursday, and an hour-long meeting with Wharton and Crosby at Glanford Park.
Wharton did not comment after the meeting and Adkins was expected to speak to the media at 3.45pm.
Are you happy Adkins appears to be staying? Tell us in our latest site poll