Nicola Cortese, chairman of the South Coast club, is believed to want to replace the former Scunthorpe boss with a higher-profile name.
This is supposedly to allow the club to attract bigger name players, but the experience of clubs like QPR - who are bottom, despite sacking Neil Warnock in January to bring in Mark Hughes for similar reasons - shows this can easily backfire.
And Le Tissier reckons Adkins, who took the Saints from the League One relegation zone to the top flight in just two seasons, deserves more time to keep the club in the top flight.
"With what he has done for the us over the past few years, I think he deserves a fair crack at it," he said.
"I think it would be incredibly harsh if he was to lose his job at this point of the season.
"It has been tricky, but it was always going to be tough with the fixtures we had. We have acquitted ourselves quite well without perhaps picking up the points that some of our play has deserved.
"I don't think we deserved to lose to Manchester United at home and, while four points is not a great total, we're still not in the bottom three."
Le Tissier criticised Cortese, believing the Italian deserves little credit for the club's rise up the divisions.
Cortese played an integral role in the now-deceased Markus Liebherr's takeover of the debt-ridden south coast club three years ago - a time in which Le Tissier was part of a rival bid by Pinnacle Group.
"I suppose the only real story will probably come out when he leaves the football club. He is not a very nice human being. It's as simple as that," added Le Tissier.
"I think a lot of the stuff will come out about him when he leaves the football club and only when he leaves.
"He has a lot of hold over a lot of people, and while the club is doing well on the pitch he will get away with it.
"When it's not quite going so well and the spotlight starts falling on other areas off the football pitch, then that's when he'll come under a little bit of pressure."