Player-coach Jason McAteer will take charge of the team for Saturday's visit of Doncaster Rovers.
Little said: "I've thought about this long and hard over recent days and am sure this is right for the Club and for me personally.
"I've really enjoyed the last three seasons here, despite the problems we have had this season.
"The play-offs last year and the quarter final FA Cup games against Millwall stand out. Finally, I would like to wish everyone at the Club every success for the future."