Paul Benson put Swindon in front in the 53rd minute and a late brace from Alan Navarro sealed a fantastic victory.
Newly-promoted Charlton were another Championship side to fall as they went down 4-3 on penalties to Orient at The Valley.
Scott Wagstaff gave Charlton a 28th minute lead but Mathieu Baudry equalised in first-half injury time and the visitors held their nerve to win 4-3 in the shootout with Ryan Brunt firing home the decisive kick.
And Millwall also missed out on penalties to League One Crawley Town at the New Den.
Hope Akpan and Nicky Adams twice put the visitors ahead only for the Lions to haul their way back through Darren Ward and Shaun Batt respectively.
The visitors eased through the eventual shootout 4-1 with Crawley keeper Paul Jones the hero as he saved from Darius Henderson and Jack Smith.
Crystal Palace were forced to come from behind to beat Exeter 2-1 at St James's Park.
John O'Flynn put the hosts ahead early on, but Jermaine Easter's penalty and a Kagisho Dikgacoi goal before half-time won it for the Eagles.
Stevenage beat AFC Wimbledon 3-1 and Yeovil won an all-League One clash 3-0 at home to Colchester.
Crisis club Portsmouth crashed 3-0 at Plymouth, who had former Iron Paris Cowan-Hall on the scoresheet, while another newly-relegated side, Coventry, squeaked past League Two Dagenham & Redbridge with Kevin Kilbane's injury-time penalty.
League Two Bristol Rovers took a surprise 26th minute lead at Ipswich through Michael Smith, but the Tractor Boys turned it round with Jason Scotland, Tommy Smith and Aaron Cresswell setting up a 3-1 win.
Burnley were yet another Championship side forced to come from behind as Chris McCann, Charlie Austin and Dean Marney cancelled out an early Chris Shuker effort in a 3-1 win at Port Vale.
There were no such problems for Leicester, who won 4-0 at Torquay with goals from Lloyd Dyer, Ben Marshall, Matthew James and Jamie Vardy, or Peterborough, who beat Southend at London Road by the same score.
Last year's beaten finalists Cardiff City blew an early lead to go down 2-1 at League Two side Northampton, while Birmingham came from a goal down to beat Barnet 5-1.