Peter Jackson's team made the perfect start as Chris Brandon's low 20-yard shot put them ahead in the second minute.
Jonathan Worthington doubled Huddersfield's lead with a speculative shot five minutes before the break and the Terriers held on despite Michael McIndoe's 64th-minute spot-kick reply for Rovers.
Swansea climbed from sixth to second thanks to a 3-1 victory over Barnsley at the New Stadium.
Kevin McLeod opened the scoring for the Swans (14) but the match was level at the break when former Leeds midfielder Jacob Burns equalised for the Tykes.
Lee Trundle was then fouled by Burns and recovered to rifle in a stunning free-kick in the 58th minute, and McLeod's second in the last minute made sure for the Welsh side.
Brentford's 1-1 draw at home to Gillingham meant they dropped from first to third.
The Gills went ahead at Griffin Park through Neil Harris' strike in first-half injury time, but Ryan Peters' header with 15 minutes to go salvaged a point for Martin Allen's men.
Rotherham moved from seventh to fourth with a 1-0 win at former boss Ronnie Moore's current club Oldham as Deon Burton scored his sixth goal for the Millers with 10 minutes to go.
A last-gasp goal by Adam Boyd denied Scunthorpe second place as they had to settle for a 3-3 draw at Victoria Park.
Billy Sharp's brace gave the Iron a 2-1 lead after Michael Proctor's sixth-minute opener for Pools, and Andrew Keogh made it 3-1 (65).
Eifion Williams pulled one back six minutes later, before Boyd's late equaliser.
Port Vale dropped to fifth with a goalless draw at MK Dons, while at the bottom Yeovil remained winless after a 3-1 defeat to fellow early-season strugglers Chesterfield.
Luciano Alvarez gave the Glovers the lead but Paul Hall equalised (44) and a double from Adam Smith earned victory for the Spireites.
Veteran striker Shaun Goater fired two goals as Southend saw off Colchester 3-1 in an all-Essex noon showdown at Roots Hall.
The former Manchester City forward gave the Shrimpers a third-minute lead which was cancelled out by Sam Stockley (19).
Mitchell Cole restored the home side's lead and Goater lobbed in a third with 12 minutes to go.
Simon Wiles headed the winner in Blackpool's 1-0 win at Bradford with six minutes left, while Chris Greenacre's 58th-minute effort helped Tranmere past Swindon.
Bournemouth and Walsall shared the points in a goalless draw at Dean Court.