The midfielder was presented with an easy chance from eight yards after a mix-up between Millers defender Gregor Robertson and keeper Gary Montgomery on 73 minutes.
The goal settled a dour battle at Millmoor, in which both teams struggled to play constructive football in the very blustery conditions.
Honours were even in an uneventful first half in which both teams battled for supremacy. Montgomery made a full length save from Marc Richards and again from a shot on the turn from former Iron striker Paul Hayes.
At the other end Martin Butler was inches away from converting a poor back pass and a ten-minute purple patch for Rotherham saw midfielder John Mullin have a 20-yarder blocked and leading scorer Deon Burton thrash a bicycle kick wide.
The best chance of the half though fell to the visitors when neat passing on the edge of the box put Hayes clear, but he could only screw wide from six yards.
Rotherham had the wind at their backs in the second half but had a let-off when good work by Jacob Burns put Chris Shuker in space five yards out, but he shot wide of goal.
Rotherham captain Colin Murdock placed a header narrowly over from a Grant Leadbitter corner as the match opened up and play switched from end to end.
Another good effort by Andy Monkhouse had Nick Colgan in the Barnsley goal stretching, but the Millers were rocked back on their heels when Devaney, on as a sub for only ten minutes, pounced for the winner.
Rotherham launched a late rally but failed to threaten and are now without a win in nine league games.