Centre-back Foster was replaced at half-time in Easter Monday's home loss to Swindon with a calf injury.
And left-back Williams followed him off the pitch shortly afterwards, suffering from an ankle problem.
Michael Rose will deputise for Williams, while the absence of Foster should see Richie Hinds drop back into defence.
This will create a gap in midfield - with Matty Sparrow likely to be prefered to Richie Ryan.
Rotherham go into the game without midfielder Michael Keane, who serves the second of a five-game ban.
Gregor Robertson will continue to deputise, as he must do for the remainder of the season, with Scott Minto out due to an Achilles injury.