The Championship side are looking for a new manager after parting company with Dowie last week, but Ferguson does not believe he is in the frame to take on the Loftus Road role.
He insisted: "No one's going anywhere. We have a plan here. The first part of the plan was completed last season when we clinched promotion and I know we have a side capable of challenging for promotion this year.
"There's no reason for any of my players or myself leaving. It's nonsense and I'm just concentrating on Peterborough United. My players have been linked with a number of clubs in recent weeks and it's just speculation. We don't want to let any of our players leave this football club."