He is expected to be offered the job full-time this week, but could turn down the position after being targeted for abuse by a section of the home support in Saturday's 1-1 draw with Shrewsbury.
Palmer said: "I am disappointed with the crowd, and will have a chat with the chairman and my family.
"If I am not going to be allowed to do my job properly, I will just walk away.
"I feel I have won more points in my first 18 matches than Keith Curle did."