The Iron took an early lead at Fratton Park through Leon Clarke, but Gabor Gyepes quickly levelled before Wes Thomas struck a late winner.
"The result is difficult to accept because I thought we dominated the majority of the game," said Knill.
"We had opportunities to get a second goal and we did not take them. I thought it was a really good performance because this is a tough place to come.
"Our performance was as good as we have played this season and we will take the positive out of that.
"I think it is three points dropped. We are progressing, our displays are so much better than they were earlier in the season.
"We got punished right at the end to ruin a good performance. If you are not focused for the entire game then that is what happens. We made some poor decisions in good positions and now we have nothing to show for it."
Michael Appleton, the Pompey boss, paid tribute to Thomas - who signed on loan from Bournemouth on Friday.
"Wes is a good footballer. I had seen him three or four times last season and he was very good, complementing Izale McLeod in attack," he said.
"I an so pleased to have got that first win under our belts and we need to go to Yeovil next week and build on what we have achieved.
"I said to the players you have to remember the difference in feeling between winning and losing. You have to want to win and it has to become a habit.’’