The City chief was truly enraged by his side's off-colour display at a rainy New York Stadium as a double from Kieran Agard either side of a Alex Revell goal helped to ease Steve Evans' Millers on their way to a comprehensive victory.
The scoreline could have looked worse for Parkinson's men if it wasn't for some smart goalkeeping from Jon McLaughlin in the first period, in particular a stunning reflex save produced by the Bantams 'keeper to push away a close range Revell header.
Parkinson said he felt 'let down' by his players after their lacklustre showing and demanded that any complacency within his squad brought on by the club's promising start to their league campaign is stamped out.
He said: "We were second best all over the pitch.
"It was a performance I haven't seen in a long time from a Bradford City team.
"I feel we have let the supporters down today (Saturday), the players have let me down and they have let the staff down.
"We all have to take a long, hard look at ourselves after that one because it simply wasn't good enough.
"If we'd played in the manner I know we can do but still gone out of the F.A Cup then I'd be only hurting a quarter of what I am now.
"We looked like a team, certainly a few individuals at least, who have just started to believe that they are better then they actually are.
"We have to nip that in the bud straight away.
"Every time we go out on the pitch, we are representing our families, we're representing the supporters and we're fighting for our livelihoods. It didn't look like that to me today though, and I am not accepting that performance.
"We came here last season and got beat (4-0), and I said to the lads before the game that we had to put a performance on this time around for our supporters, who came in their numbers again today.
"Not only that though, since that game here against Rotherham last season, we've played in a lot of big league and cup games. We're played in front of big crowds and under a lot of pressure and dealt with it well.
"That proves to me that we are a team who can deal with big occasions like today, and I expected us to go and show that again.
"There are two types of bravery in football. One is the bravery to stick your head in where it hurts and generally compete and be physical with the opposition. The other is the bravery to want the ball and go looking for it even when things are going wrong on the pitch.
"Both are equally as important as each other, and today we didn't show enough of either."
With City positioned in the Play-Off places after an encouraging start to their first campaign back in League 1, the Bantams still have plenty to build upon as the season progresses despite this latest cup loss.
Parkinson has warned, however, that a general improvement in the whole of his squad, not just those currently in his starting eleven, is needed if City are to continue competing at the right end of the division.
The Bantams chief said: "Sometimes I stand around the dressing room after a defeat and you feel for the lads not in the team, you perhaps think whether they should have been given a go instead.
"There were a few senior players who didn't even make the coach journey today and a couple more didn't make the bench.
"I don't feel like that today though. I don't feel the lads out of the team at the moment have done enough to get in it.
"We need to get back to the mentality where players are thinking 'If I have the shirt and I'm playing, I'm going to work my socks off to keep it for next week and the weeks to come'.
"Equally, I want the players out of the team to think 'I'm going to show in training that I deserve to grab that shirt off him'.
"I simply don't think the lads out of the team at the moment have done enough and I don't think the lads who have come off the bench in recent weeks have done enough.
"As a group, we will all need to do more after today's performance."
On a generally forgettable day for City and their followers, there was one moment to remember when young striker Oliver McBurnie was given his senior debut with eight minutes of normal time remaining.
The 17 year-old forward, who signed a three-year pro deal for City back in August, has scored 17 goals in 10 league and cup games for the youth team already this season.
Parkinson said: "Ollie was on the bench today because he had earned his chance. He wasn't there as a token gesture because it was the F.A Cup.
"He was on there because he has scored goals for the youth team and because he has been outstanding in training. He has shown great enthusiasm when he has trained with the first team recently, and he showed that again today."
Following the disappointment of City's early F.A Cup exit, thoughts for the Bantams now turn to next Sunday's televised clash with Coventry City as Parkinson's men return to League 1 action.
Given the nature of City's defeat at New York Stadium, Parkinson believes some sort of soul-searching is required before City line up against the Sky Blues in front of the Sky Sports cameras.
Parkinson added: "This isn't an over-reaction because of one bad performance because Rotherham played very, very well on the day, but we do have to look at ourselves very closely.
"I have to look at myself as well. We have received plenty of plaudits for last season, and we continue to do so now, but I am not bothered about awards evenings or receiving managerial trophies, I am bothered about the here and now.
"I am bothered about making sure that our terrific supporters get a performance they deserve every week.
"It is rare that we don't perform like this, I must say that, but we cannot accept a performance like that on such a big day for the supporters.
"On a derby day like this in front of a lot of our supporters, it is not a good day not to perform."