After a goalless ninety minutes and extra time period - both of which were dominated by the Bantams - Parkinson's men lost their third round replay with the Shakers 4-2 in the shoot-out following saved spot-kicks from the returning Steven Davies and Devante Cole.
Davies and Cole had both seen chances come and go during the game as City looked to press home the advantage following Bury skipper Nathan Cameron's dismissal just before the break for two bookings in the space of just three minutes.
Davies and fellow substitute Josh Morris both hit the woodwork, while Cole fired narrowly wide in the final moments of extra time.
Rory McArdle and Billy Clarke also went close for the Bantams before their eventual defeat on penalties in front of the T.L Dallas Stand end.
The loss left City in the strange position of having to exit the competition despite not conceded a single goal in their five F.A Cup ties this season.
That was no consolation for Parkinson though, who was more interested in events at the other end of the pitch.
Parkinson said: "Even with 11 vs 11, we've had chances that we just couldn't take.
"Obviously when they went down to ten men, they sat back and did a very disciplined and organised job against us.
"At times we moved the ball a bit too slowly and allowed them the opportunity to get back into their shape but we still had enough chances to win the game in normal time or extra time.
"We've hit the post and the bar, and we were all just waiting for that elusive goal to come to settle everybody down but unfortunately it didn't come.
"We have had a run now in recent games, not just this one, where we are creating a lot of good scoring chances but we are not making the opposition 'keeper work hard enough.
"No-one hid away from the ball, everybody kept showing for it, but it can be difficult to get that breakthrough.
"We've had some terrific nights in cup competitions down the years but this one is obviously very disappointing."
With City currently four points off the Play-Offs and with at least two games in hand on virtually all of the other teams around them in the table, Parkinson knows the Bantams must quickly move on from the frustration of their cup exit.
Parkinson added: "We've got to take this result on the chin and not let this disappoint effect us too much. We still have a great chance in the league but we just have to find those missing ingredients from somewhere.
"We will look to dust ourselves down and go into the weekend's game (away at Port Vale) in a confident frame of mind.
"We have to get more out of the players we have currently in the building and keep giving them belief.
"We are getting into some great scoring positions and not taking them but hopefully that will change on Saturday."
One positive from the night was the appearance of the fit-again Davies, who came on as a substitute towards the end of normal time following three months out with a knee injury.
Parkinson continued: "He's hit the bar and gone close on a couple of occasions. It was great to get him back out on the pitch and I'm sure he'll get the benefit from those match minutes."