In wet and deary conditions at Gigg Lane, City fought their way into Monday's fourth round draw with a committed display against the Shakers.
The Bantams could have even secured the win outright with Gary Liddle and James Hanson going close with two promising scoring opportunities in the second period.
Parkin said: "This was a tie with two hard-working sides going toe-to-toe against each other.
"It was heavy pitch and there wasn't too much free-flowing football involved but the lads put in a really honest performance.
"The efforts from the group was first class and it was really important that we got back to being defensively sound.
"The players reacted really well from the disappointment of last week (vs Gillingham) and it would have just been nice if we could have taken one of the chances towards the end.
"Gary (Liddle) has been scoring this season and I think we were all up on the bench when he has put the header in. Rory McArdle has gone close as well from the corner.
"Big Hans (James Hanson) has had to stretch for his chance at the end but unfortunately he's put it over.
"In fairness, Bury had a couple of decent openings as well. Ben (Williams) has made an important stop in the first period.
"Overall, I think we probably shaded it and deserved the win but we're still in the hat which is the main thing."
With nearly 3,500 supporters at Bury, City actually had slightly more fans in the ground for the tie than their hosts.
To fit all of the Bantams followers in the stadium, City were given both stands behind the goals at Gigg Lane.
Parkin added: "We have to go on record as saying a massive thank you to the supporters again.
"On a cold, wet, miserable day, they have got behind us in their numbers.
"To see both ends full of your own supporters for an away match is a unique situation - I've never had it in my career certainly.
"It is yet another example of the tremendous support we receive from our great fans.
"We now need that same support at home. We have games in hand and this cup replay to fit in - and the fans' support is going to be vital to push us on.
"There is no place better for us to play our football in than the Coral Windows Stadium when it is rocking."