Yorkshire Carnegie head coach Gary Mercer could not hide his frustration despite his side winning 32-28 at Moseley.
The visitors had second row Nathan Hannay sent off in the second half for serious foul play at the same time as prop Ben Harris was sin-binned.
Young second-rower Liam Edwards was shown a yellow card in the first half and centre James Fitzpatrick missed the last seven minutes after being sent to the bin on his debut.
Yorkshire looked to be in full control when they led 26-12 midway through the second half but then contrived to set up a frantic finish when Moseley used their numerical advantage to drive over for a couple of tries from close range.
Carnegie needed a last second penalty from Harry Leonard to seal the hard-fought win.
Speaking after the game, Mercer said: “We cannot let this situation happen again this season.
“We will review this game on Monday and put it to bed but it was disappointing how we allowed them back into the game in the last quarter through our own mistakes.”
Yorkshire crossed for the only try in the first half after a great piece of vision from scrum half Chris Pilgrim, he was supported by Oli Goss, Chris Walker and Chris Jones to keep the ball alive.
Glyn Hughes spotted a gap out wide and kicked to the corner for David McIlwaine to collect on the first bounce to score.
Leonard missed the conversion but kicked three first half penalties to give his side a 14-12 lead at the break.
The first half had mainly been a kicking duel between the two fly-halves but the second half was a much more fiery affair between both sets of forwards.
Chris Walker emerged from a mass of bodies after a 10-man drive near the Moseley line to extend Yorkshire’s lead and the visitors were eyeing a try bonus point when Edwards supported a break from near half way to score between the posts.
Leonard added the conversion to put his side 26-12 in front.
Then came the double carding for Yorkshire with replacements Harris and Hannay leaving the game.
Yorkshire got a penalty straight away despite the numerical disadvantage but from then on it was a backs against the wall performance to secure the win.
Moseley crossed for a converted try with eight minutes to go and then scored out wide with two minutes remaining from a line out drive.
Youngster Sam Olver had the chance to give the home side the lead but his touchline conversion attempt went wide and Yorkshire maintained their narrow one point lead.
The visitors stood firm and were then awarded a penalty at a scrum on the Moseley 22 metre line and Leonard kicked the goal to end the game and secure the 32-28 win.
Moseley: Carter (Penn, 70), Cheshire, King, Owen, Armstrong, Ryan (Olver, 64), Brown, Voisey (Tolmie, 61), Caves (Hobbs-Awoyemi, 61), Gardiner (Thorp, 61), Charlton, Powell (Woolford, 69), Siggery (Brazier, 64), Carpenter, Mason
Yorkshire Carnegie: Hughes, Goss, Clarke (Fitzpatrick, 65), Lucock, McIlwaine, Leonard, Pilgrim, Imiolek (Harris, 60), Nilsen (Hayes, 60), Hooper (Tideswell, 60), Hannay, Edwards (Casson, 69), Jones, Walker, Burrows. Replacement not used: Dudman
Referee: Simon Harding (RFU)