The on-loan Blues keeper seemed to go to ground extremely early as Clarets striker Steven Fletcher aimed towards goal from just inside the penalty box - a mistake which led to Burnley taking a 53rd minute lead.
The Clarets doubled their lead just nine minutes later through Andre Bikey before Seb Larsson scored direct from a free-kick in the dying seconds for Blues.
"Big Joe has held his hand up," Blues boss Alex McLeish told the press after the match.
"He feels he should have done better and I think he is an honest lad.
"Up to then I thought it was anyone's game and the first goal was always going to be important."