Blues boss Gary Rowett had refused to do the post-match press conference after being sent to the stands for arguing with the referee in the second half of the game.
"Gary was obviously very frustrated, there was a throw-in that had gone out of play next to us which we felt was clearly our ball but the assistant referee gave it the other way," Summerfield said.
"He did not swear but he made the point to the referee that the assistant referee and linesman were far away but he threw the ball down and that was enough to see him get sent to the stands.
"We are not going to blame the referee for the defeat, they have only got a few seconds to make a decision but we didn't think it was a penalty and that killed the game off.
The Blues display against the Bluebirds had also bitterly disappointed Summerfield, who added: "This will be a kick up the backside for us, the players will really have to roll their sleeves up now for the rest of the season because it will have a big impact on what happens next year.
"That performance was not up to the standards that we have set ourselves this season.
"Our passing and general play was not right, we tried to change the system a little bit and maybe that had a big effect on the players.
"But the players have to take some responsibility, they should be frustrated with that performance and we have made sure they are aware of that."