The two loanee's have now returned to their parent club's Belchatow and AZ Alkmaar respectively as manager Alex McLeish has decided aginst offering the pair long term deals.
Costly joined Blues in January and made just 3 starts and 5 substitute appearance's during his time at St Andrews.
Agustien joined on loan at the beginning of last season and made 15 starts and 5 substitute appearance's but injury forced him to miss the last four months of the promotion campaign.
Winger Scott Sinclair has also now returned to Chelsea after a five month loan spell at St Andrews.
"Carlos Costly and Kemy Agustien have gone," McLeish told the Birmingham Mail.
"Scott Sinclair returned to Chelsea.
"Kemy was upset, he really loves it here. But, unfortunately, although I know there's a better footballer in him than we've seen, I just couldn't commit myself to bringing him in for the money asked."