Blues progressed to the last 32 of the competition with a 3-2 victory against Blyth Spartans on Saturday whilst Albion hammered non-league Gateshead 7-0 at the Hawthorns.
The game at St Andrews will take place on weekend of 24/25th January.
FA Cup fourth round draw in full:
Southampton or Ipswich v Crystal Palace
Cambridge United v Manchester United
Blackburn Rovers v Swansea City
Chelsea v Millwall or Bradford City
Derby County v Scunthorpe United or Chesterfield
Preston North End v Sheffield United
Blues v West Brom
Aston Villa v AFC Bournemouth
Cardiff City v Reading
AFC Wimbledon or Liverpool v Bolton Wanderers
Burnley or Tottenham Hotspur v Leicester City
Brighton v Arsenal
Rochdale v Stoke City
Sunderland v Fulham or Wolves
Doncaster Rovers or Bristol City v Everton v West Ham
Manchester City v Middlesbrough.