Balogun says ‘outside noise’ over Trump’s red card reversal made things ‘difficult’ before World Cup loss