Ministers out to 'control damage' after PM's diatribe agains