Ferdinand Mount: 'You couldn't not be frightened of Margaret