Sarfraz Manzoor on how prejudice works both ways in British