Concern about safety is main reason many Detroiters are not