Fears that "inappropriate" homeless refuge would bring troub