Fears building in 'fragile gap' between St Albans and Hatfie