Town hall to be demolished as usage has 'significantly decli