If we lift restrictions we may have to go 'all the way back'