Understanding the potential of deep learning in physical sec