As Bartlett and his coauthors argue, deep neural networks display at least three properties that might be surprising to most practitioners with a background in statistics or even in classical machine learning (ML):