Dec 15, 2015 07:05 PM
http://blog.mrtz.org/2015/12/14/adaptive...lysis.html
EXCERPT: I just returned from NIPS 2015, a joyful week of corporate parties featuring deep learning themed cocktails, money talk, recruiting events, and some scientific activities on the side. In the latter category, I co-organized a workshop on adaptive data analysis with Vitaly Feldman, Aaron Roth and Adam Smith. Our workshop responds to an increasingly pressing issue in machine learning and statistics. The classic view of these fields has it that we choose our method independently of the data to which we intend to apply the method. For example, a hypothesis test must be fixed in advance before we see the data. I sometimes call this static or confirmatory data analysis. You need to know exactly what you want to do before you collect the data and run your experiment. In contrast, in practice we typically choose our methods as a function of the data to which we apply them. In other words, we adapt our method to the data....
EXCERPT: I just returned from NIPS 2015, a joyful week of corporate parties featuring deep learning themed cocktails, money talk, recruiting events, and some scientific activities on the side. In the latter category, I co-organized a workshop on adaptive data analysis with Vitaly Feldman, Aaron Roth and Adam Smith. Our workshop responds to an increasingly pressing issue in machine learning and statistics. The classic view of these fields has it that we choose our method independently of the data to which we intend to apply the method. For example, a hypothesis test must be fixed in advance before we see the data. I sometimes call this static or confirmatory data analysis. You need to know exactly what you want to do before you collect the data and run your experiment. In contrast, in practice we typically choose our methods as a function of the data to which we apply them. In other words, we adapt our method to the data....