Ivy Mar 31, 2021 No Comments
Catch the data science weekly news and advancements here on Ivy Blog. From AI forecasts, microbial peptides to AI replacing therapists.
Meng compares data analysis to test the saltiness of a large vat of soup. If the soup is well stirred, all you need is a tiny bit—less than a teaspoon—to tell how salty it is. In data terms, you’re taking a random sample of the soup vat. If the soup isn’t well stirred, you could drink gallons of it and still not know its average saltiness, because the part you didn’t taste might be different from the part you did taste.
Meng isn’t the first to stress the risk of selection bias. His contribution is in quantifying it. He has created what he calls a “data defect index” and has developed a formula that’s simple by the standards of mathematical statistics. Learn more in our future data science weekly news.