I don't trust any quoted statistics by default because I've (anecdotally) realised that pretty much every study is sloppily done or misinterpreted when quoted. The only time I trust a statistical analysis is when I have personally looked into it and seen that they did things right. I'm not even a stats pro or anything, the bar is in hell and any AI model, when given sufficient data and reasonably good prompting can debunk things pretty well.The other day, I spoke to a data scientist who worked on surveys. He spoke about how a reputed survey firm they hired that's been around for over half a century that can't do the most basic things right. The kind of problems are unfathomably bad—the class of errors is stuff like asking the questions in a language that's not of a specific town, finishing the questions in impossibly fast times, not actually going to any household, etc.In this case, this person had to throw close to half of the data points away because they were suspect. This just happened to be a person who cared enough to look into the data, no one in their organisation was particularly asking to do these checks. If it were a person who cared a little less and didn't add friction to their own work because of it, we would have a survey were millions of people would have been essentially lied to.Most analyses are based on bad data, based on bad assumptions, processed in bad ways to give a totally bad result in ways that cannot even cancel each other out.Actually let me pull out the classic:However, I am pretty excited about AI being able to help me interpret and understand data and studies better. If used correctly, it's great for being able to do the unreasonable effort to wade through sloppy analyses, and I find myself doing that a lot more often. In the past, I would just kind of assume a default "probably false" and move on, which often leads to missed opportunities to actually learn things about the world.