Comment by shagie

It failed in 2024 when it didn't weight its model to handle differences in demographics vs the sampled population. She accounted for age, sex, and district - but not education, party identification, or past voting behavior. (The specifics of why the model failed is up for debate, but it was the same model that had been used for many years and was reasonably accurate).

From 2023, here's a description of her polling methodology:

https://www.iowapbs.org/shows/iowapress/episode/10530/j-ann-...

    But it's difficult, even I know where to look and a lot of the methodology statements get hidden or are inadequate to make that assumption. The main thing I'm going to tell you that distinguishes us from most other polling, not all, there's some very good polls out there that do what I do, which is we don't make assumptions about what that future electorate is going to look like. We let the data -- we try to get out of the way of the data showing us. So, you'll look and they'll say how they weighted the data, this that and the other, and then they include in that who the likely demographics of the 2024 caucus are likely to be. And I go, really? How do you know what that is likely to be? Or they're weighting it based on who they voted for in the 2020 election or other factors like that, or maybe they're only talking to people who have caucused before. And this approach I call polling backward, that if you're creating a model of the future, what are you going to base it on? You're going to base it on what happened in the past. And I know that things can change. So, I want to have my fingers off the data, my dirty fingers I try to keep them off the data, so it will reveal to me how that's going to go.

From 2024 https://www.iowapbs.org/shows/iowapress/episode/11885/j-ann-...

    But for everybody else we keep them on the phone to get age, sex, what county they live in. That is going to give us some geography so that we can weight all of those people, I think there were 1,100 and some in our final poll, to look like the state of Iowa because that we know. And then we extract from that those people who made our likely voter screen and interviewed them. So, if they are proportionately more women who pass that screen, they are going to show up here but they are weighted at the statewide level. It's a little geeky. But one of the things I've been thinking about is with caucuses we take you if you definitely or probably will participate and maybe our screen was too tight. I don't know, there's no way to undo it and sort of check, well what would the probably people have said, because we very politely terminated the people who just said they would probably vote. But that has been our system since 2008.