Polly goes back to school

asiadmin | November 4, 2020  | Methodology, Politics | 17 Comments

So it’s back to school for Polly this morning (and yes she will be attending class virtually like her fellow students around the world).  I know many of you are disappointed that  Polly did not hit this one out of the park like she did over 20 elections since 2016.  So are we.  But we have to remember that Polly is an artificial intelligence, and like any intelligence, she needs to be taught, raised, and corrected when she is wrong.  That’s how she learns.  The great thing about Polly is that once she learns something, she never forgets.  She just gets better with every experience.

So what does school look like for Polly?  

On her first day back, once the results are finalized, we will ask Polly to  “show her work” so we can properly analyze what went wrong (before last night, we ourselves did not have the answer to the test so we could not coach her!).  Once we understand where Polly went wrong, we will give her the “answers to the test” and ask her to go back and re-read all those posts and then “re-create” the answers.  Then we will ask her to re-analyze her interpretation of events in light of what really happened and teach herself where she likely went wrong.  She will share these insights with us, her parents, whose job it is to raise and teach her.  

What are some of our early theories of what went wrong? 

One clue came to us early last night when Florida failed to go Democrat.  Polly had not created a separate sample of Cubans – she lumped them in as “Hispanic” and clearly they needed their own sample (Polly predicted Florida would go Democrat at 52.6%).  We have been asked if Polly speaks Spanish – yes she does, she speaks every language! – but clearly we need to factor out these linguistic groups even more.    We need to include more ethnic and regional “factors” for the next election. Cubans, Mexicans and Venezuelans have different contexts.

This is a symptom of a larger challenge: Polly needs to fine tune her ability to uncover issues that are important to specific minority groups and regions.  The smaller the group, the harder it is to uncover.  It is not a problem in the methodology overall, it is an issue of fine-tuning: Amplifying the error to make it easier to uncover.  Finding where she went astray, issue by issue, state by state.  This will take several weeks, so if you want to follow Polly’s journey, stay tuned!

Polly is going to have to do some deep studying to understand something we all know instinctively: humans are complicated.  

Photo by Ivan Aleksic on Unsplash

About these results

Submit a comment

Your email address will not be published. Required fields are marked *

17 thoughts on “Polly goes back to school

  1. Maybe just relying on Twitter isn’t the best solution. Look up the NYTimes article which asked whether FB views were really telling the story. When Ben Shapiro vids get 10 million views and shares while a progressive video gets 1 million, that’s telling you something. Why do you think Cambridge Analytica used Facebook? I’d asked earlier whether Polly was accounting for the millions of conservatives who had moved to Parler instead of Twitter. Or if the deluge of misinformation on local tv and radio and WhatsApp in the Florida market was being accounted for.

    Of course any AI is only as good as the programmers and the quality of the data used. It seems like you had terrible overconfidence in your data source.

  2. So make Polly even more obsessed about intersectional identities is going to make it better? I’d think the exact OPPOSITE lesson is in order… AI is dangerous precisely because it contains The Progressive left bias of its coders. We are more than an ethnic, racial, linguistic, sexual and religious identities. You’ve TWICE proven unable to reliably forecast US federal elections because of that bias who wants to entrust the real money makers – law enforcement, immigration, social policy – to a program so dangerously and wildly wrong so often? How long before someone just pulls the plug on Polly?

    1. Polly has accurately predicted more than 20 elections, the most recent one being in the Canadian province of BC where she produced the most accurate results ever. Polly is programmed to seek out samples that reflect the population and does not, cannot, select biased samples.

      1. I think it succeeds in Canada because it’s based on the issues, Canadians vote issues – we move party easily (most of us) based on these.

        In the US, it’s based on feelings, not reality and not issues. Those feelings are translated into tribe.

        I don’t know how you accommodate that in a model.

  3. The explanation on Hispanics may be part of Polly’s problem but Polly predicted Biden electoral college votes at 372 and he will be lucky – if the gets Nevada, Michigan and Arizona – to get the minimum 270. So Polly was not just a bit wrong. Polly was WAY off base.

    I had so much hope that Polly was right given Polly’s track record and was so very disappointed and put into a panic!

    1. Almost a week after the election and Polly has been proven right in: Michigan, Wisconsin, Arizona, Pennsylvania, and Georgia. She was wrong on Texas, Florida and North Carolina. If Biden had also taken NC that would have resulted EC votes right at the edge of Polly’s stated margin of error – 372 minus 48 for Biden and 166 plus 48 for Trump.

  4. I appreciate your openness. And I think you are brilliant people whose technical flaws are apt to more accurately discerned by yourselves than some armchair critics. I get the notion of how you can build representative samples from sources you know are biased by using other sources of information to draw selectively from your biased source enabling subsequent chronological tracking of various factors. But your work really terrifies me because of your chief scientist explaining how it was his job to oversee the training of your AI as a good parent. From seeing you on camera I am sure you are a good parent company but that leaves very open the possibility of bad parent companies which is terrifying. How can government hope to assure good parenting? Seems mission impossible. We build our technical skills incrementally and inexorably increasingly outstripping our societal skills which may even be regressing with ever poorer discernment of truth. Neither Covid nor the planet’s climate can be fooled by phoney games. I would guess we are going to blazes preceded by a social nightmare from AI.

  5. I agree with your measured response to JR Birch’s over the top comment, but I think his initial point is valid and important. Let’s assume your hypothesis is right, that a crucial miss of Polly’s was failing to account for nonhomogeneity of (Florida? other?) Hispanics. Training her to pick up on nonhomogeneity and to *identify* relevant subgroups is important. But imposing intersectional structures (in the general abstract sense, not just race x gender x … x …) seems at best a stop-gap. Next time, she’ll miss if it’s differences between specific Christian denominations, or plumbers vs electricians, that’s crucial. It’s less about potential political bias in defining *your* microstructures, and more about making Polly “smart” enough to define her own where it matters. (Of course, all of this is problem solving before the “early theories” are confirmed or rejected.)

  6. I was thinking that the Trump’s “Secret Voters” might also have something to do with Polly’s accuracy. Just a thought.

    1. I had thought the really heavy early vote might have thrown Polly. She had Biden taking Texas with about two weeks to go, when early voters were texting their friends about waiting in line to vote Democrat. Then she flipped it to Trump after 8 million Texans had voted, locked in their choice and slowed their texting. A few days before the election she flipped it back to Biden. What happened there?

  7. It would be interesting to examine whether Polly is accurately forecasting the likelihood that a given social media user profile will show up to vote. Did it detect apathy in some racialized communities and zeal in Trump’s base and, if not, how might it better predict voter turnout? Keep up the great work Erin, Ivan and team.

  8. She called the election correctly, if not every state.

    Plus, the national popular vote will like end up around 55/45, as predicted.

  9. If there’s a run-off in for the Senate seat in Georgia, I’d like to see Polly tackle that.

    1. It would also be a start for forecasting the senate and house races some time in the future.

  10. Polly only needs to learn that when she predicts election result about “Trump” she can be wrong and cannot trust anything.

    Back to school, little girl. Lol

  11. A significant proportion of Trump supporters are less well educated than Democrats. They do not use Twitter or social media and remain under-detected in the sample. How can you sample Luddites? It is easy to be wrong if Polly tries to predict E.C. vote totals when there are large states like Florida and Texas that can easily swing either way… much better to predict percentages in each state; then if she is wrong in Florida the EC spread is not off by 58 votes, it’s just a single state that was incorrect.

  12. I found the explanation for the challenges to accuracy very meaningful in the TVO interview with Paiken. Really excellent technical transparency without giving away the family jewels. Polly has really super “parents”. I shall continue to have great interest in both future predictions and post event analyses.

Comments are closed.

How reliable is your market research?

asiadmin | August 26, 2019


Where did the change happen? or, The “P” in Polly stands for Probability

shainen | October 27, 2020


Share This