What happens if you present 500 people with an argument that AI is risky?

Recently, Nathan Young and I wrote about arguments for AI risk and put them on the AI Impacts wiki. In the process, we ran a casual little survey of the American public regarding how they feel about the arguments, initially (if I recall) just because we were curious whether the arguments we found least compelling would also fail to compel a wide variety of people. 

The results were very confusing, so we ended up thinking more about this than initially intended and running four iterations total. This is still a small and scrappy poll to satisfy our own understanding, and doesn’t involve careful analysis or error checking. But I’d like to share a few interesting things we found. Perhaps someone else wants to look at our data more carefully, or run more careful surveys about parts of it. 

In total we surveyed around 570 people across 4 different polls, with 500 in the main one. The basic structure was:

1. p(doom): “If humanity develops very advanced AI technology, how likely do you think it is that this causes humanity to go extinct or be substantially disempowered?” Responses had to be given in a text box, a slider, or with buttons showing ranges

2. (Present them with one of eleven arguments, one a ‘control’)

3. “Do you understand this argument?”

4. “What did you think of this argument?”

5. “How compelling did you find this argument, on a scale of 1-5?”

6. p(doom) again

7. Do you have any further thoughts about this that you'd like to share?

Interesting things:

• In the first survey, participants were much more likely to move their probabilities downward than upward, often while saying they found the argument fairly compelling. This is a big part of what initially confused us. We now think this is because each argument had counterarguments listed under it. Evidence in support of this: in the second and fourth rounds we cut the counterarguments and probabilities went overall upward. When included, three times as many participants moved their probabilities downward as upward (21 vs 7, with 12 unmoved). 

• In the big round (without counterarguments), arguments pushed people upward slightly more: 20% move upward and 15% move downward overall (and 65% say the same). On average, p(doom) increased by about 1.3% (for non-control arguments, treating button inputs as something like the geometric mean of their ranges).

• But the input type seemed to make a big difference to how people moved!  

  

  It makes sense to me that people move a lot more in both directions with a slider, because it’s hard to hit the same number again if you don’t remember it. It’s surprising to me that they moved with similar frequency with buttons and open response, because the buttons covered relatively chunky ranges (e.g. 5-25%) so need larger shifts to be caught.

• Input type also made a big difference to the probabilities people gave to doom before seeing any arguments. People seem to give substantially lower answers when presented with buttons (Nathan proposes this is because there was was a <1% and 1-5% button, so it made lower probabilities more salient/ “socially acceptable”, and I agree):

  

• Overall, P(doom) numbers were fairly high: 24% average, 11% median.

  

• We added a ‘control argument’. We presented this as “Here is an argument that advanced AI technology might threaten humanity:” like the others, but it just argued that AI might substantially contribute to music production:

  

  
  This was the third worst argument in terms of prompting upward probability motion, but the third best in terms of being “compelling”. Overall it looked a lot like other arguments, so that’s a bit of a blow to the model where e.g. we can communicate somewhat adequately,  ‘arguments’ are more compelling than random noise, and this can be recognized by the public.

• In general, combinations of claims about compellingness, changes in probabilities and write-in answers were frequently hard to make sense of, especially if you treat the probability changes as meaningful rather than as random noise. For instance, the participant who rated an argument 4/5 compellingness, yet reduced their P(doom) from 26% to 0%, and said “The above argument is completely based on probability of AI's effects on humanity near future, some feels that it could be turn into negative way but most people feels that it is going to a good aspect for future technology.” My sense is that this was more true in the first round than the fourth, so perhaps the counterarguments are doing something there.

• This is how the different arguments fared:

  

• All arguments were on mean between 2 and 3 compelling, including the control argument

• The arguments we considered worst did roughly worst, in terms of probability change (black  boxes, large impacts, multiagent dynamics, control)

• The argument from expert opinion was the very worst though, which is interesting to me because it seems like the one that people are constantly pointing at in public, in trying to justify concerns about x-risk. 

• The top arguments for increasing P(doom) were the ones about normal human processes getting out of hand (human non-alignment, catastrophic tools, speed) then the ones about bad new agents came below (second species, competent non-aligned agents, inferiority). Compellingness looks related but not closely so. 

• We both found the experience of quickly polling the public enlivening.

If you wish to look at the arguments in more detail, they are here. If you want to analyze the data yourself, or read everyone’s write-in responses, it’s here. If you see any errors, please let us know.