I started working at AI Impacts slightly less than a year ago. Before then, I was not following developments in either AI or AI safety. I do not consider myself a rationalist and did not engage with LessWrong before starting this job. While I have mostly been working on historical case studies,

> A particular notion of creativity that involves increasing the dimension of the space of possible options, as opposed to recombining existing options.

To me, the fact that AlphaZero has come up with new go openings seems to basically refute this. You can justify it as "well it's recombining existing options of moves on the go board" but that seems as uselessly reductionistic as describing any human creativity as recombining existing options to contract various muscles.

> There are some examples of inherently quantum effects impacting the macroscopic behavior of some biological systems. These are sometimes not things where the effects can be fully captured by setting some macroscopic parameters.

Aren't classical computers capable of simulating quantum systems?

Sort of. The SchrÃ¶dinger equation is a PDE. There are numerical methods for PDEs, but they don't scale well.

For an arbitrarily complex (nonrelativistic) quantum system, each particle adds three dimensions to the PDE. The computational cost for an N particle system scales as (system size)^(3N). The best PDE solvers I'm aware of are for PDEs of up to 6 dimensions, so N=2.* Mostly what people do for many body quantum systems is to aggressively approximate things until the PDE gets small enough to solve. For example, assuming that all the free electrons in a material are basically equivalent and so you only need to solve a single 3 dimensional nonlinear SchrÃ¶dinger equation. It's really not clear when these approximations do and do not work. The brain is not an arbitrarily complex quantum system with N = 10^25 atoms: it's wildly simpler than that. But it's probably not an N=2 problem either.** It doesn't take much to make a quantum system wildly more difficult than anything we can feasibly compute.

The other potential problem is that the output is a probability amplitude, rather than a prediction of what in particular is going to happen. I would not be surprised if almost any approximation thermalizes (equal probability on all accessible states, no entanglement) even in situations where the real system does not thermalize.

* These are solving the Vlasov equation for plasma physics, rather than solving the SchrÃ¶dinger equation. Other branches of physics might try to numerically solve higher dimensional PDEs, but not that much higher.

** The thing I described in "Whole Bird Emulation Requires Quantum Mechanics" is simpler than N=2. I expect that this is mostly evidence that it's easiest to discover the simplest quantum effects in biology.

> A particular notion of creativity that involves increasing the dimension of the space of possible options, as opposed to recombining existing options.

To me, the fact that AlphaZero has come up with new go openings seems to basically refute this. You can justify it as "well it's recombining existing options of moves on the go board" but that seems as uselessly reductionistic as describing any human creativity as recombining existing options to contract various muscles.

> There are some examples of inherently quantum effects impacting the macroscopic behavior of some biological systems. These are sometimes not things where the effects can be fully captured by setting some macroscopic parameters.

Aren't classical computers capable of simulating quantum systems?

Sort of. The SchrÃ¶dinger equation is a PDE. There are numerical methods for PDEs, but they don't scale well.

For an arbitrarily complex (nonrelativistic) quantum system, each particle adds three dimensions to the PDE. The computational cost for an N particle system scales as (system size)^(3N). The best PDE solvers I'm aware of are for PDEs of up to 6 dimensions, so N=2.* Mostly what people do for many body quantum systems is to aggressively approximate things until the PDE gets small enough to solve. For example, assuming that all the free electrons in a material are basically equivalent and so you only need to solve a single 3 dimensional nonlinear SchrÃ¶dinger equation. It's really not clear when these approximations do and do not work. The brain is not an arbitrarily complex quantum system with N = 10^25 atoms: it's wildly simpler than that. But it's probably not an N=2 problem either.** It doesn't take much to make a quantum system wildly more difficult than anything we can feasibly compute.

The other potential problem is that the output is a probability amplitude, rather than a prediction of what in particular is going to happen. I would not be surprised if almost any approximation thermalizes (equal probability on all accessible states, no entanglement) even in situations where the real system does not thermalize.

* These are solving the Vlasov equation for plasma physics, rather than solving the SchrÃ¶dinger equation. Other branches of physics might try to numerically solve higher dimensional PDEs, but not that much higher.

** The thing I described in "Whole Bird Emulation Requires Quantum Mechanics" is simpler than N=2. I expect that this is mostly evidence that it's easiest to discover the simplest quantum effects in biology.