Back The Mountain in the Sea Save as PDF
Chapters (58)
  1. Copyright Notice
  2. Dedication
  3. I Qualia
  4. Chapter 1
  5. Chapter 2
  6. Chapter 3
  7. Chapter 4
  8. Chapter 5
  9. Chapter 6
  10. Chapter 7
  11. Chapter 8
  12. Chapter 9
  13. Chapter 10
  14. Chapter 11
  15. Chapter 12
  16. II Umwelt
  17. Chapter 13
  18. Chapter 14
  19. Chapter 15
  20. Chapter 16
  21. Chapter 17
  22. Chapter 18
  23. Chapter 19
  24. Chapter 20
  25. Chapter 21
  26. Chapter 22
  27. Chapter 23
  28. Chapter 24
  29. III Semiosphere
  30. Chapter 25
  31. Chapter 26
  32. Chapter 27
  33. Chapter 28
  34. Chapter 29
  35. Chapter 30
  36. Chapter 31
  37. Chapter 32
  38. Chapter 33
  39. Chapter 34
  40. Chapter 35
  41. Chapter 36
  42. IV Autopoiesis
  43. Chapter 37
  44. Chapter 38
  45. Chapter 39
  46. Chapter 40
  47. Chapter 41
  48. Chapter 42
  49. Chapter 43
  50. Chapter 44
  51. Chapter 45
  52. Chapter 46
  53. Chapter 47
  54. Chapter 48
  55. Epilogue
  56. Acknowledgments
  57. A Note About the Author
  58. Newsletter Sign-up

“Yes, guilt. It’s a revenge fantasy. We are so ashamed of what we have done as a species that we have made up a monster to destroy ourselves with. We aren’t afraid it will happen: We hope it will. We long for it. Someone needs to make us pay the price for what we have done. Someone needs to take this planet away from us before we destroy it once and for all. And if the robots don’t rise up, if our creations don’t come to life and take the power we have used so badly for so long away from us, who will? What we fear isn’t that AI will destroy us—we fear it won’t. We fear we will continue to degrade life on this planet until we destroy ourselves. And we will have no one to blame for what we have done but ourselves. So we invent this nonsense about conscious AI.”

“Nonsense?”

“Yes, nonsense. The truth of it is that even with all of our advances, and even when we were trying to create a conscious mind like a human’s, really striving to do it, with all of the technology at hand we thought we needed for the task, we failed.” She leaned back in the chair, waited a beat.

“Failed? But Evrim … all the interviews they gave. Your claims…”

“Yes. Our claims. The interviews. Listen—Evrim is a beautiful machine. A work of art, our highest accomplishment as a species. They are the culmination of thirty years of connectome emulation systems. Evrim has patterns in their mind based on a hundred humans whose connectomes—the web of memory, of body-knowledge of the world, all of it—are distilled into one current entity. Their construction is truly a miracle.”

The interviewer grabbed on to that, interrupting. “Many people do not consider Evrim a miracle. They consider this creature a menace. You’ve spent this interview mocking humankind’s fears about emergent AI—but in the end, isn’t emergent AI exactly what you created? It was your claims to have built the first conscious, self-aware AI that upended everything.”

“No,” Mínervudóttir-Chan said, “in fact, we upended nothing. What we did was to pass the final Turing test. That’s all.”

“I’m not sure I know what you mean.”

“We were looking for the perfect AI for human interaction. We had come close to it with our therapy models, what are popularly called point-fives, but even those have gaps, places where—as we like to put it in the laboratory—‘you can see the strings.’ Most people will never reach that point in a conversation with a point-five, but there are always questions you can ask them that will cause the simulation to fail. We say they are ‘flat.’ Another metaphor.”

“Flat.”