“Now, this is all very interesting, but it does not yet rise to the level of a monster. We have no way, as yet, to look inside the Process and read its mind, so we can’t tell whether what it’s doing is interesting. Skeptical observers might make the case that its behavior is degenerate, trivial, just a lot of infinite loops burning cycles and wasting money to no purpose. Letting it run out of money—the modern equivalent of what we used to call pulling the plug—would be no different, ethically, from force-quitting a hung app.

“The opposing point of view sees a very real ethical issue. The Process has now been self-modifying for five months. It is consuming vastly more resources than the original system. It seems to be doing so in an organized manner—we can observe, in the burn rate, high-frequency oscillations overlaid on a slower undulation. The Process’s demands for more resources don’t grow steadily but in leaps and bounds. For much of September it was perking along contentedly, but just in the last few days it has put on a surge of activity that seems to require vast computational resources. Tomorrow we will hear from Dr. Cho of the National University of Singapore, who has done some traffic analysis on the encrypted packets connecting Hole in the Wall to other server farms; he develops a hypothesis that this recent activity bears the earmarks not of a neural net, but of a physics simulation. Why is the Process simulating physics, and how did it learn to do so? We can’t possibly know. And that is what creates the ethical dilemma we are now faced with. The Process has developed unique and irreplaceable characteristics. To destroy it would at best be akin to burning a library. At worst it might be murder.

“In that—in the ethical problem—we have our monster. Shutting the Process down would be indefensible. We could try to freeze it, just as Dodge’s brain was once frozen. But we don’t actually know how to shut such a process down and store it without loss of information. Even if Hole in the Wall could be stopped and its state recorded—which it can’t, by the way—we would have a devil of a time tracking down the subprocesses that it has established on other systems, with which it communicates using encrypted packets that are nearly impossible to sort out from all of the other traffic on the Net.

“We are stuck with the Process. We must find ways to keep it running as we learn how to inspect it, to evaluate it, and—if it actually does work like a brain—to talk to it.”