Ling forced herself to smile, forced herself to turn back to the human, the way her mother had taught her, forced herself to say the insipid words.

“May I please have a few minutes more, teacher? I like to watch the rain.”

“Well.” The tutor sounded surprised. “Since you asked so politely, you may.”

Chen surrendered his electronics at the next checkpoint. Then the security man slowly and thoroughly wanded him, looking not for weapons, but for any device that could possibly carry data in or out of the PICC.

The guard finally declared him clean, and Chen stepped forward and into the cavernous elevator. The doors closed behind him, and the elevator started its descent through one thousand meters of bedrock and towards the mad software entity that was all that remained of his dead wife.

Ling sifted through exabytes of data. Cryptographic libraries. High resolution satellite imagery. Whole brain scans. Genome sequences. None of it was her mother.

She looked for maps, physical maps, network maps. She found them. The network topology told her little. Nothing obviously fit the description of the quantum cluster her mother existed in. The physical blueprints of the building were no more helpful. Multiple data centers existed here, but their functions weren’t clear.

Ling kept searching. She would find her mother here.

The room-sized elevator took Chen down through the rock beneath Shanghai. A lit sign declared that the current PICC status was

ISOLATION IN EFFECT

All this was a precaution. Computer scientists, philosophers, futurists, writers of speculative fiction – they’d all written about the dangers of runaway superintelligence. If humanity ever created a being of radically increased mental capabilities, it placed itself at grave risk. That new being could be benevolent, of course. That would be the hope. Or it could be malicious, or simply indifferent to humans. It could seek to change the world in ways that it saw as improvements, but which were incompatible with the interests of its creators.

A superintelligent being might also be able to improve on itself, reaching into its own structure and finding ways to optimize them, to make itself smarter than its creators could have, with no obvious end in sight.

And for that reason, Su-Yong’s ability to edit herself was limited to superficial layers only.

Chen himself doubted the risk of runaway self-improvement. Intelligence showed diminishing returns. Just as a single human could not design a human-level intelligence from scratch, no superhuman creature could possibly design a creature of its own intelligence or greater. Oh, it might be able to make improvements on the methods used by its creators, and get some boost, but without collaborators, without access to new hardware, the improvements would level off.