Words&Dirt

“Before the prospect of an intelligence explosion, we humans are like small children playing with a bomb. Such is the mismatch between the power of our plaything and the immaturity of our conduct. Superintelligence is a challenge for which we are not ready now and will not be ready for a long time. We have little idea when the detonation will occur, though if we hold the device to our ear we can hear a faint ticking sound.

For a child with an undetonated bomb in its hands, a sensible thing to do would be to put it down gently, quickly back out of the room, and contact the nearest adult. Yet what we have here is not one child but many, each with access to an independent trigger mechanism. The chances that we will all find the sense to put down the dangerous stuff seem almost negligible. Some little idiot is bound to press the ignite button just to see what happens.

Nor can we attain safety by running away, for the blast of an intelligence explosion would bring down the entire firmament. Nor is there a grown-up in sight.

In this situation, any feeling of gee-wiz exhilaration would be out of place. Consternation and fear would be closer to the mark; but the most appropriate attitude may be a bitter determination to be as competent as we can, much as if we were preparing for a difficult exam that will either realize our dreams or obliterate them.

This is not a prescription of fanaticism. The intelligence explosion might still be many decades off in the future. Moreover, the challenge we face is, in part, to hold on to our humanity; to maintain our groundedness, common sense, and good-humored decency even in the teeth of this most unnatural and inhuman problem.”

––Superintelligence: Paths, Dangers, Strategies, by Nick Bostrom, pg. 259-60

“‘Somewhere, far, far away, there’s a shitty island. An island without a name. An island not worth giving a name. A shitty island with a shitty shape. On this shitty island grow palm trees that also have shitty shapes. And the palm trees produce coconuts that give off a shitty smell. Shitty monkeys live in the trees, and they love to eat these shitty-smelling coconuts, after which they shit the world’s foulest shit. The shit falls on the ground and builds up shitty mounds, making the shitty palm trees that grow on them even shittier. It’s an endless cycle.’

I drank the rest of my coffee.

‘As I sat here looking at you,’ I continued, ‘I suddenly remembered the story of this shitty island. What I’m trying to say is this: A certain kind of shittiness, a certain kind of stagnation, a certain kind of darkness, goes on propagating itself with its own power in its own self-contained cycle. And once it passes a certain point, no one can stop it––even if the person himself wants to stop it.’”

––The Wind-Up Bird Chronicle, by Haruki Murakami, pg. 202