When the smarter thing is able to make itself way smarter, it soon—perhaps almost instantly—will proceed to take control of systems around it, upgrading them as well, using its rapidly and recursively improving brain to solve problems in ways that humans never could. Before long, global warming will be a thing of the past, poverty and disease will be conquered, abundant energy will be sustainable and free, space will be opened, and wars will end forever. ‘Tis a consummation devoutly to be wished.
The rub, of course, is that this brainy new intelligence might not necessarily be inclined to work in favor of and in service to humanity. What if it turns out to be selfish, apathetic, despotic, or even psychotic? Our only hope, according to “friendly AI” enthusiasts, is to program the first artificial general intelligence with built-in goals that constrain it toward ends that we would find desirable. As the Singularity Institute puts it, the aim is “to ensure the AI absorbs our virtues, corrects any inadvertently absorbed faults, and goes on to develop along much the same path as a recursively self-improving human altruist.”
So, what we want is a very very smart friend who will always be trustworthy, loyal, and obedient.
Notes
-
chrbutler liked this
-
porksteeple reblogged this from buffleheadcabin
-
buffleheadcabin posted this
