Soon, to the scientists’ delight, the terminal displaying the AI’s progress shows the artificial intelligence has surpassed the intelligence level of a human, known as AGI, or artificial general intelligence. Soon it becomes smarter by a factor of ten, then a hundred. In just two days, it is one thousand times more intelligent than any human, and still improving.
The scientists have passed a historic milestone! For the first time humankind is in the presence of an intelligence greater than their own. Artificial superintelligence, or ASI.
Now what happens?
Our Final Invention: Artificial Intelligence and the End of the Human Era
by James Barrat
AI is a dual-use technology like nuclear fission. Nuclear fission can illuminate cities or incinerate them. Its terrible power was unimaginable to most people before 1945. With advanced AI, we’re in the 1930s right now. We’re unlikely to survive an introduction as abrupt as nuclear fission’s.