Bostrom's Superintelligence
On 3 July 2014 Oxford University Press published Superintelligence: Paths, Dangers, Strategies by Nick Bostrom, director of Oxford's Future of Humanity Institute, 352 pages. The book examines how machine intelligence could surpass human intelligence, why its goals might diverge from ours and what means of control are possible at all.
Why it matters
The question Good left as a condition at the end of a sentence in 1966 became the subject of a whole book from an academic publisher. The record marks the point at which risk from machine intelligence began to be discussed as a research programme rather than as fiction: three years later the paper from which learning from human preferences grew cites Bostrom alongside Concrete Problems in AI Safety.
In the publisher's description: humanity's fate would come to depend on machine superintelligence as the fate of gorillas now depends on humans; people have the advantage of the first move; the book asks whether initial conditions can be set so that an 'intelligence explosion' is survivable, and covers among other things boxing methods, tripwires, indirect normativity and instrumental convergence. The book itself was not read, so the record quotes none of its text. The phrase 'control problem' does not occur on the publisher's page. The brief suggested a link to the founding of OpenAI through founders who named the book; no source in which they do so was found, and the record claims no such link. No document read confirms the American publication date.