PhilSci Archive

Artificial superintelligence and its limits: why AlphaZero cannot become a general agent

Jebari, Karim and Lundborg, Joakim (2019) Artificial superintelligence and its limits: why AlphaZero cannot become a general agent. [Preprint]

[img]
Preview
Text
Artificial superintelligence and its limits.pdf - Submitted Version
Available under License Creative Commons Attribution No Derivatives.

Download (179kB) | Preview

Abstract

An intelligent machine surpassing human intelligence across a wide set of skills has been proposed as a possible existential catastrophe (i.e., an event comparable in value to that of human extinction). Among those concerned about existential risk related to Artificial Intelligence (AI), it is common to assume that AI will not only be very intelligent, but also be a general agent (i.e., an agent capable of action in many different contexts).

This article explores the characteristics of machine agency, and what it would mean for a machine to become a general agent. In particular, it does so by articulating some important differences between belief and desire in the context of machine agency. One such difference is that while an agent can by itself acquire new beliefs through learning, desires need to be derived from preexisting desires or acquired with the help of an external influence. Such influence could be a human programmer or natural selection. We argue that to become a general agent, a machine needs productive desires, or desires that can direct behavior across multiple contexts. However, productive desires cannot sui generis be derived from non-productive desires. Thus, even though general agency in AI could in principle be created, it cannot be produced by an AI spontaneously through an endogenous process. In conclusion, we argue that a common AI scenario, where general agency suddenly emerges in a non-general agent AI, is not plausible.


Export/Citation: EndNote | BibTeX | Dublin Core | ASCII/Text Citation (Chicago) | HTML Citation | OpenURL
Social Networking:
Share |

Item Type: Preprint
Creators:
CreatorsEmailORCID
Jebari, Karimkarim.jebari@iffs.se0000-0002-7378-3662
Lundborg, Joakimjoakim.lundborg@gmail.com0000-0003-2262-4749
Keywords: artificial intelligence, machine learning, super intelligence, general intelligence, agency, intentional action,
Subjects: Specific Sciences > Cognitive Science > Action
Specific Sciences > Artificial Intelligence > AI and Ethics
Specific Sciences > Cognitive Science > Concepts and Representations
Specific Sciences > Artificial Intelligence > Machine Learning
Depositing User: Dr. Karim jebari
Date Deposited: 06 Dec 2019 06:48
Last Modified: 06 Dec 2019 06:48
Item ID: 16683
Subjects: Specific Sciences > Cognitive Science > Action
Specific Sciences > Artificial Intelligence > AI and Ethics
Specific Sciences > Cognitive Science > Concepts and Representations
Specific Sciences > Artificial Intelligence > Machine Learning
Date: December 2019
URI: https://philsci-archive-dev.library.pitt.edu/id/eprint/16683

Monthly Views for the past 3 years

Monthly Downloads for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item