[1911.01547] The Measure of Intelligence

To make deliberate progress towards more intelligent and more human-like artificial systems, we need to be following an appropriate feedback signal: we need to be able to define and evaluate intelligence in a way that enables comparisons between two systems, as well as comparisons with humans. Over the past hundred years, there has been an abundance of attempts to define and measure intelligence, across both the fields of psychology and AI. We summarize and critically assess these definitions and evaluation approaches, while making apparent the two historical conceptions of intelligence that have implicitly guided them. We note that in practice, the contemporary AI community still gravitates towards benchmarking intelligence by comparing the skill exhibited by AIs and humans at specific tasks such as board games and video games. We argue that solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experie

23 mentions: @fchollet@p_ferragu@IntuitMachine@threadreaderapp@mxwlj@rezendi@gyrhead@CarlRioux
Date: 2019/11/06 20:20

Referring Tweets

@fchollet I've just released a fairly lengthy paper on defining & measuring intelligence, as well as a new AI evaluation dataset, the "Abstraction and Reasoning Corpus". I've been working on this for the past 2 years, on & off. Paper: t.co/djNAIUZF7E ARC: t.co/MvubT2HTKT t.co/bVrmgLAYEv
@p_ferragu Excellent paper, @fchollet (t.co/a6IsKlYXTC) What I took from it: 1) The very idea of general intelligence is misleading. intelligence is specific to a context (e.g. human mind is orders of magnitude better at 2D and 3D than 4D)
@jacquesludik The Measure of Intelligence To make deliberate progress towards more intelligent and more human-like artificial systems, we need to be following an appropriate feedback signal: we need to be able to define and evaluate intelligenc…t.co/yI4l90tvY7 t.co/JKhdCYD1Wk
@IntuitMachine @fchollet's 'The Measure of Intelligence' t.co/oz5IqMbOQb is correct in that AGI tests should be with respect to anthropocentric capabilities and not some ill-defined universal intelligence. It's simple enough to create non-anthropocentric tests.
@gyrhead Abstract Reasoning Corpus app :l“We argue that ARC can be used to measure a human-like form of general fluid intelligence and that it enables fair general intelligence comparisons between AI systems and humans.” #AI #ML@fchollett.co/DsCH7sld8w
@rezendi "We then articulate a new formal definition of intelligence based on Algorithmic Information Theory, describing intelligence as skill-acquisition efficiency" 🤔 t.co/GPIek3yktL