CSE 151 Lecture Notes

---

INTRODUCTION TO ARTIFICIAL INTELLIGENCE

---

AI: A Modern Approach, Chapter 1: Introduction (continued)

Last time we talked about possible definitions for Artificial Intelligence on two possible dimensions:

• thinking vs. acting
• human vs. rational

We will view the primary goal of AI as being to develop systems which act rationally. We call this the intelligent agent approach.

Today we will talk about the foundations and history of AI.

The Foundations of AI

• Artificial Intelligence builds upon several disciplines, primarily
  • Philosophy
  • Mathematics
  • Psychology
  • Linguistics
  • Computer Engineering

• Philosophy and AI

  Philosophers have contributed to AI the notions that
  • the mind operates like a machine,
  • knowledge is stored in encoded form in the mind, and
  • correct thought processes lead to correct actions.

  The Mind

  Dualism	part of the mind is exempt from physical laws. (Descartes 1600s) This idea is still around and used to "disprove" AI.
  Materialism	the mind operates according to physical laws. (Liebniz 1600s)

  Knowldedge

  Empiricism	the senses are the source of all knowledge. (Bacon, Locke 1600s)
  Induction	Rules are learned by repeated associations. (Hume 1700s) This is the primary view of learning in AI today.
  Logical Positivism	All knowledge can be represented by testable logical sentences. (B. Russel ~1900)

  Action

  Means-ends analysis	Work backward from the goal to decide on what action(s) will help you acheive it. (Aristotle 300 BCE)
  Utilitarianism	The correct action depends on your beliefs, desires and your estimate of the probabilities of different outcomes. (Arnauld 1600s, Mill 1700s)

• Mathematics and AI

  Mathematics has given AI
  • logic for representing and reasoning with knowledge,
  • probability theory for dealing with uncertain knowledge,
  • and ways to reason about algorithms.

  Mathematical logic

  Propositional Logic (Boole 1850s)	Knowledge can be represented as sentences using and, or, not and combined according to simple, correct rules. DaffyWalksLikeDuck and DaffyTalksLikeDuck -> DaffyIsDuck Lacks expressiveness; we must introduce new symbols and a new sentence to say DonaldwalksLikeDuck & DonnaldTalksLikeDuck -> DonaldIsDuck
  First Order Logic (Frege 1879)	FOL adds quantifiers, variables, predicates and functions to propositional logic. Now we can say ForAll(x) WalksLikeDuck(x) and TalksLikeDuck(x) -> IsDuck(x)

  Probability Theory and AI

  Basic concepts of probability (Cardano 1500s)	Invented to explain games of chance.
  Bayesian Statistics (Bayes 1763)	Showed how to update probabilities given new evidence.
  Decision Theory (Von Neumann 1944)	Combines utility and probability; this is the basis for our definition of rationality.

  Reasoning about Algorithms

  Incompleteness Theorem (Goedel 1931)	No system can prove all true statements about anything as complex as the natural numbers. (But FOL is simpler and a complete proof algorithm does exist...)
  Church-Turing Thesis (1936)	The Turing machine is a complete model of computability.
  Intractability (Cobham, Edmond, Dantzig 1960s)	Problems requiring exponential time in the size of their inputs are intractable. One class of problems can be reduced to another: 3SAT -> TSP
  NP-completeness (Cook, Karp 1971)	Some problems are very hard - as hard as any other problem that can be solved in non-deterministic polynomial time. It is very unlikely any computer will ever be able to solve these problems.

• Psychology and AI

  Modern psychology has given AI
  • the view that the brain can be viewed as processing information like a machine, and
  • models of cognition.

  Cognitive psychology (James ~1900, Craik 1943) views natural cognition as

  • receive a stimulus
  • manipulate the internal model
  • perform an action

  This is also our view of what an intelligent agent does.

• Linguistics and AI

  Modern linguistics and AI have grown up together. Much work in knowledge representation is tied to linguistics.

• Computer Engineering and AI

  Computer engineering has provided the artifact upon which all (current) AI is based.

The History of AI

AI has had a short but turbulent history. Periods of euphoria have been balanced with ones of pessimism.

History has taught that AI is not easy and real progress will come through incremental growth that builds on a sound theoretical basis, uses data rigorously through learning and addresses real world, not toy, applications.

Modern success stories in AI include speech recognition, planning and expert systems.

• The Early Period (1943 - 1969)

  Early AI research concentrated on so-called weak methods.

  Weak methods use general purpose reasoning in small steps to reach conclusions.

  During this period many of the basic tools of AI were discovered or invented:

  • Inductive learning (Samuel's checker program ~1960)
  • LISP programming language (McCarthy 1958)
  • The resolution proof method for FOL (Robinson)
  • Perceptrons (early neural nets)

  A number of interesting problems were tackled early on:

  • Logic theorist (Newell, Simon 1956)
    automatically proved theories about natural numbers
  • GPS (Newell, Simon)
    solved problems using means-ends analysis
  • microworlds programs: Blocksworld, Analogy (IQ tests)
  • perceptron learning (Rosenblatt)
  • search methods

  The problem was, things didn't scale up. The problems tackled were all intractable, so only toy examples can be solved using weak methods. This caused much funding to be cut in the 1960's and 1970's.

• The Current Period (1969 - Present)

  Current AI research emphasizes the use of domain-dependent knowledge in order to overcome the intractibility of AI problems. These are sometimes referred to as strong methods.

  Successful AI systems include:

  • DENDRAL (Buchanan 1969),
    predicts molecular structure from mass spectrometry data.
  • LUNAR (Woods 1973),
    answers questions about lunar rocks phrased in English.
  • R1, (McDermot)
    the first commercial expert system, configured VAX systems.

  Several advances have been made in basic AI techniques.

  • FRAMES (Minsky, 1975),
    a technique for organizing knowledge by arranging facts in a taxonomical hierarchy.
  • Backprop (Rumelhart, Hinton 1982),
    the learning method for neural nets was rediscovered.
  • Belief networks (Pearl 1988),
    a rigourous system for reasoning with uncertainty.

  These have lead to major advances in

  • expert systems,
  • machine learning,
  • neural nets,
  • speech recognition,
  • natural language processing, and
  • planning systems