Lecture 01: Overview of Artificial Intelligence

The recommended textbook is:

Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig (Fourth Edition).

---

What is AI & Intelligence?

For thousands of years, humans have tried to understand how we think—specifically, how a mere "handful of matter" can perceive, understand, predict, and manipulate a world far larger and more complicated than itself.

Artificial Intelligence (AI) attempts not just to understand intelligent entities, but to build them.

Defining Intelligence

Intelligence is a complex concept with no single, universally accepted definition; philosophers, psychologists, scientists, and engineers view it differently. However, key characteristics include:

• Autonomous: Capable of operating without constant instructions.
• Adaptive: Able to change behavior based on environmental or situational changes.

At the core of operation for both living organisms and intelligent machines is data (visuals, sounds, measurements). We consume and process data to make informed decisions, making data understanding fundamental to AI algorithms.

---

The Four Core Philosophies of AI

The approaches to defining and building AI can be categorized into four main perspectives:

1. Acting Humanly: The Turing Test Approach

Proposed by Alan Turing in 1950 in his landmark paper "Computing Machinery and Intelligence", the Turing Test provides an operational definition of intelligence. A computer passes if a human interrogator, after posing written questions, cannot distinguish whether responses come from a human or a machine.

To pass the standard Turing Test, a machine requires:

• Natural Language Processing (NLP): To communicate successfully in human language.
• Knowledge Representation: To store what it knows or hears.
• Automated Reasoning: To answer questions and draw new conclusions.
• Machine Learning: To adapt to new circumstances and detect patterns.

The Total Turing Test requires physical interaction with objects and people in the real world, adding:

• Computer Vision & Speech Recognition: To perceive the physical world.
• Robotics: To manipulate objects and move about.

2. Thinking Humanly: The Cognitive Modeling Approach

This approach seeks to uncover the actual inner workings of human minds through three methods:

1. Introspection: Catching our own thoughts as they occur.
2. Psychological Experiments: Observing human behavior in controlled settings.
3. Brain Imaging: Observing the physical brain in action.

Once a precise theory of the mind is formed, it can be expressed as a computer program. The interdisciplinary field of Cognitive Science brings together AI computer models and experimental psychology to build testable theories of human thought. (Note: Opinions vary widely among researchers on cognitive modeling).

3. Thinking Rationally: The "Laws of Thought" Approach

Grounded in Aristotle's syllogisms (patterns of logical arguments yielding correct conclusions from correct premises, e.g., "All men are mortal; Socrates is a man; Therefore Socrates is mortal").

In the 19th century, logicians created precise notations for objects and relations. By 1965, programs existed that could, in principle, solve any solvable problem described in logical notation (though they could loop indefinitely if no solution existed).

The Logicist tradition aims to build intelligent systems on formal logic. However, this approach faces major challenges:

• Informal real-world knowledge is rarely 100% certain and is difficult to express formally.
• There is a vast gap between solving a problem "in principle" and solving it "in practice" due to computational complexity.

4. Acting Rationally: The Rational Agent Approach

An Agent is anything that acts. While all computer programs act, computer agents are expected to do more:

• Operate autonomously
• Perceive their environment
• Persist over a prolonged time period
• Adapt to change
• Create and pursue goals

A Rational Agent acts to achieve the best outcome or, under uncertainty, the best expected outcome.

Why the Rational Agent Approach Dominates:

• Broader than Logic: Rationality encompasses correct inference plus reactive, goal-driven, and adaptive behaviors.
• Mathematically Defined: Standards of rationality permit provable agent designs.
• Handles Uncertainty: Incorporates decision theory and probability.
• Limited Rationality: Acknowledges that perfect rationality is often computationally unfeasible in complex environments, leading to bounded/limited rationality.
• Cross-Disciplinary Links: Connects AI with control theory, operations research, statistics, and economics.

---

A Detailed History of AI

The history of AI features cycles of extreme optimism followed by periods of disillusionment and funding cuts ("AI Winters").

Foundations & The Birth of AI (1943 – 1956)

• 1943: Warren McCulloch and Walter Pitts proposed the first artificial neuron model, drawing from neuron physiology, propositional logic (Russell & Whitehead), and Turing's computation theory.
• 1949: Donald Hebb introduced Hebbian Learning, a rule for updating synaptic connection strengths between neurons.
• 1950: Marvin Minsky and Dean Edmonds built SNARC, the first neural network computer (simulating 40 neurons).
• 1950: Alan Turing published "Computing Machinery and Intelligence", introducing the Turing Test, machine learning, genetic algorithms, and reinforcement learning.
• 1952: Christopher Strachey and Arthur Samuel independently developed early checkers-playing programs.
• 1956: The landmark Dartmouth Workshop, organized by John McCarthy (and attended by Minsky, Shannon, Rochester, Newell, Simon, etc.), where the field was officially named "Artificial Intelligence". Newell and Simon demonstrated the Logic Theorist.

Early Enthusiasm & Microworlds (1957 – 1973)

• 1957: Herbert Simon predicted computers would become chess champions and prove major math theorems within 10 years (took ~40 years).
• 1958: John McCarthy defined Lisp (the dominant AI programming language for 30 years) and proposed the Advice Taker concept.
• 1959: Newell & Simon developed the General Problem Solver (GPS), modeling human problem-solving protocols and giving rise to the Physical Symbol System Hypothesis.
• 1959: Herbert Gelernter developed the Geometry Theorem Prover at IBM; Arthur Samuel demonstrated checkers programs learning via reinforcement learning.
• Divergence of Philosophy: McCarthy emphasized formal logic, while Minsky prioritized practical program functionality (developing an anti-logic outlook).
• Microworlds:
  • Stanford: Focused on logical reasoning (QA systems, Shakey robot).
  • MIT: Focused on narrow problem domains—SAINT (calculus), ANALOGY (geometric analogy), and STUDENT (algebra story problems).
  • Early neural network advancements included Adalines and Perceptrons.

First AI Winter & Obstacles (1970s)

• Failure Reasons: Over-reliance on "informed introspection", underestimation of combinatorial explosion, and intractable computational complexity.
• 1973 (Lighthill Report): Sir James Lighthill's report to the British government criticized AI's failure to handle combinatorial scaling, leading to major reductions in UK research funding.
• Perceptron Limitations: Minsky and Papert published Perceptrons (1969), proving severe limitations of single-layer neural networks, which halted neural net funding for over a decade.

Expert Systems Era (1969 – 1986)

• Shift from general "weak methods" (search-based) to domain-specific knowledge-intensive systems:
  • DENDRAL: Inferred molecular structures from mass spectrometer data.
  • MYCIN: Diagnosed blood infections using ~450 rules, matching human expert performance.
  • R1 (XCON): Configured computer systems commercially at Digital Equipment Corporation (DEC).
• Tools evolved using Prolog and Minsky's frames.
• Large national initiatives emerged (Japan's Fifth Generation project, US MCC).
• Second AI Winter: Disillusionment followed as companies struggled to maintain complex rule bases, handle uncertainty, or enable systems to learn.

Neural Networks Return & Probabilistic AI (1986 – 2000)

• Mid-1980s: Rediscovery of the Back-Propagation algorithm and publication of Parallel Distributed Processing (PDP) by Rumelhart & McClelland.
• Connectionist Models: Emerged as competitors to symbolic logic, excelling at learning parameters from data.
• 1988 – Probabilistic Reasoning: Judea Pearl published "Probabilistic Reasoning in Intelligent Systems", establishing Bayesian Networks for handling uncertainty.
• Reinforcement Learning: Rich Sutton linked RL to Markov Decision Processes (MDPs) from operations research.
• Reintegration of subfields (computer vision, robotics, NLP) around statistical modeling.

Big Data & Deep Learning (2001 – Present)

• 2001+ (Big Data): Massive web datasets showed that increasing dataset size often surpassed fine-tuning complex algorithms (e.g., word-sense disambiguation, image completion).
• 2011: IBM Watson won Jeopardy!, reshaping public awareness of AI.
• 2012: AlexNet (CNN by Krizhevsky, Sutskever, Hinton) won ImageNet, launching modern deep learning.
• 2014: GANs (Generative Adversarial Networks) introduced by Ian Goodfellow et al.
• 2015: ResNets (He et al.) enabled training extremely deep networks; YOLO (Redmon et al.) revolutionized real-time object detection.
• 2016: DeepMind's AlphaGo defeated world champion Go player Lee Sedol.
• 2018: Google introduced BERT, transforming NLP.
• 2020: OpenAI released GPT-3 for large-scale language generation.
• 2022: OpenAI released DALL-E for text-to-image synthesis.

---

State of the Art Applications

Modern AI methods are deployed across diverse domain areas:

• Robotic Vehicles & Autonomous Driving
• Speech Recognition & Natural Language Understanding
• Game Playing (Go, Chess, Starcraft)
• Spam Fighting & Fraud Detection
• Robotics & Industrial Automation
• Machine Translation
• Autonomous Planning & Logistics Scheduling (e.g., DARPA's DART tool, which single-handedly offset DARPA's 30-year investment in AI research).