Lecture 02.2: Agent Programs and Representations

Agent Program Types

An agent is comprised of an architecture and a program. The agent program runs on the physical architecture to produce the agent function f.

There are four basic types of agent programs (along with a fifth overarching type):

1. Simple Reflex Agents

These agents select actions based purely on the current percept, ignoring the rest of the percept history. They use condition-action rules (e.g., if car-in-front-is-braking then initiate-braking).

• Limitation: These agents only succeed if the environment is fully observable. In partially observable environments, they are prone to infinite loops or irrational choices.

2. Model-Based Reflex Agents

This is the most effective way to handle partial observability. The agent keeps track of the part of the world it cannot currently see. It maintains an internal state that depends on the percept history. Updating this state requires two kinds of knowledge:

1. Transition Model: How the world evolves independently of the agent.
2. Sensor Model: How the agent's percepts derive from the state of the world.

3. Goal-Based Agents

Knowing the current state of the environment is not always enough. For example, at a road junction, knowing you are at a junction doesn't tell you which way to turn. These agents need goals—information that describes desirable situations. They use planning and search to predict the future and choose actions that achieve the goal.

4. Utility-Based Agents

Goals alone only provide a binary "happy" or "unhappy" state. Often, there are multiple ways to achieve a goal, but some are quicker, safer, or cheaper. A utility function maps a state onto a real number, describing the associated degree of happiness. These agents act to maximize their expected utility.

5. Learning Agents

All the agents described above can be built as learning agents. A learning agent consists of several components:

• Learning Element: Responsible for making improvements.
• Performance Element: Responsible for selecting external actions (the traditional "agent" core).
• Critic: Provides feedback on how well the agent is doing against a fixed performance standard.
• Problem Generator: Suggests new exploratory actions to help the agent discover better strategies, rather than just exploiting known ones.

---

Agent Characterisation and Goals

An agent is responsible for satisfying specific goals. These goals can include:

• Achieving an exactly defined status: Reaching a specific final state.
• Achieving constrained status: Reaching a state that satisfies a rule (e.g., "the smallest block is on top").
• Continuously keeping a status: Maintaining an unstable status indefinitely.
• Maximizing utility: E.g., filling a basket with mushrooms that yield the maximum possible price.

The state of an agent includes its internal environment plus its knowledge and beliefs about its external environment.

---

How Components of Agent Programs Work (Representations)

To perform complex reasoning, agents need to represent the state of the world internally. There is an axis of increasing expressiveness in representation: Atomic → Factored → Structured.

1. Atomic Representation

The state is a "black box" with no internal structure.

• Example: In a route-finding algorithm, cities are treated as atomic states identifiable only by their name.
• Usage: Standard search algorithms, game-playing, Hidden Markov Models (HMMs).

2. Factored Representation

Each state is split into a set of variables or attributes.

• Example: A driving problem includes variables like GPS coordinates, gas tank level, and oil warning light status.
• Usage: Constraint satisfaction, propositional logic, planning algorithms, Bayesian networks.

3. Structured Representation

The world is understood as having objects with relationships. It goes beyond simple variables and values to describe dynamic interactions.

• Example: Observing a complex situation like "a truck reversing into a driveway, blocked by a loose cow."
• Usage: Relational databases, first-order logic, natural language understanding.

Trade-off: Higher expressiveness allows for more concise descriptions but requires more complex reasoning and learning algorithms.

Localist vs. Distributed Representation (Concept-to-Memory)

When dealing with memory models (especially neural networks):

• Localist: Each concept is stored in a single, unique memory location (one-to-one mapping). It is simple but not robust to noise; if one unit fails, the entire concept is lost.
• Distributed: A concept is represented across many memory locations, and each location contributes to multiple concepts. It is robust—even if some units are corrupted, the concept can still be approximately recovered.