Rationality in AI: Rational Agents, Decision Theory, and Limits
Rationality in AI is often defined by the principle of maximizing expected utility. Decision theory formalizes this principle, but bounded rationality recognizes that real agents have limited information and computational power. AI systems apply reasoning using knowledge bases and inference engines.
What is a rational agent in AI?
The concept of a rational agent is central to artificial intelligence. The textbook Artificial Intelligence: A Modern Approach includes chapters on intelligent agents and on making simple and complex decisions, reflecting the field's focus on rational choice.
The introductory AI course notes at UC Berkeley emphasize decision making as one of the core competencies of intelligent systems, along with problem solving, reasoning, and learning, and they place these activities on logical and probabilistic foundations.
A rational agent is typically defined as one that chooses actions to maximize expected utility, given its beliefs and desires. Decision theory provides the normative framework for such choices, focusing on how preferences and uncertainty should be handled.
Expected utility theory
According to the Stanford Encyclopedia of Philosophy on decision theory, the standard normative decision theory, called expected utility theory, holds that under uncertainty, an agent should choose the option with the highest expected desirability or value.
The theory requires that an agent's preferences over prospects satisfy certain axioms, including completeness and transitivity. Completeness means that for any two options, the agent can compare them; transitivity ensures that consistent ordering is maintained across multiple options.
These axioms allow the construction of a utility function that represents preferences numerically, so that choices can be evaluated by computing expected values.
Bounded rationality
Herbert Simon introduced the term bounded rationality to describe a concept of rational behavior that fits the actual information and processing limits that organisms face, as explained in the Stanford Encyclopedia of Philosophy on bounded rationality.
Simon's Nobel Prize citation highlights his pioneering research on decision-making within economic organizations, noting that he argued that all people deviate from strictly rational choices when making decisions, and that companies act as adaptable systems with physical, personal, and social components, as recorded by the Nobel Prize organization.
Bounded rationality often leads to satisficing, a strategy where an agent chooses the first available option that meets a minimum threshold, rather than exhaustively searching for the optimal one.
Reasoning in AI systems
AI reasoning systems typically consist of two core components: a knowledge base and an inference engine, according to IBM's explanation of AI reasoning. The knowledge base stores structured knowledge about the domain, while the inference engine applies logic and reasoning methods to derive conclusions.
IBM describes several types of reasoning used in AI, including deductive reasoning, which draws specific conclusions from general facts, and abductive reasoning, which formulates the most likely explanation for observed data. Analogical reasoning transfers knowledge from similar past situations, and commonsense reasoning applies general knowledge about the world.
Although AI reasoning aims to mimic human reasoning, IBM notes that AI still needs considerable work to truly reason like humans due to the dynamic and uncertain nature of many real-world problems.
| Aspect | Perfect rationality | Bounded rationality |
|---|---|---|
| Information | Complete information about options | Limited access to relevant information |
| Computational capacity | Ability to solve complex optimization | Restricted computational capacities |
| Decision rule | Maximize utility | Satisficing: choose first acceptable option |
Frequently asked questions
What is a rational agent in artificial intelligence?
A rational agent is one that chooses actions to maximize expected utility according to its beliefs and desires, as formalized by decision theory.
What is expected utility theory?
Expected utility theory is a normative decision theory that says one should prefer the option with the greatest expected desirability, calculated by summing the utilities of possible outcomes weighted by their probabilities.
What is bounded rationality?
Bounded rationality is the idea that agents have limited access to information and computational capacity, so they often use satisficing rather than strict optimization to make decisions.
How does AI reasoning work?
AI reasoning typically uses a knowledge base to store structured information and an inference engine to apply logical or probabilistic methods to generate predictions and conclusions.