Prof. Mark Nelson, Spring 2025
This course is an introduction to the field of Artificial Intelligence: the study of architectures, algorithms, and techniques for building machines that either are intelligent, or at least seem to act intelligently, for some definition of intelligence.
AI can be seen as a collection of techniques — search, optimization, planning, evolution, machine learning, logical inference — or as more of a big-picture research agenda. Approaches to AI often mix conceptual and technical vocabulary: Each comes with a set of assumptions and problem formulations that sets up a way of thinking about "intelligence", paired with algorithms and data structures to actually implement it on a computer.
This course is loosely split into two halves, focusing on different goals, though we'll see some overlap in methods between them:
This is mainly a technical computer science class, not primarily about AI policy, business, philosophy, law, or ethics, but we will discuss some of those issues throughout. The way I teach the subject primarily uses code rather than mathematical notation to illustrate AI techniques, so a strong mathematical background is not necessary, but some level of comfort reading and writing pseudocode and Python code is.
No required textbook. We'll use a mixture of textbook chapters, papers, tutorials, technical blog posts, and course notes I've written up in interactive Colab notebooks.
If you do want a general overview book, these two are good, but missing the most recent advances:
Comprehensive reference book and the standard text in the field. 1136 pages!
A good overview of AI. Doesn't get into mathematics or code, but is readable cover to cover.
As much as possible, coding in this class will be done on the Google Colab platform. This is a free cloud-hosted version of Jupyter notebooks, with Python as the programming language. Its main advantages are that you can easily share notebooks, and it provides free GPUs. If you're curious, I co-wrote a short paper about why I think it's a good fit for AI classes.
Week 1: Introduction
Course overview, formalities, etc. Intro to AI as a field, and some of its history and major schools of thought. Relationship to other fields such as cognitive science, statistics, game theory, logic, operations research, linguistics, and philosophy.
Week 2: States and actions
One of the things people want AI to do is make decisions. Drive cars, pilot drones, trade the stock market, evaluate business strategies, or play against you in a videogame. General formulation of decision-making in terms of states and actions. Baseline "AI" method: Brute-force search through state/action space using breadth-first, depth-first, and iterative-deepening strategies.
Week 3: Heuristic search and planning
A hallmark of AI is very general approaches, versus problem-specific algorithms. Can we do better than brute-force search without getting too problem specific? Here we look at general ways of formulating and solving decision-making problems. Goal-directed planning; A* heuristic search; Monte Carlo tree search.
Week 4: Adversarial search
What happens if there's an opponent actively working against us? Then things become recursive: I want to anticipate what they'll do and vice versa. Minimax criterion for optimal zero-sum adversarial decision making. Alpha-beta search and variants (negamax, graph search, etc.). Some history on the outsized role of chess in 20th-century AI, and Go and real-time videogames later on.
Week 5: Reinforcement learning (RL)
So far we've been searching for optimal actions given a known environment. An alternate approach is to learn directly from experience. Introduction to the Markov decision process (MDP) formalism; policy and value functions; and several RL algorithms. Examples in the Gymnasium RL framework.
Week 6: Deep reinforcement learning
In practice, RL can solve small problems but runs into scaling issues quickly. Deep reinforcement learning pairs RL with a deep neural network to approximate the policy and/or value function. Includes a crash-course introduction to neural networks.
Week 7: Neuroevolution
An alternate approach to training a deep neural network to act in an environment is to "evolve" it, using a computational simulation of the process of evolution by natural selection. Introduction to evolutionary computation, with a focus on applying it to evolving neural networks.
Week 8: Open-endedness and project discussion
So far we've always given the AI system a specific, quantifiable goal up front: win a game, minimize driving time, etc. Intelligent lifeforms seem to operate in a more open-ended way. Survey of some of the main ideas in open-ended AI. Intrinsic motivation; novelty search; meta-level architectures. Discussion of final projects.
[spring break]
Week 9: Language models
Introduction to modeling natural language text computationally. Text as data; tokenization; language modeling as autoregressive sequence modeling ("predict the next word"). Markov-chain / n-gram language models. Using models for discriminative and generative tasks.
Week 10: Large (neural) language models (LLMs)
Scaling up to modern language models based on (usually) very large neural networks. Vector word embeddings; neural sequence modeling; pre-training; the Transformer architecture. Prompt engineering; in-context learning; evaluating base models.
Week 11: Tuning base models
Specializing a "base" pre-trained LLM to further tasks. Fine-tuning on new data; supervised instruction-tuning; reinforcement learning from human feedback (RLHF). Evaluation again: Turing tests; human-performance benchmarks; debate over what (if anything) a chatbot really "knows".
Week 12: Building bigger systems out of LLMs
Ways to either put LLMs inside another system, or the other system inside the LLM, to build end-to-end AI software. Retrieval augmented generation (RAG); constrained generation; language model crossover; neurosymbolic AI; intelligent agents.
Week 13: Generative image models
An introduction to diffusion-based generative image models. We can't go as deep into these as we did into LLMs, but they are another large sub-field. Commercial models like OpenAI Dall-E, Adobe Firefly, and Midjourney, and open-source models like Stable Diffusion.
Week 14: Final project meetings
Individual/group meetings and work sessions for your final projects.
468 vs. 668 difference: Students in 668 should have a research-oriented component to the final project (will discuss this individually).
Homeworks and in-class exercises should be submitted individually, but you can consult with other people while working on them. For the final project, I recommend doing it in a group, but that is not required.
No exams! Final project presentations will be during the scheduled final exam slot: Friday, May 2, 11:20am-1:50pm.