Prof. Mark Nelson, Fall 2024
This course is an introduction to the field of Artificial Intelligence: the study of architectures, algorithms, and techniques for building machines that either are intelligent, or at least seem to act intelligently, for some definition of intelligence.
AI can be seen as a collection of techniques — search, optimization, planning, evolution, machine learning, logical inference — or as more of a big-picture research agenda. Approaches to AI often mix conceptual and technical vocabulary: Each comes with a set of assumptions and problem formulations that sets up a way of thinking about "intelligence", paired with algorithms and data structures to actually implement it on a computer.
This course is loosely split into two halves, focusing on different goals, though we'll see some overlap in methods between them:
This is mainly a technical computer science class, not primarily about AI policy, business, philosophy, law, or ethics, but we will discuss some of those issues throughout. The way I teach the subject primarily uses code rather than mathematical notation to illustrate AI techniques, so a strong mathematical background is not necessary, but some level of comfort reading and writing pseudocode and Python code is.
No required textbook. We'll use a mixture of textbook chapters, papers, tutorials, technical blog posts, and course notes I've written up in interactive Colab notebooks.
If you do want a general overview book, these two are good, but missing the most recent advances:
Comprehensive reference book and the standard text in the field. Not really something you read start to finish like a book – 1136 pages and quite dense!
A good overview of AI. Doesn't get into mathematics or code, but is readable cover to cover.
As much as possible, coding in this class will be done on the Google Colab platform. This is a free cloud-hosted version of Jupyter notebooks, with Python as the programming language. Its main advantages are that you can easily share notebooks, and it provides free GPUs. If you're curious, I co-wrote a short paper about why I think it's a good fit for AI classes.
Week 1: Introduction
Course overview, formalities, etc. Intro to AI as a field, and some of its history and major schools of thought. Relationship to other fields such as cognitive science, statistics, game theory, logic, operations research, linguistics, and philosophy.
Week 2 (half week): States and actions
One of the things people want AI to do is make decisions. Drive cars, pilot drones, trade the stock market, evaluate business strategies, or play against you in a videogame. General formulation of decision-making in terms of states and actions.
Week 3: Search and planning
A hallmark of AI is very general approaches, versus problem-specific algorithms. Here we look at general ways of formulating and solving decision-making problems. Goal-directed planning; uninformed search; heuristic search. Breadth-first, depth-first, iterative-deepening, and A* search. A look at state space size and scaling.
Week 4: Adversarial search
What happens if there's an opponent actively working against us? Minimax criterion for optimal zero-sum adversarial decision making. Alpha-beta search; Monte Carlo tree search. Variants such as negamax, graph search, etc. Some history on the outsized role of chess in 20th-century AI, and Go and real-time videogames as successors.
Week 5: Reinforcement learning (RL)
So far we've been searching for optimal actions given a known environment (or at least a model of one). An alternate approach is to learn directly from experience. Introduction to the Markov decision process (MDP) formalism; policy and value functions; and several RL algorithms. Examples in the Gymnasium RL framework.
Week 6: Deep reinforcement learning
In practice, classical RL can solve small problems but runs into scaling issues quickly. Deep reinforcement learning pairs RL with a deep neural network to approximate the policy and/or value function. Includes a crash-course introduction to neural networks.
Week 7: Neuroevolution
An alternate approach to training a deep neural network to act in an environment is to "evolve" it, using a computational simulation of the process of evolution by natural selection. Introduction to evolutionary computation, with a focus on applying it to evolving neural networks.
Week 8: Open-endedness
So far we've always given the AI system a specific, quantifiable goal up front: win a game, minimize driving time, etc. Intelligent lifeforms seem to operate in a more open-ended way. Survey of some of the main ideas in open-ended AI. Intrinsic motivation; novelty search; meta-level architectures.
Week 9: History of generative AI
To set the stage, a survey of some systems and methods dating all the way back to the 1950s. Poetry generators; early chatbots; generative art; cybernetic art; evolutionary art; a-life; procedural content generation.
Week 10: Language models
Introduction to modeling natural language text computationally. Text as data; tokenization; language modeling as autoregressive sequence modeling ("predict the next word"). Markov-chain / n-gram language models. Using models for discriminative and generative tasks.
Week 11: Large (neural) language models (LLMs)
Scaling up to modern language models based on (usually) very large neural networks. Vector word embeddings; neural sequence modeling; pre-training; the Transformer architecture. Prompt engineering; in-context learning; evaluating base models.
Week 12: Tuning base models
Specializing a "base" pre-trained LLM to further tasks. Fine-tuning on new data; supervised instruction-tuning; reinforcement learning from human feedback (RLHF). Evaluation again: Turing tests; human-performance benchmarks; debate over what (if anything) a chatbot really "knows".
Week 13: Building bigger systems out of LLMs
Ways to either put LLMs inside another system, or the other system inside the LLM, to build end-to-end AI software. Retrieval augmented generation (RAG); constrained generation; language model crossover; neurosymbolic AI; intelligent agents.
Week 14 (half week): Generative image models
An introduction to diffusion-based generative image models. We can't go as deep into these as we did into LLMs, but they are another large sub-field. Commercial models like OpenAI Dall-E, Adobe Firefly, and Midjourney, and open-source models like Stable Diffusion.
Week 15: Final project meetings
Individual/group meetings and work sessions for your final projects.
468 vs. 668 difference: Students in 668 should have a research-oriented component to the final project (will discuss this individually).
Homeworks and in-class exercises should be submitted individually, but you can consult with other people while working on them. For the final project, I recommend doing it in a group, but individual is ok too.
No exams! Final project presentations will be during the scheduled final exam slot: Thursday, December 12, 2:30-5:00pm.