Applied Bayesian Statistics
Political Science 435
Spring, 2003
Professor: Jeff Grynaviski
Office: Pick 528
Email: grynaviski@uchicago.edu
Webpage: http://home.uchicago.edu/~grynav
Phone: 773-702-2370
Course Description
This course is designed as an introduction to Bayesian data analysis. To be clear, this course does not aspire to provide students with a smorgasbord of fancy new statistical tools to play with. Instead, it is designed to introduce you to an entirely different mode of inference that incorporates your beliefs about the world and the data that you collect to further inform those beliefs into a coherent whole. Unfortunately, this means that for much of the quarter we will be teaching you a different (and in some respects harder) way of doing really simple things that you already know how to do. However, by starting with very simple models, we will find stony ground on which to build increasingly complex models, many of which could not be estimated without the set of computational tools and inferential framework developed by applied Bayesians. By the end of the term, you should be in a position to extend the Bayesian approach in a wide assortment of directions that you can tailor to your own interests.
Required Course Materials
Peter Congdon. 2001. Bayesian Statistical Modeling (available at the Coop once the publisher replenishes its stock).
WinBugs 1.4. Downloadable from http://www.mrc-bsu.cam.ac.uk/bugs/winbugs/contents.shtml
Other Useful Texts
Gelman, Carlin, Stern, and Rubin, Bayesian Data Analysis
Gilks, Richardson, Spiegelhalter (editors), Markov Chain Monte Carlo in Practice
Gill, Bayesian
Methods: A Social and Behavioral Sciences Approach
Lee, Bayesian Statistics:
An Introduction
Course Assignments
Course grades will be based in equal parts on the following
three criteria:
Class participation and attendance.
The course is designed as an introduction to an applied methodology, not as an opportunity to read and discuss the great works. Unlike many graduate classes, this means that you might learn something in class that you wouldn’t get from the readings. It also means that the bulk of class time will be spent in lecture. This is not to say that discussion or questions are not encouraged. I will try to reserve some time for discussion and you are always, always encouraged to ask questions. The best questions usually takes the form of something like, “huh?” or “I don’t understand that?” If you don’t understand, chances are others are in the same boat, so do not be timid about asking questions.
6 or 7 homework assignments.
In addition to the regular reading assignments and working through the WinBugs code that comes with Congdon’s text, you will be expected to do a number of (mostly) computer assignments designed to get you familiar with the tools for applied Bayesian work.
Independent research project.
Students will be expected to write a fully Bayesian research paper where they apply a technique that they learned in class to a research question that interests them. Miracles are not expected—a Bayesian multiple regression would be more than adequate, especially since you may run into computational problems beyond the purview of the class with more sophisticated applications.
To facilitate finishing the project, I strongly encourage each of you to pick a data set the first week of classes that you will use for the project. Then, as you work through the homework exercises, to apply the techniques from the homework to your data set. By the end of the term, all you will have to do to finish is write an introduction and conclusion and staple the results together.
In choosing your data set, I recommend that you pay attention to the following criteria. One, identify a data set that is standard in your field of study and which scholars have used for simple analyses (i.e. multiple regression; logistic regression; event count models). Two, there should be different populations within the data set whose behavior may not “pool” (e.g. normal people and folks from the South; developed and lesser developed countries). Three, there should be some dependent variable in the data set for which there are competing extant theories in the literature.
Note: I do not accept
incompletes unless you have a personal crisis of some sort.
Tentative Course Schedule
Introductory Lecture
(Powerpoint Viewer)
Lavine. 2000. “What is Bayesian statistics and why everything else is wrong”
Western, Bruce and Simon Jackman. 1994. “Bayesian Inference for Comparative Research.” American Political Science Review 88: 412-423. JSTOR
- This piece provides a nice, easy to read, application of Bayesian methods. It will give you a pretty good feel for what’s in store.
Review of probability
theory and the Bayesian Setup (Powerpoint)
Congdon, Chapter 1.
Degroot, Probability and Statistics. Chapter 1 and 2.1 – 2.2. Reg. Reserve/Photocopy
Definitti. 1937. “Foresight: Its Logical Laws, Its Subjective Sources.” Photocopy
- This is an optional paper for those of you interested in intellectual history. This paper provided the intellectual foundations for subjective probability theory.
Review of Integral
Calculus and Univariate PDFs
and CDFs
Kleppner. Quick Calculus. Chapter 3. Online Reserve.
Degroot, Probability and Statistics Chapter 3.1 – 3.3. PDFs and CDFs. Reg. Reserve.
Homework 1-Due
Friday, April 18, 2003
Multivariate
Probability Models
Degroot, Probability and Statistics
Chapter 3.4 – 3.7. PDFs and CDFs. Reg. Reserve.
Part II. Bayesian Analysis of Basic Models
Bayesian analysis of
one-parameter models
Congdon, Chapter 2.5-2.6
Gill. 2002. Bayesian Methods. Chapter 3.
Homework 2-Due
Friday, April 25, 2003
Bayesian analysis of
the one-parameter normal model with WinBugs introduction
Congdon, Chapter 2.1-2.2
- Excel implementation of MCMC
Congdon, Chapter 2.3-2.4
Gilks, Richardson, and Spiegelhalter.
1996. “Introducing Markov Chain
Gill, ND. “A Primer on Markov Chain Monte Carlo.” (GS-view)
Jackman,
Simon 2000. "Estimation and Inference via Bayesian
Simulation: An Introduction to Markov Chain
- This paper is optional for now given that its purpose is to show how to estimate more complicated models than we are talking about in class, but it is still quite useful if given a close read.
Homework 3-Due
Slave Revolts Data—revised
for WinBugs
Hierarchical Models
and Bayesian Shrinkage
Congdon, Chapter 5
Bernardo, Jose M. 1984. “Monitoring the 1982 Spanish Socialist Victory: A Bayesian Analysis.” Journal of the American Statistical Association 79: 510-515. JSTOR
Homework 4-Due
Part III. Bayesian Analysis of the General Linear
Model
Bayesian Regression
with improper and conjugate priors
Congdon, Chapter 4.
Bayesian Regression
with conjugate and convenient priors
Western, Bruce. “Unionization and Labor Market Institutions in Advanced Capitalism, 1950-1985.” American Journal of Sociology 99:1314-1341.
Gelman. 1996. “Inference and Monitoring Convergence.” In MCMC in Practice.
The Choice of Priors
and Bayesian Hypothesis Testing
Berk, Richard A., Bruce Western, and Robert E. Weiss 1995. "Statistical Inference for Apparent Populations." Sociological Methodology 25:421-458.
Bollen. 1995. “Apparent and Nonapparent Significance Tests.” Sociological Methodology 25:459-268.
Firebaugh. 1995. “Will Bayesian Inference Help? A Skeptical View” Sociological Methodology 25:469-472.
Rubin. 1995. “Bayes, Neyman, and Calibration.” Sociological Methodology 25:473-479.
Berk, et al. “Reply” Sociological Methodology 25: 481-485.
JSTOR
Congdon, Chapter 10.
Hierarchical Linear
Models, part I
Hierarchical Linear
Models, part II
Congdon, Chapter 8
Spiegelhalter, et al. 1996. “Hepatitis B: a case study in MCMC methods.” In MCMC in Practice.
Western, Bruce. 1998. “Causal Heterogeneity in Comparative Research: A Bayesian Hierarchical Modeling Approach.” American Journal of Political Science 42: 1233-1259. JSTOR
Lindley and Smith. 1972. “Bayes Estimates for the Linear Model.” Journal of the Royal Statistical Society, Series B, 34: 1-41. (JSTOR Optional)
Homework
5. Due May 29, 2003.
Data for homework 5.
Congdon, Chapter 8
Quinn, Martin, and Whitford. 1999. “Voter choice in multi-party democracies: A test of competing theories and models.” American Journal of Political Science 43: 1231-1247. JSTOR
Martin 2001. “Congressional Decision Making and the Separation of Powers.” American Political Science Review 2001. JSTOR
Example: WinBugs code and data for a Poisson regression model with example of slow convergence.
King, et al. 2001. “Multiple Imputation of Missing Data.” American Political Science Review 95: 49-69. JSTOR
Latent Variable
Models
Jackman. 2000. “Estimation and Inference are Missing
Data Problems: Unifying Social Science Statistics via Bayesian Simulation.” Political
Analysis 8:4, pp. 307-332.
Advanced Topics
ecological inference; latent variables