PP5105 Public Policy Research and Evaluation
Instructor: Dr. Wu Xun
Office Hours: Wednesday 2:00-4:00 PM
This course introduces students the basic concepts and methods in social science research and their applications in public policy analysis. It focuses on the development of students’ professional skills in research design, data collection techniques, data analysis, and reporting and presentation of research findings. Main topics for the course include validity of research, measurement, experiments, sampling, survey research, qualitative data collection techniques and impact assessment. A strong emphasis will be given to cost-benefit analysis, an important tool for policy research and evaluation.
The objectives of the course are to: 1) familiarize students with main research methods commonly used in public policy research and evaluation; 2) enable students to formulate a general research strategy for a given policy problem; 3) enable students to design and implement field work for policy research; 4) enable students to conduct cost-benefit analysis for a particular policy or program.
The required textbooks are:
O’Sullivan, E., G. R. Rassel and M. Berner. Research Methods for Public Administrators. 4th edition, Addison Wesley Longman, 2003 [ORB].
Boardman, A. E, D. H. Greenberg, A. R. Vining and D. L. Weimer. Cost-Benefit Analysis:
Concepts and Practice, Second Edition. Prentice-Hall, Upper Saddle River, NJ, 2001.
Course Requirements and Grading Criteria
There will be four assignments and a group project during the term. All four assignments will require the students to individually prepare professional memoranda that apply techniques developed in the course to particular situations. The group project—also referred to as the class enrichment exercise in this course—requires the students to work in groups of four on particular topics and they are expected to present their work to the class.
There will be one exam in the course—a final. The exam will be open book.
In determining course grades, the following weights will be used:
Four Assignments 40%
Group Project 10%
Class Participation 10%
Final Exam 40%
PP5105 Public Policy Research and Evaluation
Topics and Reading Assignments
(Class 1) Introduction to Policy Research
(9:00-12:00 AM, Wednesday, 10 January)
Booth, W. C., G. G. Colomb, and J. M. Williams. The Craft of Research. The University of Chicago Press. 2003.
Chapter 3 From Topics to Questions
Chapter 4 From Questions to Problems
Chapter 7 Making Good Arguments: An Overview
(Class 2) Research Designs for Policy Research and Evaluation (1)
(9:00-12:00 AM, Wednesday, 17 January)
Chapter 1 Beginning a Research Project: The Preliminary Steps
Chapter 2 Designs for Description (pp. 24-37)
Chapter 3 Designs for Explanation (pp. 55-66)
Case: Portland Housing Market
Assignment: Come to class prepared to explain whether or not you would support the proposed expansion of the UGB boundary by 17,000 acres, mostly in southeast Clackamas County. In thinking about your decision, please consider:
- What benefits supporters claim for the UGB? What are the costs detractors attribute to the UGB?
- In theory, how would UGB affect housing prices?
- What empirical evidences do you have in evaluating the claim that UGB has caused long-run pressure on housing prices in Portland? Are these evidences convincing?
- What other data would be useful in determining whether or not UGB has caused long-run pressure on housing prices in Portland?
Lab session #1: Univariate Analysis
[ORB] Chapter 11 Univariate Analysis
(Class 3) Research Designs for Policy Research and Evaluation (2)
(9:00-12:00 PM, Wednesday, 24 January)
[ORB] Chapter 3 Designs for Explanation (pp. 66-92)
Card, D. and A. Krueger. Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania, American Economic Review, 84(4), September 1994.
Tan, J. P., J. Lane and G. Lassibille. Student Outcomes in Philippines Elementary Schools: An Evaluation of Four Experiments, The World Bank Economic Review, Vol. 13, No. 3, 493-508.
Lab session #2: Statistical Tests
[ORB] Chapter 12 Examining Relationships among Variables: Tests of Statistical Significance
(Class 4) Research Design Workshop
(9:00-12:00 AM, Wednesday, 31 January)
Case: Public Transit and Minority Employment
Assignment: come to class prepared to discuss the following questions:
- What is the research question that Steve is seeking to answer? What theories help you to understand the potential answers to the question? Are there issues that the theories do not address?
- What hypotheses are suggested by the question and how would you test them?
- What research method(s) would you use (experiments, quasi-experimental designs, non-experimental approaches, etc.)? What sort of data would be useful (cross-sectional, trend, panel etc.)? What unit of analysis would you use (individual, family, household, etc.)?
Lab session #3: Bivariate Analysis
[ORB] Chapter 13 Examining Relationships among Variables: Contingency Tables with Measures of Association, Analysis of Variance
(Class 5) Measurement in Policy Research
(9:00-12:00 AM, Wednesday, 7 February)
[ORB] Chapter 4 Measuring Variables
Palfrey, C., C. Phillips, P. Thomas, and D. Edwards. Policy Evaluation in the Public Sector: Approaches and Methods. Avebury, Vermont, USA. 1992. Chapter 9 Evaluating Equality and Equity.
Coudouel, A., J. Hentschel and Q. Wodon (2001) 'Well-being Measurement and Analysis', in: World Bank, A Sourcebook for Poverty Reduction Strategies. Washington D.C.: World Bank.
First Assignment due
Lab session #4: Multivariate Analysis
[ORB] Chapter 14 Regression Analysis and Correlation
(Class 6) Qualitative Data Collection Techniques
(9:00-12:00 AM, Wednesday, 14 February)
Chapter 2 Designs for Description (pp. 37-47)
Chapter 6 Contacting and Talking to Subjects
Caudle, Sharon L. “Using Qualitative Approaches,” in Wholey, Joseph S. (et al.) Handbook of Practical Program Evaluation. Jossey-Bass Publishers, San Francisco, 1994. pp. 69-95.
Darlington, Yvonne and Dorothy Scott. Qualitative Research in Practice: Stories from the Field. Allen & Unwin, Australia, 2002. Chapter 3 In-depth Interviewing, pp. 48-73
Lab session #5: The Use of Multivariate Analysis in Policy Research
Wu, Xun. “Corporate Governance and Corruption: A Cross-Country Analysis,” Governance: An International Journal of Policy, Administration and Institutions, Vol. 18, No.2, 151-170, 2005.
Wu, Xun, Julian Lampietti and Anke Myers. “Coping with the Cold: Space Heating and the Urban Poor in Developing Countries,” Energy Economics 26, 2004.
(February 17-25 Mid-semester break)
(Class 7) Quantitative Data Collection Techniques
(9:00-12:00 AM, Wednesday, 28 February)
Chapter 5 Sampling
Chapter 7 Data Collection: Questions and Questionnaires
Case: Conducting Social Research in Hong Kong
Lab session #6: Survey Design Workshop
Second Assignment due
(Class 8) Impact Assessment in Policy Evaluation
(9:00-12:00 AM, Wednesday, 7 March)
Rossi, Peter and Howard Freeman. Evaluation: A Systematic Approach. 6th Edition. Sage Publications, 1999. Chapter 5—Strategies for Impact Assessment
Ravallion, Martin. The Mystery of the Vanishing Benefits: An Introduction to Impact Evaluation, The World Bank Economic Review, Vol. 15, No. 1, 115-140.
(Class 9) Introduction to Cost-Benefit Analysis (CBA)
(9:00-12:00 AM, Wednesday, 14 March)
Chapter 1 Introduction to Cost-Benefit Analysis
Chapter 2 Conceptual Foundations of Cost-Benefit Analysis
Chapter 3 Basic Microeconomic Foundations of Cost-Benefit Analysis
Case: Liconsa and the Program of Social Assistance for Milk
Assignment: Come to class prepared to discuss the following questions:
- Do you think the Liconsa milk program is effective in helping the poor? What are the costs to the government or taxpayers of the program? What are the benefits to the participating families and how would they value them? Are there benefits or costs to parties other than the government and participating families?
- Do you think the Progresa/Oportuindades program is effective in helping the poor? What are the costs to the government or taxpayers of the program? What are the benefits to the participating families and how would they value them? Are there benefits or costs to parties other than the government and participating families?
Lab session #7: Advanced Skills in Excel
Third Assignment due
(Class 11) Identifying Costs and Benefits in Cost-Benefit Analysis
(9:00-12:00 AM, Wednesday, 21 March)
Chapter 4 Valuing Benefits and Costs in Primary Markets
Chapter 5 Valuing Benefits and Costs in Secondary Markets
Case: The Hong Kong-Macau-Zuhai Bridge Proposal
Assignment: Come to class prepared to explain whether or not you would support the proposed Hong Kong-Macau-Zuhai Bridge. In thinking about your decision, please consider:
- What are the advantages of the bridge cited by its supporters? What are the disadvantages cited by its critics?
- Suppose you were doing a financial analysis of the bridge from the perspective of a potential private concessionaire. What would you include as benefits and what as costs?
Suppose you were doing a social benefit cost analysis of the bridge. What would you regard as the bridge’s benefits and what as its costs? Would you value them at market prices?
Lab session #8: CBA Exercises
(Class 12) Discount Rate and Decision Rule in Cost-Benefit Analysis
(9:00-12:00 AM, Wednesday, 28 March)
[BGVW] Chapter 6 Discounting Future Benefits and Costs
Case: West Side Highway Proposal
Assignment: Imagine that you are a staffer to Transportation Secretary William Coleman and that he has asked you to evaluate the cost-benefit analysis in the FEIS and explain to him whether it supports the recommendation to build the modified outboard expressway rather than one of the smaller and less expensive alternatives. In formulating your answer, please consider:
1. What categories of costs and benefits should be considered in deciding whether or not to build the West Side Highway? Are all relevant costs and benefits listed in case Exhibit 2? Do any omissions of appropriate benefits or costs, or inclusions of inappropriate ones, bias the recommendations?
2. Are the benefits and costs in Exhibit 2 properly valued?
Lab Session #9: Discount Rate
(Class 13) Incorporating Risk and Uncertainty in Cost-Benefit Analysis
(9:00-12:00 AM, Wednesday, 4 April)
[BGVW] Chapter 7 Dealing with Uncertainty: Expected Value, Sensitivity Analysis and Value of Information.
(Class 14) Valuation Techniques for Cost-Benefit Analysis
(9:00-12:00 AM, Wednesday, 11 April)
Chapter 13 Valuing Impacts from Observed Behaviour: Other Revealed Preference Methods
Chapter 14 Valuing Impacts through Surveys: Contingent Valuation
Chapter 15 Shadow Prices from Secondary Sources
Lab session #10: Sensitivity Analysis
Fourth Assignment due
Appendix A: List of Class Enrichment Exercises:
1. Constructing Indices for Policy Research (07/02)
2. Measuring Inequity in Health (07/02)
3. In-depth Interview(14/02)
4. Focus Group (14/02)
5. Meta-analysis (28/02)
6. Qualitative Data Analysis: Case Study (28/02)
7. Qualitative Data Analysis: Boolean Approaches (07/03)
8. Qualitative Data Analysis: Fuzzy-set Methods (07/03)
9. The Use of Discrete Choice Models in Policy Research (14/03)
10. Framing Effects in Policy Research (14/03)
11. Optimization Models in Policy Research (21/03)
12. Decision Tree Analysis (04/04)
13. Monte Carlo Simulation (04/04
14. Multi-attribute Analysis (11/04)