Estimating the Influence of Financial Aid on Student Retention
Serge Herzog, Ph.D.
January 17, 2008
The greatest limitation in establishing causality in observational studies on the effect of financial aid is the presence of endogeneity or selection bias associated with aid status. To control for this statistical confoundedness that besets the research corpus to date, this study estimates the effect of financial aid on freshmen retention at a moderately selective, public university using propensity score-matching in multi-stage regression analyses. The correlational pattern that emerged from twenty-four logit models suggests higher-income students accrue a retention benefit from financial aid, unlike low-income students, net of first-year academic experience and type and amount of aid received. Conversely, retention of low-income freshmen is more likely due to academic performance compared to those from high-income background. Findings on the effect of aid are consistent with the economics of moral hazard and unobservable behavior.
However, the finding that well-prepared students from economically disadvantaged background cannot afford to enter a higher education institution hinges on the definition of “academically prepared” and on what constitutes “sufficient” enrollment of low-income students. ACSFA defines “college prepared” as “minimally qualified” on the basis of a student’s highest score in terms of high school grades (GPA), class rank, college admission test (ACT/SAT score), or the national college-readiness index (ACSFA, 2002; Berkner & Chavez, 1997). Accordingly, a GPA of 2.7 or an ACT score of 19 is considered “minimally qualified” (Greene and Winters, 2005). For example, the University of Nevada, Reno, a moderately selective flagship institution, requires a minimum 2.75 GPA for admission; the California State University system, encompassing the state’s less selective universities, requires students with a verbal ACT test score of less than 25 (on a scale of 36) to take remedial courses. Hence, the ACSFA study overestimates the number of college-prepared high school graduates. Applying three, more meaningful criteria to define “college ready,” Greene and Winters (2005) estimate that the 4-year college-going rate among US high school graduates surpasses their readiness rate on the basis of 2001 through 2003 national enrollment numbers (NCES, 2005). In short, more high school graduates enter 4-year institutions than are estimated to be prepared. Similarly, closer examination of Gerald and Haycock’s data source (Hill and Winston, 2006) reveals that the 50 flagship universities (one per each US state) captured 38% of the nearly 110,000 low income students that took the ACT/SAT nationwide in 2003 and were well prepared according to Gerald and Haycock (i.e., scored at least a 24 on the ACT or 1110 on the SAT). Since the flagships make up less than 2% of the total number of over 2,800 US colleges and universities that confer a Bachelor’s or higher degree, their 38% share of well-prepared low income students suggests are rather impressive capture of the materially disadvantaged. Judging an institution’s success in catering to low-income solely on the percentage of enrolled students that received a Pell grant (the largest aid program for low-income students) is equally flawed due to institutional variations in program offerings, the proportion of Pell-eligible students enrolled at any given time (e.g., foreign and non-degree-seeking students are ineligible), and demographic differences in the primary geographic recruitment area (Tebbs and Turner, 2005).
A recent analysis of nationally representative enrollment data of college students in the United States fails to identify entry barriers for prepared low-income students and instead stresses the lack of enrollment persistence following initial college matriculation among such students. Lack of continued college participation of low-income students is attributed primarily to inadequate academic preparation and student self-reported reasons other than financial aid (Adelman, 2007). However, the influence of aid on student enrollment persistence is mediated by a complex web of interrelated factors: including the timing, type, and amount of aid and how they correlate with persistence in the presence of other student attributes (Pascarella and Terenzini, 2005). In spite of voluminous research on student success, enrollment retention models yield little insight to evaluate the impact of financial aid. The problem stems from a preoccupation with, first, descriptive studies that fail to control for the multiple influences that govern student behavior; second, studies that employ inferential statistics without subject randomization or controls for treatment selection bias; and third, studies that omit to underpin quantitative findings with qualitative explanations to guide institutional interventions for enhancing student success (St. John, 2006).
where (1) assumes a set of covariates is reduced to a scalar function to yield a propensity score p (X) in treatment assignment D, leading to a condition of ignorable treatment selection, while (2) expresses the balancing property of propensity scores, where observations for both treatment and control group within subclasses (strata) have similar or identical values of p (X).
Scores are estimated via a non-ordered multinomial logistic regression model with aid status as the dependent variable and socio-demographic attributes, pre-collegiate academic preparation, college major, first-year credit load, campus residency, and timing of the university-entry test as independent variables (Table 1). Correlation with the need for financial aid governs the selection of these variables. For example, income background and academic preparation likely influence the need for support and eligibility for different types of aid. Similarly, how early freshmen take the admission test prior to enrollment indicates their intent to enter college and may likely influence the range of available aid as students get admitted.
Though propensity score-matching may not fully account for endogeneity bias in aid status (Lunceford and Davidian, 2004; Rubin, 2004; Rosenbaum and Rubin, 1984; Titus, 2006)—particularly in the presence of omitted variables strongly related to the outcome and uncorrelated with the propensity score—matching based on a minimum of five strata is estimated to remove 90 percent of bias associated with observed covariates (Cochran, 1968). Table 2 shows little imbalance in the propensity for aid and the distribution of observed characteristics associated with aid. Following balance verification techniques by Lee (2006) and Agodini and Dynarski (2004), of the 120 Bonferroni-adjusted t-tests, including tests for each of 20 covariates across 6 strata, only 4.2% (5 out of 120) showed a significant difference between gift-aided and unaided students; slightly larger (9%) is the imbalance in covariates between students with loans and those with no aid. Five percent of all tested covariates may differ by chance alone. Matching students with only gift aid to those with no aid required an extra split in one stratum to ensure sufficient covariate balance without adding interaction or quadratic terms to the scoring model, as recommended by Dehejja and Wahba (2002). To maximize the likelihood of matches, observations were divided into strata of equal score range. The percentage of matched cases ranged from 91 to 97 for models comparing gift-aided to unaided students, and 83 to 93 for models comparing students with loans in their aid package to unaided students. Thus, propensity score pairing resulted in few case deletions due to the large common support area, as illustrated with box plots in Figures 1 and 2. Using observations within the common support area that match aided with unaided students on the basis of their propensity to receive aid, the study is designed to offer a more accurate estimate of the true effect of aid on student persistence compared to conventional multivariate models (e.g., Herzog, 2005; Dowd, 2004; Somers, 1995; Stampen and Cabrera, 1988; Voorhees, 1985).
Results are based on approximately 5,000 first-year students at a public university in the United States and measure the correlation between the likelihood of receiving aid (i.e., the propensity score stratum) and second-year retention both before and after taking into account first-year math experience and cumulative grade point average (GPA).
Stepwise inclusion of these two variables is critical in estimating the influence of aid on student retention, since academic performance and success in math have been identified as key determinants in enrollment persistence (Adelman, 2004, 1999; Pascarella and Terenzini, 2005; Herzog, 2005). Variable entry at the second step also includes controls for the amount and type of money received and the amount of remaining need after accounting for all aid received, which in turn dictated that the cohorts analyzed be limited to spring-retained freshmen that entered the previous fall, since aid is typically allocated on an annual basis. Moreover, students whose income background could not be ascertained from the three data sources used—namely the institutional student information system, ACT’s Student Profile Section (SPS), and the College Student Survey administered by the Cooperative Institutional Research Program (CIRP)—were excluded, as well as foreign students and varsity athletes—yielding 71% of all freshmen in the cohorts. To identify the presence of selection bias, results from matched students are juxtaposed with those from unmatched students. Because merit-based financial aid is awarded to academically stronger students, the analysis included collinearity tests for all variables to ensure regression models were not overspecified. The less intuitive odds ratio is interpreted in terms of percentage change—using a linear transformation of the log odds (p*[1-p]*β) per Morgan and Teachman (1988)—in the probability of second-year re-enrollment in the discussion of model results. All statistical output was generated with SPSS 14.0 for Windows.
The positive effect of aid is cancelled out, however, after taking into account academic performance. Without considering the EFC, a one letter-grade rise in GPA correlates with a 12% increase in persistence probability, while doing well in advanced math raises that probability by an additional 5%. Posing the counterfactual argument, had an aided freshman not received grant and/or scholarship support (the ‘matched average treated’ column), the chance for persistence would have dropped on average by an estimated 3% (3.06), or slightly more than the average freshman (2.94); conversely, had an unaided freshman received such support (the ‘matched average untreated’ column), the persistence benefit would have been less than students on average (2.76 vs. 2.94). But in either case, a student’s GPA and math performance would have rendered the change in aid status statistically inconsequential on the potential for persistence.
To gauge the influence of financial aid on retention of students from different income background, the study furnishes separate estimates by EFC level. An EFC of less than $4,000 includes students from low-income backgrounds that are eligible for federal need-based grants, such as Pell Grants and Supplemental Education Opportunity Grants. On average, they received between $2,000 and $3,000 in such aid for the first year (Table 8). Middle income students with identified financial need are typically those with an EFC between $4,000 and $10,000 ($9,700 in adjusted 2005-$). They derive need-based assistance from state and institutionally funded sources, receiving on average between $800 and $900 (Table 8). In contrast, students with an EFC of over $10,000 ($9,700 in adjusted 2005-$) have little or no identified financial need and, therefore, rely principally on merit-based aid and unsubsidized loans (Table 8). Notably, low-EFC students face considerably higher remaining need after factoring in all aid received in comparison to their high-EFC peers, likely due to some constraints in taking on greater loan aid. Hence, the following results control for amount and type of aid as well as net remaining need. For 982 freshmen, the EFC could not be determined, since they did not fill out a federal aid application form (FAFSA).
As Table 3 shows, there is no correlation between the chance of receiving gift aid and retention of low-income freshmen (EFC of less than $4,000) net of GPA and math experience, and the propensity for receiving aid remains insignificant regardless of low-income students’ level of academic success (i.e., insignificant GPA-propensity score level interaction). Similarly, the amount and type of aid does not correlate with retention of such students after factoring in their academic experience (Table 4). However, a one letter-grade increase in GPA is associated with a 15% rise in retention of low-income freshmen receiving aid, but doing well in math does not enhance their retention when controlling for aid selection bias.
For freshmen expected to carry a greater share of the college attendance cost, namely those with an EFC of $4,000 to $10,000, neither the likelihood of receiving gift aid nor the amount or type of aid correlates with second-year persistence net of GPA and math experience. However, after taking into account aid self-selection bias, doing well in advanced math tends to improve (alpha 0.08; Table 4) their retention probability by almost 10% over non-remedial students of similar income, who only pass a college algebra course. As with low-income students, retention of middle-income freshmen is positively associated with first-year grades, with an estimated 10% change in the probability of second-year return corresponding to a one letter-grade change in the GPA.
Freshmen benefiting from grant and/or scholarship aid are those faced with more than $10,000 ($9,700 in adjusted 2005-$) in EFC and are typically ineligible for need-based financial assistance. Gift aid may improve their persistence chance by 18% (6 x 3.01), after controlling for GPA and math experience (Table 3), and for every $1,000 in Millennium scholarship aid, retention of such students is estimated to rise on average by almost 3% (2.91, Table 4). The counterfactual proposition suggests that without the gift aid, recipients would have experienced a slightly greater drop in persistence than students on average (3.26 vs. 3.01, Table 3). The average potential decline in persistence is only minimally affected by the amount of gift aid high-EFC students received (3.26 vs. 2.98 in Tables 3 and 4). Curiously, high-EFC freshmen who managed to complete only remedial math in their first year were 11% more likely to persist vis-à-vis those with the typical math experience (i.e., who pass a first-year college algebra course).
Retention of freshmen with an unknown EFC shows no correlation with the propensity for receiving grants and/or scholarships. A one letter-grade rise in GPA is associated with a 15% increase in their persistence (Table 3), and performing well in math may further improve their chance to return for the second year (alpha 0.79, Table 4). Notably, a regression-based analysis without propensity score matching suggests a negative correlation between receipt of gift aid and retention, a result that is not supported when matching aided and unaided students on the likelihood of receiving such aid (Table 3).
To test whether these findings on the linkage between ability to pay (EFC) and the influence of gift aid on retention are an artifact stemming from the choice of EFC groupings, separate estimates based on the remaining financial need freshmen faced (i.e. the amount calculated after subtracting the EFC from the total cost of attendance) were calculated. Accordingly, students with no remaining need could not qualify for need-based financial aid, as their EFC covered the entire cost of attendance; those with remaining need were eligible for such aid due to their limited ability to pay for the cost of attendance. A student’s financial burden is assessed more accurately after factoring in all aid received. Therefore, grouping freshmen by remaining need controls for the net cost based on all aid received. Freshmen with no remaining need after the EFC exhibit a significant positive relationship between the likelihood of receiving gift aid and second-year retention, net of aid self-selection bias and academic experience. In turn, the propensity for receiving grants and/or scholarships shows no significant association with retention of students with remaining need (Table 5). But, doing well in math correlates with a 9% to 12% increase in retention for those with remaining need; in contrast, freshmen with no remaining need fail to show an association between doing well in math and second-year persistence. Together, these results parallel the correlations found along EFC-grouped students in Tables 3 and 4. Similarly, the finding that remedial math students with a high EFC are more likely to persist is echoed in Table 5, where remedial freshmen with no remaining need are 18% more likely to persist after failing first-year math compared to the typical freshman who passed first-year college algebra.
To extend the analysis to students who borrowed money in order to matriculate, the impact of aid on second-year retention of freshmen with loans in their aid package is estimated in Tables 6 and 5. As is the case with students supported by gift aid only, the probability of receiving aid that includes loan money has no bearing on second-year return of low-EFC freshmen, whether unmatched or matched on propensity for aid support (Table 6). Though institutional grant aid for low-income students is positively correlated with their persistence (a 7% rise per $1,000), as is Millennium scholarship aid and other merit-based aid, the amount and type of aid no longer matters after taking into account first-year grades (GPA) and math experience (Table 7). As before, retention of low-EFC students is significantly associated with GPA and math experience; the latter correlated with a 12% decline in persistence if the student was in need of math remediation and failed to pass a first-year math course, or a 22% decline if the student failed to complete any math course (matched average effect in Table 7). In contrast, math experience does not weigh in on retention of high-EFC students. Both high and low-EFC students are affected similarly by the GPA. Only middle-income students (EFC of $4,000 to $10,000) appear to be influenced by the amount of aid they received. Namely, state-funded grants designed to facilitate university access for needy students are estimated to raise persistence by almost 8% (per $1,000) net of self-selection bias associated with receiving aid that includes loan money (Table 7). Such grants are typically awarded to cover unmet need after receipt of federal grants. Like high and low-EFC students, those in the middle tier exhibit a significant correlation between GPA and retention.
Results show that regression based on unmatched students fails to identify the positive impact of gift aid on retention when academic experience is not considered (Table 3). This estimation bias persists even after controlling for first-year grades and math experience of high-EFC students. Selection bias is present also when gauging the influence of aid that includes loans. That type of aid package appears as a negative correlate in retention (change in p = -7.66) without knowing a student’s EFC, propensity for receiving aid, or amount of aid received (Table 6).
A propensity score approach also offers testing of the counterfactual hypothesis on the potential effect of aid, which cannot be ascertained with standard regression. Results reveal that had aided freshmen with a high EFC received no gift aid in the form of grants and scholarships, their probability to return for a second year of study would be lower than students on average (Tables 3 and 4). This potential loss in enrollment persistence is estimated after taking into account first-year grades and math experience. Yet, no potential loss exists for students with gift aid that did not complete a FAFSA (and, hence, for whom the EFC remains unknown), had they not received such aid. This insight contrasts with the finding based on unmatched students without an EFC that shows a significant negative correlation between aid and retention (Table 3). The negative relationship may indicate statistical bias associated with aid selection.
Separate estimates by EFC and remaining need, controlling for first-year grades and math experience, together with propensity score-matched students provides a framework to gauge more accurately the financial aid-retention nexus. Among aided freshmen with no need for loans, those from low-income background (i.e., low EFC) accrue greater retention benefits from academic success than higher-income students (i.e., higher EFC), while the latter are more likely to benefit from financial aid than the former. The data do not suggest allocating more aid to low-income students would result in improved retention; for example, the amount of federal grants to low-EFC freshmen (i.e., those eligible for federal Pell grants) is negatively correlated with retention prior to factoring in their first-year academic experience, and neither the amount nor the type of aid shows any correlation with retention after accounting for academic experience (Table 4). Adding loans into the aid package likewise does not significantly affect the persistence of low-income students. If freshmen retention is an important institutional goal, a dual strategy that emphasizes academic success for low-income students coupled with greater financial assistance to higher income students is estimated to maximize overall retention.
This finding stands in contrast with the cumulative research that indicates aid benefits needy students the most, and where the absence of a positive impact is interpreted as the result of insufficient support (Pascarella and Terenzini, 2005; Carter, 2006; Somers, 1995). Results here are consistent, however, with the moral hazard theory that links asymmetry of information on academic progress between student and donor and the limited cost associated with failure to depressed motivation to excel academically (Bodvarsson and Walker, 2004). Accordingly, grant aid, which does not entail a payback obligation and which is usually not tied to academic merit (e.g., Pell Grant or Supplemental Education Opportunity Grant [SEOG]), reduces a student’s cost of investment in the event of failure and hence increases the incentive to engage in academically risky behavior; conversely, scholarships are expected to reduce such risks, as eligibility is a function of academic performance. The absence of a positive retention correlation associated with grant aid for low-EFC students (i.e., less than $4,000) may not be surprising; similarly, the positive association of state grants with retention of students with an EFC of $4,000 to $10,0000 (i.e. those facing higher cost of failure) is consistent with the risk-taking rationale. In turn, students with no loans facing the highest cost of failure (i.e., EFC greater than $10,000) rely principally on scholarship aid that requires satisfactory academic progress, since they are ineligible for low-income grants. It is precisely that group of students, which accrues a significant retention benefit from merit-based aid, lending further credence to the moral hazard explanation. The latter is steeped in contract theory (Mirrlees, 1999; Arrow, 1968; Pauly, 1968), where the assumption of expected utility maximization in people is conditioned by the presence of unobservable behavior, which gives rise to the influence of asymmetric information in an exchange relationship, in this case between donor and student aid recipient. 
The finding that money fails to have a significant positive impact on the academic outcome of needy students is echoed in other studies. Examining the effect of short-run credit constraints on college dropout via detailed panel data on students’ financial condition, Stinebrickner and Stinebrickner (2004) estimate that 85% of enrollment attrition is caused by factors unrelated to availability of financial resources. Using a nationally representative sample of college-age youth, Keane (2002) finds that easing of financial constraints would have a marginal effect on educational attainment of low-income students. Earlier, Cameron and Heckman (1998) arrived at the same conclusion, stressing that family environmental factors largely account for differences in college outcome. Similar findings emerge in more recent studies that gauge the effect of scholarship support on college participation and completion (Bergin, Cooks, and Bergin, 2007; DesJardins and McCall, 2006; Dynarski, 2005).
In addition to Bodvarsson and Walker (2004), who show that self-financed students are less likely to experience academic failure than those with little or no personal investment, the moral hazard rationale underpins results from Van Long and Shimomura (1999). They argue that anticipation of financial support by the needy, coupled with the aleatory nature of education outcomes—being a product of both innate ability and effort that is largely unobservable—reduces the incentive to excel academically and thus lowers the college completion rate of low-income students. Moral hazard due to low personal investment risk is also cited in Dávila and Mora (2004), who discovered that the level of academic effort among students with self-employed entrepreneurial parents correlated inversely with the level of anticipation to secure employment in a family-owned business.
Though the findings in this study may be limited due to lack of data on students’ affective well-being during the freshmen year—information that is typically collected with survey questionnaires, but unavailable here—empirical research shows that actuarial data from institutional databases, as used here, gauge a student’s persistence odds more accurately (Caison, 2007). Failure to produce a positive link between financial aid and retention of low-income students does not suggest a lessening of support. Instead, the latter may be procured more effectively in form of academic interventions, given the evidence presented. Second, the results are based on the experience of students at a moderately selective public research university with a sizable segment of first-year commuter students who live off campus. These, and other aspects of the academic and social climate, may vary significantly across institutions. Caution should be exercise, therefore, in drawing inferences about the effect of aid at institutions enrolling distinctly different types of students.
ACSFA [Advisory Committee on Student Financial Aid]. (2002, June). Empty Promises: The Myth of College Access in America. Washington, DC: Author.
Adelman, C. (2007). Do we really have a college access problem? Change (July-August): 48-51.
Adelman, C. (2004). Principal Indicators of Student Academic Histories in Postsecondary Education, 1972-2000. Washington, DC: U.S. Department of Education.
Adelman, C. (1999). Answers in the Tool Box: Academic Intensity, Attendance Patterns, and Bachelor’s Degree Attainment. Washington, DC: U.S. Department of Education.
Agodini, R., and Dynarski, M. (2004). Are experiments the only option? A look at dropout prevention programs. The Review of Economics and Statistics 86(1): 180-194.
Alon, S. (2007). The influence of financial aid in leveling group differences in graduating from elite institutions. Economics of Education Review 26: 296-311.
Arrow, K. J. (1968). The economics of moral hazard: further comment. The American Economic Review 58(3): 537-539.
Baser, O. (2006). Too much ado about propensity score models? Comparing methods of propensity score matching. Value in Health 9(6): 377-385.
Bergin, D. A., Cooks, H. C., and Bergin, C. C. (2007). Effects of a college access program for youth underrepresented in higher education: a randomized experiment. Research in Higher Education 48(6): 727-750.
Berkner, L., and Chavez. L. (1997, October). Access to Postsecondary Education for the 1992 High School Graduates (NCES 98-105). Washington, DC: National Center for Education Statistics.
Bodvarsson, O. B., and Walker, R. L. (2004). Do parental cash transfers weaken performance in college? Economics of Education Review 23: 483-495.
Burdman, P. (2005, October). The Student Debt Dilemma: Debt Aversion as a Barrier to College Access. (CSHE.13.05). Berkeley, CA: Center for the Studies in Higher Education.
Caison, A. L. (2006). Analysis of institutionally specific retention research: a comparison between survey and institutional database methods. Research in Higher Education 48(4): 435-451.
Cameron, S. V., Heckman, J. J. (1998). Life cycle schooling and dynamic selection bias: models and evidence for five cohorts of American males. Journal of Political Economy 106(2): 262-333.
Carter, D. F. (2006). Key issues in the persistence of underrepresented minority students. New Directions for Institutional Research 130(Summer): 33-46.
Cochran, W. G., (1968). The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics 24: 205-213.
College Board. (2004, January 10-12). Breaking Down the Barriers: Renewing Our Commitment (A Report on the College Board Colloquium, Laguna Beach, CA). New York: Author.
D’Agostino, R. B. (1998). Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Statistics in Medicine 17: 2265-2281.
Dávila, A., and Mora, M. T. (2004). The scholastic progress of students with entrepreneurial parents. Economics of Education Review 23: 287-299.
Dehejia, R. H., and Wahba, S. (2002). Propensity score-matching methods for non-experimental causal studies. The Review of Economics and Statistics 84(1): 151-161.
DesJardins, S. L., and McCall, B. P. (2006, October). The Impact of the Gates Millennium Scholars Program on Selected Outcomes of Low-Income Minority Students: A Regression Discontinuity Analysis. Paper presented at Fisher Hall, Princeton University, November 8, 2006.
Dow, J. K., and Endersby, J. W. (2004). Multinomial probit and multinomial logit: a comparison of choice models for voting research. Electoral Studies 23: 107-122.
Dowd, A. (2004). Income and financial aid effects on persistence and degree attainment in public colleges. Education Policy Analysis and Archives, 12(21). Retrieved March 26, 2007 from http://epaa.asu.edu/epaa/v12n21/.
Dynarski, S. (2005, August). Building the stock of college-educated labor. Kennedy School of Government, Harvard University, and National Bureau of Economic Research.
Fitzgerald, B. K. (2004). Missed opportunities: has college opportunity fallen victim to policy drift? Change 36(4): 10-19.
Gerald, D., and Haycock, K. (2006). Engines of Inequality: Diminishing Equity in the Nation’s Premier Public Universities. Washington, DC: The Education Trust.
Greene, J. P., and Winters. M. A. (2005, February). Public High School Graduation and College-Readiness Rates: 1991-2002 (Education Working Paper No. 8). New York, NY: The Manhattan Institute).
Hahs-Vaughn, D. L., and Onwuegbuzie, A. J. (2006). Estimating and using propensity score analysis with complex samples. The Journal of Experimental Education 75(1): 31-65.
Heller, D. E. (Ed.). (2002). Condition of Access: Higher Education for Lower Income Students. Westport, CT: ACE Praeger Series on Higher Education.
Herzog, S. (2005). Measuring determinants of student return vs. dropout/stopout vs. transfer: a first-to-second year analysis of new freshmen. Research in Higher Education 46(8): 883-928.
Hill, C. B., and Winston, G. C. (2006). How scarce are high-ability, low-income students? In McPherson, M. S., and Schapiro, M. O., (Eds.), College Access: Opportunity or Privilege? New York, NY: College Board.
Imai, K., and Van Dyk, D. A. (2004). Causal inference with general treatment regimes: generalizing the propensity score. Journal of the American Statistical Association 99(467): 854-866.
Keane, M. P. (2002). Financial aid, borrowing constraints, and college attendance: evidence from structural estimates. The American Economic Review 92(2): 293-297.
Lee, W. (2006, August 11). Propensity Score Matching and Variations on the Balancing Test. Melbourne, Australia: Melbourne Institute of Applied Economic and Social Research.
Long, N. V., and Shimomura, K. (1999). Education, moral hazard, and endogenous growth. Journal of Economic Dynamics and Control 23(5-6): 675-698.
Luellen, J. K., Shadish, W. R., and Clark, M. H. (2005). Propensity scores: an introduction and experimental test. Evaluation Review 29(6): 530-558.
Lunceford, J. K., and Davidian, M. (2004). Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Statistics in Medicine 23: 2937-2960.
Mirrlees, J. A. (1999). The theory of moral hazard and unobservable behavior: Part I. Review of Economic Studies 66(1): 3-21.
Mirrlees, J. A. (1997). Information and incentives: the economics of carrots and sticks. The Economic Journal 107(444): 1311-1329.
Mocan, N. H., and Tekin, E. (2006). Catholic schools and bad behavior: a propensity score matching anlaysis. Contributions to Economic Analysis & Policy 5(1): 1-34.
NCES [National Center for Education Statistics] (2005, January). Enrollment in Postsecondary Institutions, Fall 2003; Graduation Rates 1997 & 2000 Cohorts; and Financial Statistics, Fiscal Year 2003 (NCES-2005-177). Washington, DC: US Department of Education.
Pampel, F. C. (2000). Logistic regression: A primer. Sage Quantitative Applications in the Social Sciences Series #132. Thousand Oaks, CA: Sage.
Pascarella, E. T., and Terenzini, P. T. (2005). How College Affects Students: A Third Decade of Research. San Francisco, CA: Jossey-Bass.
Pauly, M. V. (1968). The economics of moral hazard: comment. The American Economic Review 58(3): 531-537.
Reindl, T. (2007, March). Hitting Home: Quality, Cost, and Access Challenges Confronting Higher Education Today. Lumina Foundation for Education. Retrieved April 10, 2007, at http://www.collegecosts.info/publications/
Rosenbaum, P. R., and Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika 70(1): 51-55.
Rosenbaum, P. R., and Rubin, D. B. (1984). Reducing bias in observational studies using subclassification on propensity score. Journal of the American Statistical Association 79(387): 516-524.
Rubin, D. B. (2004). On principles for modeling propensity scores in medical research. Pharmacoepidemiology and Drug Safety 13: 855-857.
Singell, L. D. Jr. (2004). Come and stay a while: does financial aid effect retention conditioned on enrollment at a large public university? Economics of Education Review 23: 459-471.
Somers, P. (1995). A comprehensive model for examining the impact of financial aid on enrollment and persistence. Journal of Student Financial Aid 25(1): 13-27.
Stampen, J. O., and Cabrera, A. F. (1988). The targeting and packaging of student aid and its effect on attrition. Economics of Education Review 7(1): 29-46.
Stinebrickner, T. R., and Stinebrickner, R. (2004). Credit Constraints and College Attrition. Paper presented at the Canadian Employment Research Forum, Ryerson University, Toronto, Ontario, June 3-4.
St. John, E. P. (2006). Lessons learned: institutional research as support for academic improvement. New Directions for Institutional Research 130(Summer): 95-107.
Tebbs, J., and Turner, S. (2005). College education for low-income students. Change 37(4): 34-43.
The Pell Institute. (2004, Fall). Indicators of Opportunity in Higher Education. Washington, DC: Author.
Titus, M. A. (2007). Detecting selection bias, using propensity score matching, and estimating treatment effects: an application to the private returns to a master’s degree. Research in Higher Education 48(4): 487-521.
Van Long, N., and Shimomura, K. (1999). Education, moral hazard, and endogenous growth. Journal of Economic Dynamics and Control 23: 675-698.
Voorhees, R. A. (1985). Student finances and campus-based financial aid: a structural model analysis of the persistence of high-need freshmen. Research in Higher Education 22(1): 65-92.
Zeckhauser, R. (1970). Medical insurance: a case study of the tradeoff between risk spreading and appropriate incentives. Journal of Economic Theory 2: 10-26.
Zhou, Z., and Lam. P. (2007). Discussion of statistical and regulatory issues with the application of propensity score analysis to nonrandomized medical device clinical studies. Journal of Biopharmaceutical Statistics 17: 25-27.
Footnotes (click on a footnote number to return to the paper)
1. Though a multinomial logit model assumes the independence of irrelevant alternatives, unlike a multinomial probit model, the latter is not well suited for estimating the probability of an underlying nominal, qualitative outcome variable that is unevenly distributed, as is the case in this study (due to the assumption of a cumulative normal distribution). Also, the probit model failed to converge at a global optimum. The logit vs. probit issue is examined in Dow and Endersby (2004) and Pampel (2000).
2. Of twenty-four tested regression models, two models exhibited intolerable collinearity in the variance decomposition matrix (> 0.80; condition index > 15) associated with first-year GPA; removal of that variable did not affect the statistical significance threshold or direction of the other predictors.
3. EFC-based groupings correspond to the institution’s need to better understand how a change in aid may influence retention of freshmen at the selected EFC levels.
4. Second-order interaction regression results are not listed in tables, but available from the author.
5. The Millennium scholarship is available to all in-state high school graduates with a 3.0 GPA (on a 4-point scale). The scholarship covers tuition only and is the largest source of merit-based aid to in-state students.
6. Zeckhauser’s (1970) formal model of contingent contracts in medical insurance, where payments are a function of expenditures, provides the intellectual foundation of moral hazard theory (Mirrlees, 1999).
|Figures and Tables|
Table 3 (click to enlarge)
Department of Education Reform
University of Arkansas
201 Graduate Education Building
Fayetteville, AR 72701