Assessments of Institutional Effectiveness

 

The Effectiveness and Efficiency of Postsecondary Institutions in the United States: 2010-2012 Baseline Results (2015)

This brief summarizes the results of two working papers that addressed the problem of using raw graduation rates and degrees produced per expenditure as indicators of effectiveness and efficiency. Institutional effectiveness is defined as the difference between an institution’s actual graduation rate and the rate that would be expected given the institution’s structural attributes, the types of students served, financial resources, and the geographical context. Institutional efficiency is estimated as the difference between actual educational expenditures and the expenditures that would be predicted from degree production levels, faculty attributes, and location. Highly effective colleges and universities are identified within each state.

The Occupational Relevance of a College Education: An Examination of Education-Job Match among Bachelor’s Degree Recipients

This research brief examines the nature, prevalence, and corollaries of the match between education and work among bachelor’s degree recipients. 

Principal Findings

  • 74 percent of college graduates were either employed full-time or enrolled in a post-baccalaureate program.
  • The unemployment rate was highest among alumni with a major in the humanities (10 percent) and lowest among alumni with a major in engineering (3.5 percent).
  • 78 percent of alumni who were employed full-time but not enrolled reported that their job was closely or somewhat related to their undergraduate major.
  • The prevalence of severe major-job mismatch among recent college graduates changed little between 1994 (22 percent) and 2009 (23 percent).
  • Among bachelor’s degree recipients with full-time employment, a job closely related to one’s major (i.e., high congruence) was obtained by more than 50 percent of students in computer and information sciences, engineering, healthcare fields, business, and education.
  • 37 percent of college graduates with major-incongruent jobs accepted work outside of their major primarily due to the unavailability of relevant jobs. Other primary reasons included working conditions, job location, and family (17 percent); pay and promotion (16 percent); change in career interests (9 percent); and other factors (21 percent).
  • 10 percent of all bachelor’s degree recipients who were employed in some capacity but not enrolled reported that their job was not related to their major primarily due to the unavailability of relevant jobs.
  • 88 percent of college graduates with jobs closely related to their major were satisfied with opportunities to use their education in their current job.
  • Approximately 85 percent of college graduates with jobs closely related to their major reported overall satisfaction with their jobs, compared to 50 percent of alumni with jobs unrelated to their major.

  

The Traditional Approach to Developmental Education: Background and Effectiveness

This report seeks to portray some of the difficulties that arise in conceptualizing remedial success rates and determining the effectiveness of developmental programs. Research on the effectiveness of the traditional approach is summarized, focusing on the effects of remedial assignment, enrollment, and completion. Several policy implications are offered.

Implications for Policy and Practice

  • Less than 40% of high school graduates are college-ready, and nearly half of all students take at least one remedial course at a postsecondary institution. Moreover, the disparities in remedial enrollment by ethnicity and income mirror the achievement gaps observed in the PK-12 sector. This suggests that the problem of high enrollment in remedial education during college must be partly addressed in the PK-12 sector. Two promising interventions for promoting PK-16 alignment are dual enrollment and early skills assessment (Rutschow & Schneier, 2011). 
  • Estimates of success rates in remedial education greatly depend upon how students are categorized. Student cohorts should be defined by referral and enrollment status, course subject, severity of skill deficit, and academic intentions. Cohorts should be tracked over time to account for differences in remedial course sequences and student circumstances. 
  • Indicators of student success should be consistent with the objectives of remedial education and institutional mission. Remedial coursework is frequently intended to enable students to complete college-level coursework, which would suggest measures of academic achievement and pass rates in college gateway courses. Further, colleges generally hold the aim of facilitating progress towards a credential for all students, and thus measures of student persistence, transfer, and degree completion should be used to track longer-term success rates.
  • Simple comparisons of remedial and “college-ready” student outcomes will typically fail to demonstrate whether developmental education is effective. Evaluations of student outcomes should account for differences in academic preparation and intentions, the accuracy of student placement, variation in program quality, and utilization of support services, among other factors.
  • The review of effectiveness research suggests that the completion of remedial education on average does not hinder and may improve student persistence. However, students capable of college-level work who are misplaced into remedial education appear to be most at risk of departure. This underscores the importance of establishing appropriate program requirements and ensuring that student placement processes are accurate.