Applying logic models and theories of change to undergraduate education (draft)

By John Sarvey

Many debates about higher education seem to just bring up longstanding debates --liberal arts vs. professional/vocational skills – or try to blame the high cost on one thing or another – emphasis on expensive research-oriented faculty with low-teaching loads, increases in non-faculty administrators, increase in student amenities, or students and families not making college choice decisions based on accurate information about cost vs. value.

One way to analyze the challenge in higher education is to look at what high-performing non-profit organizations have increasingly adopted as a best practice – logic models, theories of change, and measuring outcomes. While many community-based social service agencies have been forced to adopt these models, hardly any college or university has had its funding tied to such measures of performance.

If we critically examined the prevailing assumed logic model or theory of change for undergraduate education, we would encounter a number of assumptions as well as missing elements.

1.       We do not have clearly articulated and agreed upon outcomes for undergraduate education.
Different stakeholders care more about some than others. Here are the main potential outcomes to choose from:

·         The degree itself, which is taken by many (but not all) in society as a proxy for a certain level of education and skills.

·         Access to certain jobs that require a college degree, or jobs that require specific degrees (i.e. nursing, engineering).

·         Prestige, status and connections (for those that attend elite institutions).

·         A core “liberal arts” set of skills in critical thinking, reading, writing, problem-solving, etc.

·         Subject matter mastery in the major.

·         Access to further graduate or professional education.

·         Civic skills, knowledge, disposition, identity, values.

2.       Whatever we determine to be our intended outcomes (whether core, subject matter, vocational, or civic) there should be a set of activities or inputs that directly develop that set of knowledge and skills. For the most part, the primary set of activities in the predominant model of college education is for students to take a certain set of courses, with two of these, three of those, this set in preparation for the major, another set within the major, etc. The assumption is that successful completion of these set of course requirements are a legitimate proxy for having gained the intended set of skills and knowledge. However, this assumption is based on further underlying assumptions that are not necessarily reliable:

·         The primary teaching activities (lectures, small group discussion, readings, papers, exams) are effective methods for students to learn the intended material and develop the skills.

·         Graded course requirements (tests, papers, projects) are sufficient measures that the student has gained the intended skills and knowledge.

·         Successful short-term learning of course content equals long-term retention of that course content.

·         Successful completion of a course equals attainment of the course learning objectives.

·         The sum of the learning objectives across all those courses adds up to the overall set of learning outcomes for the degree or for what it means to be a graduate of that institution. (Colleges often have vision statements about what all of their students will learn and develop, but the specific skills and attributes rarely actually make it into individual course learning objectives.)

·         There is no possible way of assessing overall skills and knowledge of seniors prior to graduating (either across the institution or even within majors). Individuals and colleges cannot do it, and heaven forbid, that any outside third-party do it or that any assessment would be imposed by the federal government.

3.       Even if we could rely on, with confidence, the logic model that this set of activities leads toward the desired outcomes, we have additional assumptions which dominate higher education – assumptions which may hold true some of the time, but certainly not most or all of the time.

a.       Excellent researchers are ideal teachers of undergraduates.

b.       Even if some researchers are not ideal teachers, the proximity to their research adds educational value to nearby students. (They cross through the airspace of undergrads.)

c.       The highest status, privileges and rewards should be conferred on the best researchers, regardless of their teaching ability and performance.

d.       The path to earn a PhD and secure a tenure-track position does not need to include a single course on how to teach undergraduates or on student development.

e.       Faculty who excel at teaching but not at research should be relegated to secondary status, paid less, and offered no job security. Excellence in teaching should not constitute any guarantee of continued employment.

f.        Learning that takes place outside of formal registered courses does not really exist and therefore does not count. And certainly would not be measured. (Northeastern’s co-op program is one exception. It definitely “counts” and is institutionally supported.)

g.       Student development is more the responsibility of student affairs staff than faculty, but since it’s outside the curriculum and student affairs personnel are secondary to faculty, it doesn’t really count.

Even some of the smallest social serving nonprofits today are forced, by their funders, to demonstrate and measure outcomes and to validate that all of the assumptions and links throughout their logic model are valid and reliable. If a nonprofit cannot, demonstrate with its own data that a particular chain or assumption is valid, then it’s often required to point to other research that does.

The straightforward application of logical models and theories of change to undergraduate education would immediately help to better align institutions around a clear set of outcomes and a set of activities or inputs that clearly lead to those intended outcomes. It would also lead institutions to at least attempt to measure the overall mastery of skills and knowledge of their graduating seniors.