Metrics and misconceptions: Why measuring outputs in higher education isn’t enough

Employers want workers who can navigate change, understand how to prioritize tasks and utilize innovative thinking—and they are looking for institutions to show that their graduates possess those skills.
Mandalyn Gilles
Mandalyn Gilleshttps://www.insidetrack.org/
Dr. Mandalyn Gilles is the director of research at InsideTrack.

As we reflect on a waning academic year, a surprising number of colleges and universities—from Florida State University to the University of Utah—are celebrating their highest-ever graduating classes. As impressive as these kinds of numbers are, however, they never tell the whole story. We are, after all, joining graduates at the very end of their journeys, and each of those journeys was filled with unique challenges and experiences.

That they graduated tells us very little about how they got there—or where they are going next. This is especially true of the class of 2023, whose time at college was immensely impacted by COVID-19. The pandemic turned a spotlight on the hardships that many students face, such as food and housing insecurity, financial challenges and mental health. These all profoundly impact learning outcomes and student success, but they are rarely reflected in simple outputs like the number of graduates.

In fact, summative measures such as graduation rates fail to capture much of the student experience at all. A graduation rate does not tell you how many students were worried about where their next meal would come from or how they would pay for their next class semester. It does not measure students’ information literacy or civic engagement. And it does not provide any insights into whether students have attained the skills and knowledge they need to turn their degree into a career. Addressing the challenges and needs of today’s learners requires a more nuanced approach to measuring student success, one that considers the whole student experience at a more granular level.

It’s time for institutions and researchers to prioritize a more holistic system of assessing and improving student outcomes rather than relying solely on surface-level metrics like outputs. This shift can lead to more effective interventions and policies that support student success and provide a wary public with the information they need to confidently invest in higher education.


More from UB: 4 reasons university and private training partnerships are accelerating


Outputs versus outcomes—what’s the difference?

It’s not just an argument over semantics. Outputs convey quantity, or how many learners institutions have reached. Outcomes, on the other hand, tell a story of quality, or what learners can now know, think or do due to institutional efforts. While outputs are valuable measures of institutional health, they come with a significant shortcoming: They provide no information about the quality of the education students receive. Outcomes tell that story, proving how institutions effectively support learner development and produce career-ready individuals.

It’s up to institutions to dig deeper into the wide range of academic and nonacademic factors that determine student success. Outcomes point the way.

According to the National Association of Colleges and Employers (NACE), employers are no longer just searching for candidates with a degree. They are searching for learners with well-developed competencies such as self-development, decision-making, and critical thinking skills. They want workers who can navigate change, understand how to prioritize tasks and utilize innovative thinking—and they are looking for institutions to show that their graduates possess those skills. Many institutions cannot honestly demonstrate to employers that this is the case because they have yet to capture those outcomes.

Likewise, a growing number of prospective learners are questioning the value of a degree. Individuals have understandable concerns about college costs and whether the money, time and effort they must invest will lead to rewarding, high-paying careers. Students need to know whether they can count on their institution to provide them with the academic, financial and personal support necessary to not only graduate but thrive. Restoring their confidence requires far more than pointing toward an institution’s graduation rate. Imagine, however, if an institution could clearly illustrate how it supports students at every step of their academic career, from ensuring they have basic needs resources to helping students secure the specific skills employers seek.

To accomplish this, institutions can conduct detailed assessments focused on learning outcomes, augment programming and services based on the findings of those assessments, and then reassess. For example, James Madison University used this approach to improve students’ information literacy, a competency included in its general education curriculum and mandated by Virginia’s State Council of Higher Education. The university redesigned both its online program for teaching information literacy and its assessment tools for measuring skills development.

Even as students’ information literacy improved, JMU continued to assess, modify and reassess the program. The process of improving student outcomes requires a continuous and sustained effort.

Simply relying on outputs is not enough to demonstrate the depth and breadth of what learners gained from the college experience—or the challenges they overcame in the process. To truly account for the value of higher education and enhance the student journey, we need to understand not just the final destination, but the path learners have traversed to get there.

Categories:

Most Popular