The National Conversation

Meaningful Measurement

By Heather DiLeo

Business leaders in the U.S. challenge the usefulness of a college degree, the media reports that high school graduates are opting to move to the city and begin work rather than attend college, entrepreneurial capitalists make significant investments in education, and reformers debate competing visions of higher education鈥檚 purpose. The year is 1889.

As tempting as it is to view our era as ahistorical, the educational climate of the late 19th century was, in important ways, similar to the one in which we find ourselves. Andrew Carnegie鈥檚 recommendation that 鈥渇uture captain[s] of industry鈥 choose the 鈥渟chool of experience,鈥 over a college education is echoed today by the likes of billionaire Facebook investor Peter Thiel, who pays college age 鈥渇ellows鈥 to drop out of school.

An hour of classroom or 鈥榮eat鈥 time per week plus two hours of homework, times 15 weeks, equals one credit

Following the Civil War, American colleges and universities responded to changing attitudes regarding society, religion, and learning to pursue aims historian Laurence Veysey describes in his history of the American university as utilitarian (vocational, civic), research focused, or liberal cultural. These competing claims on the core purpose of education remain rivals in various stages of reconciliation.

By the government鈥檚 reckoning, according to its new College Scorecard, there are today roughly 7,000 institutions of higher learning in the U.S.鈥攆rom community colleges to liberal arts colleges, from research institutions to for-profits. Their differences in mission and approach are vast, yet the Scorecard invites students and families to use the same, necessarily blunt metrics: graduation rate, student debt load, alumni earnings, and the like.

What nearly all of these institutions have in common is an unarguably superficial system of measuring a student鈥檚 progress鈥攊tself a vestige of Andrew Carnegie鈥檚 day. While a few have left it behind, nearly all schools use the credit hour as a proxy for learning. An hour of classroom or 鈥渟eat鈥 time per week plus two hours of homework, times 15 weeks, equals one credit.

Moving beyond the credit hour is an invitation to recognize learning that happens outside of the classroom

Originally conceived at the turn of the 20th century not as a measure of college-level learning but as a means of calculating faculty workload and qualification for pensions, the credit hour proved so convenient a standard that it gradually achieved its present power and presumptive meaning in higher education. Credits are the standard that state and federal governments use to award student grants and school funding; faculty are often compensated per credit hour taught; and families pay tuition and students achieve degrees on this basis.

We tend to assume that a third-year student is more advanced than her second-year counterpart. And wonder about the fifth-year student who hasn鈥檛 achieved her diploma. Should we?

The credit hour is surely as poor a measure of educational value or meaning as the time card, by itself, is a pale record of work performance. Some institutions are seeking a more meaningful alternative.

鈥淐ompetency- or capacity-based education [CBE],鈥 a growing trend in education, 鈥渇lips that on its head and says it鈥檚 not about how many hours you spend in the classroom. It鈥檚 not about how much homework you do. It鈥檚 about what you learn. There are ways to demonstrate that learning but how you get there is rather secondary to actually getting there,鈥 says Zeke Bernstein, dean of research, planning, and assessment at Bennington.

MEASURING MEANING

Bernstein and educators like him are interested in the potential for colleges and universities to re-emphasize the core purpose of education by re-thinking what they measure. Moving beyond the credit hour is an invitation to recognize learning that happens outside of the classroom and to encourage areas of a student鈥檚 development that exceed course content and academics.

鈥淪tudents develop a lot of capacities outside the classroom in their independent work in ways that are very apparent to faculty. They could have an experience over a summer that transforms their motivation, their ability to work with a team, and so on. You would want that to be reflected in a way that鈥檚 much more dimensional than a credit or a grade. You could think about it as a complement to the narrative evaluation鈥攕omething that provides students with an even more nuanced view of their strengths and weaknesses,鈥 says Bennington President Mariko Silver.

MM slide 3 img

In 2013, the Obama administration began to provisionally recognize and allow funding for CBE programs that 鈥渙rganize [course] content according to competencies鈥攚hat a student knows and can do鈥攔ather than using a traditional scheme.鈥

As many as 600 U.S. institutions are currently exploring capacity-based learning for a variety of reasons.

MEASURING SKILL

鈥淭he motives [for exploring CBE] are sometimes economic or sometimes geared toward trying to accommodate an increasingly diverse student body,鈥 says Michael Rose, American education scholar and professor of Social Research Methodology in the UCLA Graduate School of Education and Information Studies.

Organizations such as the New America Foundation argue the injustice of the idea that time in school is necessarily more valuable than time spent working. As Amy Laitinen writes in a policy paper for New America, 鈥淲hile some students earn credits for little more than sitting in class, millions of professionals who have acquired college-level learning on the job have no way to get credit for their learning.鈥

鈥淚f we鈥檙e looking at a nontraditional student population, there can be real merit for them [in a CBE program]. If you get someone who has been a tool and die maker for 20 years and is coming back into a mid-level technical program, that could make some difference in how long he has to be in school,鈥 says Rose.

Rose is referring to a species of CBE sometimes called direct assessment, whereby students鈥 prior learning, perhaps through work experience, is assigned college credit, thus shortening their time in the classroom.

Many well-intentioned efforts to quantify the value of higher education run the risk of defining success by what is most easily measured, rather than by what is most meaningful

These and other credentialing initiatives, such as those spearheaded by the Gates and Lumina Foundations, generally see 鈥渃apacities鈥 through the lens of job training, which has implications far beyond how the term is defined.

鈥淚n some people鈥檚 mouths, 鈥榗apacities鈥 is another way to say skills. But the idea of skills in the higher ed conversation has become fairly narrow and flat to mean vocational skills,鈥 says President Silver. 鈥淭he hope is that by talking about capacities we are reopening, in a sense, the idea of what skills are.鈥

THE BROAD DEFINITION

Several institutions address the idea of what skills or capacities are in broader and more interesting ways. It is in these examples that some of the most exciting innovations in higher education today are being tested.

Historian of mathematics and former Bennington faculty member Glen Van Brummelen helped to found Quest University, Canada鈥檚 first private liberal arts and sciences university, in British Columbia in 2007.

Like Bennington, Quest鈥檚 inquiry-based approach to learning asks students, a number that has grown from 74 eight years ago to 700 today, to identify a question that directs their learning in their last two years.

This and other features of the education such as month-long intensive 鈥渂lock鈥 courses, which students take one at a time, and experiential learning are designed to produce creative graduates who are prepared to innovate. Van Brummelen asserts that developing soft skills like collaboration and communication, acquired both in and out of the classroom, is at least as valuable as mastering course content.

鈥淎t Quest as at Bennington, we don鈥檛 believe that classroom learning is sufficient to prepare students to be much more than professors. It only makes sense that students engage with possible careers as part of their selection of their life paths in college, and it鈥檚 baffling that so many universities have thought otherwise.鈥  

Quest students spend as much as a semester on experiential learning to understand how collaboration and compromise work in real settings. Van Brummelen believes this allows students to operate more effectively in their work but also in other areas of their lives. These experiences often determine students to change their educational focus.

鈥淚magine the career prospects lost to students who devote their energies, say, to preparing to be a teacher, and then find out too late that it鈥檚 not for them.鈥

Students don鈥檛 become leaders, Van Brummelen says, by sitting at desks or taking tests but by engaging in collaborative work both in and out of the classroom, by leading groups, presenting, posing questions, and implementing their solutions at some local level. 鈥淚n other words, [by] learning to do exactly what they鈥檒l need to do to change the world when they graduate.鈥

Mark Somerville, professor of electrical engineering and physics, is a founding faculty member at Olin College of Engineering in Needham, Massachusetts, a school that seeks to 鈥渞adically change engineering education and fuel innovation.鈥

His book, A Whole New Engineer, co-written with Professor Emeritus of the University of Illinois at Urbana-Champaign David Goldberg, chronicles how engineering, once the pioneering and visionary profession that produced the Ferris wheel and electric illumination came, over time, to be narrowly defined by technical expertise and argues for the need for engineers with imagination, intuition, and emotional intelligence as well as technological skill.

The capacities required for engineers to transform from technicians and problem solvers to true innovators include a broad array of capacities. Somerville and Goldberg refer to them as 鈥渕inds鈥濃攁nalytical mind, design mind, and people mind, among others.

Teaching the whole person, he explains, is superior to taking a solely cognitive approach to grow engineers鈥 ability to empathize, to recognize what others鈥 needs are, and to come up with creative solutions to these problems.

Meaningful Measurement Slide 2 img

Considering the student as a whole person isn鈥檛 merely an ethical obligation but a practical one. While we might be tempted to focus on technological skills as among the most valuable today, Somerville reminds us that more and more of these functions are becoming automated.

鈥淚n the 21st century, the kinds of skills that people actually need to be successful are innately human characteristics, [such as] creativity and innovation. If you want someone to develop as a creative person, one of the things that鈥檚 really important is to provide them with the space where they are encouraged to develop as whole people. That is valuable in its own right. But it鈥檚 also important because those characteristics are important for students to be leaders, and ultimately to survive in a world where rote learning skills are really no longer as valuable as they once were.鈥

A provocative theme in Somerville鈥檚 work is the idea of diminishing returns on expertise as information becomes less expensive and more open sourced. This idea has powerful implications not just for engineers鈥 educations but for higher education generally.

鈥淭his does not mean that returns on expertise are zero. There remains the need to do research and to develop knowledge. On the other hand, historically the university has been all about expertise. I would say that the liberal arts college tradition has been the first to break out of that expertise framing a little bit and get more into a developmental framing.鈥

A VISION FOR THE FUTURE

In turning their curricula to a set of broad learning outcomes, institutions like Quest and Olin have succeeded in opening the aperture of student success within their respective educational categories. Like Bennington and other colleges in the progressive tradition, such explorations provide a proving ground for schools of all kinds looking for new ways forward.

The hope is that by talking about capacities we are reopening, in a sense, the idea of what skills are

In a similarly pioneering spirit, last year Stanford University asked its Hasso Platner Institute of Design鈥攌nown simply as the 鈥攖o undertake a year-long exploration of what the undergraduate college of the future might look like. Freed of the armature that currently dictates how education is accessed and delivered, the coalition of designers, faculty, and students at the d school proposed a vision for Stanford that embraces four core principles.

  • Education is nonlinear, distributed over the course of an individual鈥檚 life so students can study, leave to work, and loop back to Stanford for more.
  • 鈥淧aced鈥 learning does away with semesters and allows students to sample areas of interest by means of day- or weeklong courses for 6 to 18 months, before choosing a program and 鈥渕ov[ing] forward with intention.鈥 Students progress at their own pace and leave when they鈥檙e ready to apply their learning in the world.
  • Disciplinary areas are combined and reconceived as information hubs offering skills such as quantitative reasoning and communication graduates capture in a 鈥渟kill print鈥 rather than a transcript.
  • Majors are replaced with missions, focused on what students are doing rather than what they have mastered.

While only some of this is current reality at Stanford, the ideas are suggestive of how vast the possibilities for curricular reform are. Implicit in all of them is a re-framing of student outcomes, oriented around capacities that serve a life鈥檚 work rather than an accumulation of credits.

鈥淲hat institutions measure says a lot about what they value,鈥 says President Silver. 鈥淢any well-intentioned efforts to quantify the value of higher education run the risk of defining success by what is most easily measured, rather than by what is most meaningful.鈥

51成人猎奇 is among the institutions currently looking to capture a broader, and arguably more meaningful, set of outcomes when measuring student learning. In one such effort, it has created an employer evaluation rubric to track the capacities students develop during their annual Field Work Terms.

鈥淲e are very interested in developing ways of acknowledging and tracking learning that鈥檚 happening in other contexts鈥攊n a club or organization or in the dorms or houses,鈥 says Bernstein. 鈥淲e know students learn in these contexts but right now we don鈥檛 measure it.鈥

And if it鈥檚 not measured, there鈥檚 no way to assess the competencies they demand, or to assign them a value.

鈥淚magine if in every one of those contexts we were asking the same questions about student learning,鈥 Bernstein continues. 鈥淪uddenly, that鈥檚 what people would be talking about as they were embarking on their learning鈥攏ot how many hours they sit in class, but whether they鈥檙e developing capacities that will serve them throughout their lifetime.鈥