Understanding the “new” learning outcomes of a college education

At the Augustana Board Retreat a couple of weeks ago, Allen Bertsche (Director of International Programs) and I hosted a discussion with members of the Board, administrators, and faculty about a fundamental shift that has occurred in higher education over the past several decades.  While a college education used to be primarily about acquiring content knowledge, today the most important outcomes of a college education are a broad range of complex cognitive, psychosocial, and interpersonal skills and dispositions. These outcomes transcend a student’s major choice and are applicable in every facet of life.  In short, although content is still necessary, it is no longer sufficient.  In recent years Augustana has identified outcomes like critical thinking, collaborative leadership, and information literacy as fundamental skills that every student should develop before graduation.

 

During our conversation at the Board Retreat, Kent Barnds (Vice President of Enrollment, Communications, and Planning) pointed out that, while some of us might grasp the ramifications of this shift, perspective students and their families are still firmly entrenched in the belief that content acquisition is the primary goal of a college education.  In their minds, a college’s value is directly related to the amount of content knowledge it can deliver to its students.  As many of you know, when prospective students and families visit, they often ask about opportunities to obtain multiple majors while participating in a host of experiences.  By comparison, they rarely ask about the exact process by which we develop critical thinking or cross-cultural skills in students.

 

I think it would do us some good to consider what the current calendar discussion looks like to those who believe that the cost of tuition primarily buys access to content knowledge.  The students quoted in the most recent Observer about the 4-1-4 calendar discussion exemplify this perspective.  Their rationale for keeping the trimester system is clearly about maximizing content acquisition – more total courses required for graduation equals more total content acquired, and shorter trimesters allow students to minimize the time spent acquiring content that they don’t need, don’t like, or don’t want.  With tuition and fees set well over $40,000 next year, it’s not hard to see their concerns.

 

Now please don’t misunderstand me – I am much more interested in what we do within the calendar we choose than whether we continue on trimesters or move to semesters.  Nor am I suggesting that student opinions should or should not influence this discussion.  But if we’re trying to have a conversation about student learning – with or without students – and we don’t share a common definition of the term, then we are likely doomed to talk right past each other and miss a real opportunity to meaningfully improve what we do regardless of whether or not the faculty votes to alter the calendar.  On the other hand, if we can more clearly spell out for students, parents, (and ourselves) what we mean when we talk about “student learning” and why our focus on complex skills and outcomes is better suited to prepare students for life after graduation, not only might it temper the tensions that seem to be bubbling up among our students, it might also allow us to help them more intentionally calibrate the relationship between their current activities and obligations and their post-graduate aspirations.

 

So no matter where you sit on the semester/trimester debate, and no matter what you think about the shift in emphasis from content acquisition to the development of skills and outcomes, I would respectfully suggest that we need to better understand the presumptions that undergird each assertion in the context of the calendar discussion. In my humble opinion, as Desi used to say to Lucy, we still “got some ‘splainin’ to do.”

 

Make it a good day,

 

Mark

What does Finland have to teach us about assessment?

Welcome back!  During the break I hope you were able to enjoy some time with loved ones and (or) recharge your intellectual batteries.  I will admit that I spent part of the break embracing my inner geek, reading about the amazing improvements in Finland’s student achievement scores since they instituted a new national education policy in the 1970s.  Previously, Finland had been decidedly average.  Today, their scores are consistently among the best in the world – particularly in reading and science.  As a result, the U.S. and the U.K. – countries with substantially lower scores – are very interested in finding out what might be driving this educational success story.

 

The point of my column this week isn’t to delve into the details of Finland’s success, but rather to consider one aspect of Finland’s approach that I think is particularly applicable to our current conversation about educational outcomes and improved student learning.  So here are a few links if you are interested in reading more about Finland educational success or about the exam that is used to measure student achievement.  Instead.

 

“Accountability is something that is left when responsibility has been subtracted.”

 

If you’ve already read the Atlantic Monthly article I hyperlinked above, you know that this statement is attributed to Pasi Sahlberg, an individual deeply involved in Finland’s educational transformation.  The principle to which he refers asserts that unless an educational endeavor is intentionally designed to produce a specific outcome, it is difficult to argue that gains on that outcome are entirely attributable to the educational endeavor in question.  However, as society has increasingly demanded that education prove its worth, it is deceptively easy to start by testing for an educational effect without ever asking whether the experience is really designed to best produce it.  To make matters worse, then we mandate improvement without addressing the systematic dysfunction that created the problem in the first place.

 

My sense of Augustana’s evolution regarding student learning outcomes is that we are in the midst of a process to make explicit what we have long valued implicitly.  We are trying to be clearer about what we want our students to learn, be more transparent about those efforts, and maximize the educational quality we provide.  In this context, Sahlberg’s comment on accountability and responsibility struck me in two ways . . .

 

First, the process of identifying outcomes and designing an educational program to meet those outcomes requires us to take full responsibility for the design of the program we are delivering.  When something is repeatedly greater than the sum of its parts, it isn’t just a happy accident.  Designing a successful educational program is more than just making pieces fit together – it’s constructing the pieces so that they fit together.

 

Second, just because an outcome idea sounds like it might be valid doesn’t make it so.  But in the absence of anything else, accountability measures that mean very little can all too easily become drivers of institutional policy – sometimes to the detriment of student learning.  However, the inverse can also be true.  An institution that takes full responsibility for the design of its educational programs and the system within which they exist will likely far exceed typical accountability standards because such an institution can make coherent, empirically-grounded, and compelling arguments for why it does what it does; arguments that quickly evaporate when a pre-packaged accountability measure is hurriedly slapped onto the back end of an educational process.

 

So I’d like to close by suggesting that we consider the statement quoted above in this way: If we take explicit responsibility for student learning and the design of the educational programs we provide, demonstrating our accountability – to our students or our accreditors – will be relatively easy by comparison.

 

Make it a good day,

 

Mark

The dynamics of tracking diversity

Over the past few weeks I’ve been digging into an interesting conundrum regarding the gathering and reporting of “diversity” data – the percentage of Augustana students who do not identify as white or Caucasian.  What emerges is a great example of the frustratingly tangled web we weave when older paradigms of race/ethnicity classification get tied up in the limitations of survey measurement and then run headlong into the world in which we actually live and work.  To fully play out the metaphor (Sir Walter Scott’s famous text, “Oh what a tangled web we weave, when first we practice to deceive”), if we don’t understand the complexities of this issue, I would suggest that in the end we might well be the ones who get duped.

For decades, questions about race or ethnicity on college applications reflected an “all or nothing” conception of race/ethnic identification.  The response options included the usual suspects – Black/African-American, White/Caucasian, Hispanic, Asian/Pacific-Islander, and Native American, and sometimes a final category of “other” – with respondents only allowed to select one category.  More recently, an option simply called “two or more races” was added to account for individuals who might identify with multiple race/ethnic categories, suggesting something about our level of (dis)interest in the complexities of multi-race/ethnic heritage.

In 2007, the Department of Education required colleges to adopt a two-part question in gathering race/ethnicity data.  The DOE gave colleges several years to adopt this new system, which we implemented for the incoming class of 2010.  The first question asks whether or not the respondent identifies as Hispanic/Latino.  The second question asks respondents to indicate all of the other race/ethnicity categories that might also apply.  The response choices are American Indian, Asian, Black/African-American, Native Hawaiian/Pacific-Islander, and White, with parenthetical expansions of each category to more clearly define their intended scope.

While this change added some nuance to reporting race/ethnicity, it perpetuated some of the old problems while introducing some new ones as well.  First, the new DOE regulations only addressed incoming student data; it didn’t obligate institutions to convert previous student data to the new configuration – creating a 3-4 year period where there was no clear way to determine a “diversity” profile.  Second, the terminology used in the new questions actually invited the possibility that individuals would classify themselves differently than they would have previously.  Third, since Augustana (like virtually every other college) receives prospective student data from many different sources that do not necessarily comport with the new two-part question, it increased the possibility of conflicting self-reported race/ethnicity data.  Similarly, the added complexity of the two-part question increased the likelihood that even the slightest variation in internal data gathering could exacerbate the extent of inconsistent responses.  Finally, over the past decade students have increasingly skipped race/ethnicity questions, as older paradigm of racial/ethnic identification have seemed increasingly less relevant to them.  This means that the effort to acquire more nuanced data could actually accelerate the increasing percentage of students who skip these questions altogether.

As a result of the new federal rules, we currently have race/ethnicity data for two groups of students (freshmen/sophomores who entered after the new rules were implemented and juniors/seniors who entered under the old rules) that reflect two different conceptions of race/ethnicity.  Although we developed a crosswalk in an attempt to create uniformity in the data, for each additional wrinkle that we resolve another one appears. Thus, we admittedly have more confidence in the “diversity” numbers that we reported this year (2011) than those we reported last year (2010).  Moreover, the change in questions has set up a domino effect across many colleges where, depending upon how an institution tried to deal with these changes, an individual institution could come up with vastly different “diversity” numbers, each supported by a reasonable analytic argument (See this recent article in Inside Higher Ed).

Recognizing the enormity of these problems, IPEDS only requires that the percentage of students we report as “race unknown” be less than 100% during the transition years (in effect allowing institutions to convert all prior student race/ethnicity data to the unknown category). And lets not even get into the issues of actual counting.  For example, the new rule says that someone who indicates “yes” to the Hispanic/Latino question and selects “Asian” on the race question must be counted as Hispanic, but someone who indicates “no” to the Hispanic/Latino question and selects both “Asian” and “African-American” to the race question must be counted as multi-racial.  Anyone need an aspirin yet?

But we do ourselves substantial harm if we get hung up on a quest for precision.  In reality, the problem originates not in the numbers themselves but in the relative value we place on those numbers and the decisions we make or the money we spend as a result.  Interestingly, if you ask our current students, they will tell you that they conceive of diversity in very different ways than those of us who came of age several decades ago (or more).  Increasingly, for example, socio-economic class is becoming a powerful marker of difference, and a growing body of research has made it even more apparent that the intersection of socio-economic class and race/ethnicity produces vastly different effects across diverse student types.

I am in no way suggesting that we should no longer care about race or ethnicity.  On the contrary, I am suggesting that if our conception of “diversity” is static and naively simplistic, we are less likely to recognize the emergence of powerfully influential dimensions on which difference also exists and opportunities are also shaped.  Thus, we put ourselves at substantial risk of treating our students not as they are, but as we have categorized them.  Worse still, we risk spending precious time and energy arguing over what we perceive to be the “right” number under the assumption that those numbers were objectively derived, when it is painfully clear that they are not.

Thanks for indulging me this week.  Next week will be short and sweet – I promise.

Make it a good day,

Mark

What is so delicious about ambiguity?

Welcome to a new academic year!  For those of you who have been away, it’s great to see you again.  For those of you who are new to Augustana, I look forward to getting to know you and supporting your educational efforts.

As many of you know by now, my primary focus at Augustana is to facilitate the continuous process of improving student learning. This means that, like most of us, I can be found wearing many different hats.  Sometimes you will find me designing and implementing on-campus studies to gather data that we need.  Sometimes you might find me discussing the findings from our data and the implications of those findings.  And other times you might find me collaborating with a wide range of individuals or groups to design policies, programs, or professional development to ensure that our efforts to improve student learning actually bear fruit.

So why am I also writing a weekly column in the Faculty Newsletter?

During the last year, I was struck by the degree to which many of us actually don’t know about the things we do really well.  As a result, it appeared to me that we often missed opportunities to take maximal advantage of these successes.  Sometimes these “successes” were happening only in isolated instances.   Other times these educational strengths were occurring repeatedly but without much connection to other similar and often complimentary successes.  And sometimes these successes were totally coincidental.

With this in mind, I hope this column will help us learn a lot more about ourselves, our students, the relative impact of our educational efforts, and the ways that we might improve in this collective endeavor. Sometimes I’ll share a nugget of data or information that struck me as interesting.  Sometimes I’ll pose a question that I think might help us be more intentional in what we do. Finally, from time to time I will take a specific belief or claim about some aspect of student learning at Augustana College and put it to the test.

I am not promising truth, justice, or beauty.  However, I am promising that I will try to inspire you to think more deeply about our students, our efforts, and our collective investment in this work.  I also hope that these columns will inspire conversations that lead to additional questions and, ideally, to a deeper understanding of the work that we do.

So – what would you like to know?  What ‘myth’ or claim would you like to see me put to the test (be careful what you wish for!)?  And what should I call this column?

Make it a good day,

Mark