Participation: A Prerequisite for Improvement

Usually I post to my blog on Monday mornings, but I hope you’ll indulge this early post and keep it in mind as you begin your week.

Sexual assault is a problem on virtually every college campus. Yet it is only very recently that colleges and universities, no doubt pushed by public outcry and increasing stern federal action, have begun to face the need to more fully understand and address this issue.

Within the last year, Augustana substantially revised a host of policies regarding sexual assault. But other than those cases that are reported, we don’t know nearly enough about our students’ perception of, and experiences with, sexual assault on campus.

For the last two weeks we’ve been participating in a survey of campus climate and sexual assault conducted by the Higher Education Data Sharing Consortium. This upcoming Friday, March 27th, the data collection phase of our participation in this survey will end. Although we repeatedly invited responses from all students, as of last Thursday we had only received responses from 570 individuals. While that means we’ve heard from almost 25% of our study body (good enough in statistical terms to make some inferences based on the results), we need to hear from as many students as possible. This is in large part because the most useful information is likely to come from those who are most reticent to share their experiences, making the number of total responses all that much more important.

So I am asking – no matter if you interact with students as their instructor, their mentor, their work supervisor, or even their friend – that you encourage your students to complete this survey. Please remind them that participation in this survey is a prerequisite for improvement. In other words, we can’t improve what we do as a college if we don’t know what our students experience.

I know you have plenty of things on your mind as you prepare for this week. But your comments, even if they are brief, will demonstrate the degree to which Augustana is serious about facing this issue and eradicating sexual assault from our campus. I know that eliminating sexual assault might seem like a rather high bar; I just don’t know how we could aim for anything less.

So please mention this survey to your students. They all received an email on Sunday evening inviting those students who had not participated one last time to respond. Their unique link to the survey was in that email. They can complete it any time this week, but after Friday the survey will no longer be accepting new data.

Thank you for your help.

Make it a good day,



This week gonna need some laughs!

Since it’s finals week, since it’s snowing (AGAIN!), and since you all are going to be busy grading and shoveling for the next several days, I decided this this was as good a Monday as any to share some faux news stories that will hopefully make you laugh and momentarily forget the work piling up outside and inside.

An oldie but goodie from the Onion’s vault . . .

Professor Deeply Hurt by Student’s Evaluation

An Assessment Coordinator’s Dream from the Cronk News . . .

One Learning Outcome to Rule Them All

And finally, another Onion article that cuts it a little close . . .

University Implicated in Checks-for-Degrees Scheme

Hang in there everyone!  See you in a few weeks.

Make it a good day,


The Holiday Wish List for a Measurement Geek

Sincerely apologies to anyone who tried to find a new post on my blog yesterday. Apparently our server went “walk-about” over the weekend and our IT folks have been working day and night to salvage everything that was no longer operational.  I think that we are in the clear today, so I’ll try to put this post up a day late.


This is the week where I can’t help but overhear all the talk of the holiday gifts that people are getting for their spouses, partners, kids, friends, or in-laws.  And it struck me that there aren’t nearly enough suggestions for measurement folks who need to just own their geekdom and go big with it.  So here are a few ideas, discoveries, and possibilities.

  • Statistics ties.  Any formula, pie chart, or dumb stats pun on a tie.  Because nothing bludgeons humor to death better than a stupid stats pun.
  • The children’s book Magnus Maximus, a Marvelous Measurer.  It’s a pretty fun book with wonderful illustrations.  And it’s never too early to stereotype your profession.
  • The world’s largest slide rule.  Of course, it’s located in Texas.
  • The complete DVD set of the TV show NUMB3RS. This show managed to tease my people with the hope that someday complex math skills could really save a life. And yet, to this day I’ve never been in a public venue where someone suddenly yelled frantically, “Is there a statistician in the house!?”
  • A Digicus. They were made in the late 70s and early 80s by the electronic’s company Sharp. Apparently many Japanese were suspicious of the digital calculator when it was first introduced, so the Digicus was created to allow people to check their calculator results against an abacus. And you thought higher ed types were skeptical of change???
  • And last but not least, anything by the band Big Data. Yes, there is a band called Big Data. They describe themselves as a “paranoid electronic music project from the internet.”  Okey dokey.

Make it a good holiday break,


Expanding our Academic Challenge Distinction beyond the First Year

Since 2011, two national studies of successful learning outcome improvement through educational assessment have highlighted our efforts at Augustana College.  First, the National Institute of Learning Outcomes Assessment (NILOA) published a report detailing the ways that a small group of uniquely successful institutions developed and maintain a positive culture of assessment and improvement.  Second, the National Survey of Student Engagement (NSSE) conducted an in-depth study of eight institutions, chosen from an original pool of 534 colleges and universities that had made significant gains on various NSSE benchmark scores, to identify some of the organizational values and practices that allow these institutions to make such clearly demonstrable improvements in their educational environments.

The data point that most clearly jumped out to both research teams involved the degree to which our first-year scores on the NSSE Academic Challenge benchmark increased between 2003 and 2009.  This benchmark scale asked a series of questions about the amount of time and effort students must put into their coursework to meet academic expectations and has been a staple of NSSE and the Wabash National Study.  As many of you know, we can pin our own improved Academic Challenge scores to the overhaul of our general education and LSFY programs about seven years ago, when a preponderance of earlier data simply didn’t comport with the kind of institution we wanted to be.  And even though we continue to note, discuss, and tweak perceived weaknesses that have emerged since implementing AGES, we shouldn’t let these more recently identified concerns detract from the fact that our earlier efforts were thoroughly successful in improving the educational quality of Augustana’s first year experience.

Yet the evidence of an improved educational environment (as represented by an increase in the academic challenge experienced by our students) did not seem to extend beyond the first year.  In our 2009 NSSE report, despite a significant difference in first-year academic challenge scores between Augustana and a group of 30 similar residential liberal arts colleges, our fourth-year academic challenge scores remained no different than other institutions.  Many of us were troubled by the possibility that the distinction in academic quality that we might have established in the first year could have eroded entirely by the end of the fourth year.  Although senior inquiry was intended to help us increase our level of academic challenge in the fourth year, the 2009 NSSE report did not reflect any impact of that effort (likely because SI was not fully implemented until 2010 or 2011).  So when we received our Wabash National Study four-year summary report a few weeks ago, I specifically wanted to examine our seniors’ overall score to the Academic Challenge scale to see if we’d made any progress on this rather important measure of educational quality.

(At this point, the empathetic side of my brain/soul/elven spirit/gaseous particles has guilted me into offering a pre-emptive apology.  I am going to talk about some numbers without giving you all the detailed context behind those numbers.  If you want more context, you know where to find me.  Otherwise, try to hang in there and trust that the changes these numbers represent are substantial and worth discussing.)

The Wabash National Study evidence suggests that, once again, our efforts to respond to assessment data with changes that will improve Augustana’s educational quality seem to have born fruit.  Between 2009 and 2012, our seniors’ Academic Challenge score jumped from 62.6 to 64.3 – a statistically significant increase.  Moreover, the difference between our mean score and the average Academic Challenge score of the 32 similar institutions that participated in the Wabash National Study (61.0) was statistically significant – suggesting that something we are doing during the fourth year distinguishes the academic quality that we provide from those institutions.  For my own information and confidence in this conclusion, I also looked at the 2012 NSSE annual report just to see if these Wabash Study numbers differed in any meaningful way from the much larger sample of institutions that participated in NSSE.  Again, our Academic Challenge scores placed us above the NSSE average of similar liberal arts institutions (62.5) and well above the overall NSSE average (58.4).

All of this evidence seems to point toward a familiar and heartening – if not downright exciting – conclusion.  Our efforts to improve the educational quality of an Augustana experience are working (or as the famous line goes, “I love it when a plan comes together!” . . . yes, I just quoted Hannibal Smith from the 1980s TV show “The A-Team” in a blog about institutional research.  I’m fired up – deal with it.).  The academic challenge our students’ experience in their fourth year appears to have increased.  And while we don’t have comparative data on the degree to which this effort has increased our students’ learning outcome gains (because we don’t have identical pretest-posttest outcomes data from 2009), it is clear from the Wabash National Study data that our 2012 Wabash Study participants repeatedly made larger learning outcome gains than students at the 32 similar institutions participating in same study.

Later this year we will receive the full Wabash study dataset that will allow us to examine the responses to each individual question in this scale.  I am looking forward to digging deeper into that data.  But for the time being, I think we deserve to take a moment and congratulate ourselves as a community of educators dedicated to the success of our students.  Although we continually hear critics of higher education lament that institutions refuse to collect the kind of data necessary to meaningfully assess themselves, or that faculty perpetually resist making the kind of changes that might substantively improve an institution’s educational quality, we now have multiple sources of evidence to demonstrate that, while we might not be without reproach, we have living, breathing evidence of our successful efforts to improve the Augustana education.

Are we there yet?  No.  Will we ever be there?  Of course, not.  But are we genuinely walking the walk of an institution committed to its students and its educational mission?  Absolutely.

Make it a good day,



62.6 to 64.3

The post-Thanksgiving haze

Believe it or not, I try to have a life outside of educational assessment and improvement of student learning.  That means – for example – participating in all of the normal stuff that people do over the Thanksgiving holiday.  So over the past five days I’ve packed suitcases, adapted to changes in travel plans, made conversation with all manner of family, and wished I hadn’t eaten __________.  I find it a bit troubling that although I spend an inordinate amount of time thinking about learning from past behaviors to improve future behaviors I can’t seem to learn from my previous mistakes regarding serving size, mashed potatoes, and gravy.

All this is simply to say that I didn’t write a thing last weekend.  Sorry.  And my fingers might actually now be too fat to fit onto a normal keyboard.  So you’ll have to wait til next week for another post.

Usually, I write about data findings that are ambiguous in some way.  This week, I can only write about something that was delicious.  Literally.  And most of me now regrets that second helping.  Actually, maybe the regret is really about the third helping . . .

Make it a good day,


Busy is as busy does . . .

Hey Folks,

This is the time of the term when everyone conjures up whatever remaining powers they have left to slog through finals, grade furiously, and put the term out of its misery.  Or, if you have a slightly more optimistic view of life (and I hope you do), you are overcome with a surge of pride in your students for all they have learned, all they have endured, and all they have become over ten short weeks.  See, that wasn’t so hard now, was it?

To be honest, I’m not inclined to say much this week only because I don’t think many of you have the time to read my blathering about some little data point that has me all atwitter.  And aside from that somewhat uncomfortable image, the last thing I want this blog to become is long, myopic, and just too much.

So I’ll throw this out into the cybertron and let you do what you want with it.  I’ve been privileged to be involved with a number of senior inquiry and service-learning projects this term.  I’ve been very impressed and even proud of the work that I’ve seen these students produce.  They’ve thought carefully about their research, wrestled with tough problems, dealt with mishaps and unpredictability, and throughout have remained honest, genuine, and intent on doing their best work.  Was it all perfect?  Of course not.  Was is supposed to be?  no.  But did I see growth that should make a college proud?  Damn straight.

Even though I am constantly talking about ways that we might improve, it is important to remind ourselves that we often do very good work.  And we deserve the chance to step back from time to time and soak it all in.  You put your heart into the work of making young people better.  And in many cases you help students realize a little bit more of who they aspire to become – even when they don’t fully know who that is or why it might be important.

So – grade like a banshee.  Then relax like a champion.  You deserve it.

Make it a good day,


Look mom, it’s a blog!

Hi everybody,

Yes, its true.  What was once a simple column has now turned into a blog.

What difference will it make?  None.  This column will focus on the same topics that it has explored in the past.  Sometimes I’ll talk about an interesting finding from our student data, sometimes I’ll test a claim that has been made publicly, and sometimes I’ll muse about the various tensions that arise when one seriously commits to striving for perpetual improvement.

Yes, I’ll continue to be snarky from time to time.  But now, you can call me on it in the comments section and point out my flaws, my unsubstantiated leaps, or my bad grammar for all to see.  Of course, you can also throw me a bone everyone once in a while and tell me what you liked or what made you stop and think for a second or two.

Mostly, I hope you’ll add your perspective and make this blog a conversation dedicated to thinking about our work and making change for the better.

So here it is . . . Delicious Ambiguity.  Stay hungry, my friends.

Make it a good day,


Smile! Its the end of the academic year (almost!)

At this point in the term, there isn’t a lot of time for deep, contemplative thought.  Instead, it strikes me that a good laugh is the best source of that little extra fuel to get through the last week of the academic year.  So I thought I’d supply a little higher ed humor.  Here are links to some of the best spoof news stories about higher education in the past couple of years.  If nothing else, they’ll give you one more way to procrastinate grading!


Bard College Named Nation’s No. 1 Dinner Party School


New College Graduates To Be Cryogenically Frozen Until Job Market Improves


Area Man First In His Family To Coast Through College


There are so many more, but time is of the essence.  See you next fall!


Make it a good day!



A positivity distraction

As you slog your way through the snow and the grading and the (hypothetical) curriculum reconstruction this week, I hope you will take a moment to wire your brain for positive thoughts.  I don’t have much to say today – I’m feeling a little beat down myself – but I watched this TED talk last night and it was just the tidbit I needed to get my head straight.


Make it a good day (sometimes I’m really am talking to myself),