A new (and maybe better) way to understand the impact of an Augustana education

As you probably know by now, the new Augustana 2020 strategic plan places our graduates’ success after college at the center of our institutional mission.  In real terms, this means that what our students learn in college matters to the degree that it contributes to their success after college.  Put another way, even if our students learn all kinds of interesting knowledge and complicated skills, if what they have learned can’t be effectively and meaningfully applied to life after college, then we haven’t really done our job.

Now whether you think that this is the last nail in the liberal arts coffin or the long-awaited defibrillator to revive liberal arts education, our own success hinges on something else that I’m pretty sure we haven’t thought much about. Exactly what are we talking about when we talk about a successful life after college? Do we have a working definition of what might make up a successful life for an Augustana graduate? In order to grapple with those vertigo-inducing questions, we have to know a lot more about what happens to our graduates after college.  But do we have anything more than vague notions about our own graduates’ lives?

I’m afraid that the answers to those questions are probably no, no, and no. In part, it’s because these are big, hard questions.  And to be fair, I don’t know of a college that has tried to get a real handle on these ideas.  So . . . . here we go . . . .

This is the kind of research project that can keep you up at night.  Because it isn’t just about getting data to figure out the relationship between one thing (an Augustana college experience) and another thing (a successful life after college).  For starters, these are two monstrously complicated constructs.  Distilling them down to some essential qualities may well be impossible.  I’m not saying that it’s NOT possible; I’m just admitting to the fact that I’m intimidated by the very idea of trying to identify a set of valid essential qualities. And as if that weren’t enough, we (higher education researchers writ large) have yet to have developed a conceptual framework that is complex enough to account for the almost infinite range of ways in which people’s lives evolve. To date, every effort to link alumni success to their college experience has presumed a straight line – even when we know that very few of us traveled a straight path to get to where we are now.

So over the past six months or so, Kimberly and I have built a multi-stage study in an attempt to get at some of these questions.  We settled on calling it “The Winding Path Study” (all credit to Kimberly for the title) and we have organized it around two initial stages, with room for additional exploration.  First, we had to find a conceptual framework that fit the way that people live their lives.  We found one that I think works that comes out of sociology and anthropology called Life Course Perspective. Essentially, this framework describes lives as amazingly complex and almost infinitely unique, yet full of three common elements – trajectories, transitions, and turning points. While Life Course scholars have extended definitions for each of these terms that I won’t try to summarize here, I think we all know what these terms mean just because we can likely point to moments in our own lives where the impact of these concepts became clear.

Next, we built a survey (but of course!) to try to get a better sense of the range of trajectories, transitions, and turning points that our graduates have experienced.  I hoped that we might get 1000 responses.  From these respondents, I hoped that we might find 100 that were willing to participate in a 30 minute interview.

Well, apparently we struck a chord.  We got 1000 responses from Augustana alumni in the first 12 hours of the survey, and finished with 2,792.  In addition, over 1200 respondents said that they would be willing to participate in a 30 minute interview.

I’ll share more about this project in the next several months as we pore through the data. One thing that jumped out at me as I began to watch the data coming in was the extent to which people were willing to tell us surprisingly personal details about their lives.  Our respondents wrote and wrote and wrote. We now have a treasure trove of data that we have to read through and organize.  At the end of this project, however, we will likely have a much greater understanding of the range of life courses that our alums have taken. Better yet, we hope to find some patterns that will help us think about the way that we guide our students during college.

The goals of the Augustana 2020 strategic plan are lofty and complicated.  I’m not sure we even realized how challenging this plan would be when the Board approved it in the winter or when we designed it last fall.  But now that we’ve started to roll up our sleeves, I think we already have information on our graduates that most colleges could only wish that they had.  Now comes the fun part!

Make it a good day,

Mark

 

Do we privilege extroverts?

Over the past couple of decades, researchers have uncovered all sorts of ways in which certain types of students experience college differently.  Racial and ethnic minority, international, LGBTQ, first-generation, lower SES, and even politically conservative students encounter marginalizing experiences that can undercut the quality of their education.  Interestingly, researchers examining systemic differences in the ways that students experience college have spent the vast majority of their energy parsing students by demographic traits.  Far fewer studies explore whether certain personality traits might disadvantage specific groups of students.  Today I want to describe some new findings from our own Augustana data regarding more introverted students.  I think these findings are worth considering seriously if we are going to ensure that all students are given an equal opportunity to feel like they belong on campus.

Two items in the mid-year freshman survey address key aspects of academic and social acclimation.  One question asks, “How many of your professors did you talk to outside of class about how to best succeed in their course?”  The other question asks students if they have “begun participating in at least one student organization that interests them.” We know acclimation is important because students who don’t acclimate to Augustana tend to feel like they don’t fit in and then are much more likely to struggle, leave, or both.  And we know about the connection between acclimation and a sense of belonging because we also ask freshmen in the mid-year survey about the degree to which they feel like they belong on campus.  In addition to a host of research findings that lay out the positive relationship between acclimation, fit, and student success, our analysis has produced similar findings in each of the last several years.

But now that we have been able to link the Student Readiness Survey (data collected in the summer prior to enrollment) with the mid-year freshman survey (data collected at the end of winter term), we can add a potentially important personal disposition to the mix. This measure is a three-item scale called Comfort with Social Interaction that asks students to indicate their level of comfort meeting new people and interacting in a large, unfamiliar social setting.  So the goal of this analysis was to see if, even after accounting for acclimating behaviors (talking to professors and joining student organizations), this personal disposition continued to impact students’ sense of belonging on campus.

First, we found that the Comfort with Social Interaction scale produced a statistically significant positive effect (in a statistical sense) on the number of professors that freshmen talked to outside of class.  In other words, students who were less comfortable with social interaction were significantly less likely to talk to their professors than those students with high Comfort with Social Interaction scores.  This finding held even after controlling for gender (because we know that female students tend to seek out professors more often than their male counterparts).

Second, we found that the Comfort with Social Interaction scale also produced a statistically significant positive effect on the likelihood that a student had begun participating in a student organization that interested them.  Put another way, students with lower Comfort with Social Interaction scores were significantly less likely to have begun participating in a student organization that interested them.

Finally, we investigated whether the Comfort with Social Interaction score might influence a freshman’s sense of belonging on campus even after taking into account whether or not he or she had joined a student organization.  Sure enough, even after accounting for joining a student organization, the Comfort with Social Interaction produced a statistically significant positive effect.  In fact, our analysis found that the degree of discomfort with social interaction (i.e. a low score on the above scale) could ultimately produce a larger negative impact on a sense of belonging on campus than simply not belonging to a student group.

Both of these findings seem to hold important implications independently.  The impact of a student’s comfort with social interaction on the number of professors he or she talked to outside of class is important because it suggests that simply inviting students to faculty office hours may not be enough, especially if the students who might benefit most from such interactions are also more introverted.  This may well mean that those students need some incentive to initiate such an interaction and “break the ice.”  Although we often infer that students who don’t come to office hours are less engaged in the course, this may well be a mistaken conclusion.  In addition, this finding might even translate to the nature of student participation in class discussion.

Indeed, there seem to be multiple instances in our interactions with freshmen where we seem to pathologize introversion.  In talking about these findings with some of the Student Affairs administrators, they reflected on how often residence hall staff or peer mentors might worry about students who they don’t see as often hanging out in common areas.  While it is possible that these students may be struggling, it is also true that they may simply prefer environments with fewer people.  Pressing these students to participate in activities that aren’t comfortable for them may well simply contribute to a sense of isolation and marginalization.  It may even be that our goals for freshman orientation don’t fully take into account the needs of our more introverted students at a time when they probably need us to show that we welcome them into our community too.

Taken together, these findings make me wonder if we have unintentionally created a culture at Augustana that privileges extroverts and makes it more difficult for introverts to find a niche.  I’ve had several conversations in the last year or two with faculty from disciplines that stereotypically enroll “less socially adept” individuals about their own senior survey data that hints at a lack of a “space” for those students on our campus.

I’m no where close to having some sort of smoking gun proof of such a cultural squeeze that pervasively excludes introverted students.  However, I think this is an issue that is worth considering more seriously.  So I would ask you, based on your own observations or experience might we privilege extroverts?  What do we do to make sure that more introverted students have the support necessary to acclimate – even if it takes them longer to do so?  How might we make our community more welcoming to all students regardless of their comfort with social interactions?

Make it a good day,

Mark

 

What Millennials Regret about College

A couple of months ago, The Pew Research Center published The Rising Cost of Not Going to College.  The primary finding of this study was pretty straightforward: Yes, a college degree predicts a substantially higher average income than those without a college degree.  Moreover, the gap in average salary between those with and without a college degree has widened, suggesting that a college degree is even more critical to individual economic success than it was for previous generations.

Although these findings add another powerful piece of evidence to rebut those who might argue that a college degree is no longer worth it, this study also asked respondents to reflect on several key within-college decisions and consider whether they should have done some things differently.  I thought I’d share on of these findings that I think might inform our work as we help our current students prepare for life after graduation.

The Pew survey asked, “Thinking back to when you were a college student, do you think that any of the following things would have better prepared you to get the kind of job you wanted, or not?”  The response options included choosing a different major, gaining more work experience, starting to look for work sooner, and studying harder.

By a substantial margin, respondents chose “gaining more work experience” more than any other option.  50% of the individuals surveyed indicated gaining more work experience, while 38% selected studying harder. Choosing a different major was selected least often (29% of respondents).  More interesting still, respondents categorized as millennials (those born after 1980 who were age 25 to 32 in 2013 when the survey was conducted) were much more likely to regret not gaining more work experience than older respondents.

I find this result interesting because millennial students are the ones who are most likely to have been met with a college engagement and involvement philosophy during their first year.  Although the theories of student engagement and involvement are several generations old, the popularity of the National Survey of Student Engagement and the NSSE juggernaut has helped turn student engagement and involvement into a pervasive theme across all types of higher education institutions.  Yet the proportion of individuals who indicated that gaining more work experience during college would have better prepared them to get the kind of job they wanted suggests to me that at least some proportion of those individuals recognize that they should have given up some of the time they spent doing other things in college in order to gain more work experience.  This work experience would have most likely been located off campus.  As such, this finding seems to throw a wrench into the engagement and involvement mantra.

With that said, it is true that increasing involvement and engagement in student organizations or other educationally purposeful activities was never included as an option on the survey.  So it’s entirely possible that these responses would have differed if that option had been included.  Interestingly, this omission does seem to indicate the degree to which at least one higher education research center thinks that student involvement and engagement might prepare students to succeed in their future employment.

Please don’t misunderstand me – I’m not suggesting that we shouldn’t care about student engagement and involvement.  On the contrary, the evidence clearly indicates that these behaviors are crucial for student learning and success.  However, I think that it would do us good to revisit the way that we design many of the activities in which student participate.  For student activities and organizations – do the experiences coordinated by these organizations encompass aspects of accountability, long and short-term planning, problem-solving with consequences if the problem is not fixed (or if the solution produces negative unintended consequences), and image management?  Likewise, for on-campus student employment – are students asked to engage in complex work that requires them to think, plan, design, implement, and adapt?

Some of the explanation for the millennials’ response to this question may well result from the harsh economic conditions into which these student graduated in the last several years. However, I am not sure that this reality explains all or even part of their response. For there is a long list of institutional examples where involvement initiatives have focused on making sure that students feel satisfied in the present or feel like they fit in right now – a construct that focuses less on effectively preparing students for their future and more an making sure that they will like things at their present institutions enough to stay.

Of course, there are a lot of big-picture things we can do to impact whether the wealth of activities in which our students participate actually prepare them to succeed in the future. But one of the best things about the work we get to do is that we get to influence our students one by one.  So I hope you’ll find a way this week to help a student stretch him or herself and choose experiences that will best prepare them to succeed in the future, no matter if they aren’t quite as much fun in the present.

Make it a good day,

Mark

When an adviser’s suggestion become guidance that a student follows

Remember the Student Readiness Survey (the SRS)?  We built this survey and compendium report two years ago to give each first year student and his or her adviser a better way to start a recurring conversation about strategizing to succeed during the first year.  We modeled the SRS after research that examines the various psychological and behavioral factors that can influence college success – things like academic habits, academic confidence, propensity to persist, stress management, etc. Faculty advisers who have used the reports as they were intended have found that their interactions with first year students have changed dramatically.

However, we’ve only had anecdotal data to suggest that this might be an effective tool until now.  So in our new mid-year survey of first year students, among several questions about the interactions between student and their adviser, we included one item that specifically focuses on the SRS.  Students responded to this statement.

  • “My first year adviser helped me understand my Student Readiness Survey (SRS) results.”

The available response options and the response distributions were:

We never talked about them (what is the SRS?) 76 20%
only briefly 74 20%
yes, but they weren’t all that useful 131 35%
yes, and they influenced how I approached the beginning of my freshman year 94 25%

As you can infer from the response options above, we would like to find that the students who selected, “Yes, and they influenced how I approached the beginning of my freshman year,” also had more positive responses to other items that we know are important for a successful first year.

Although we still have a lot to analyze, we’ve already found one statistically significant relationship that I think is worth sharing.  Another item on the same survey asked students to respond to the statement,

  • “My first year adviser made me feel like I could succeed at Augustana.”

Student responses to this item were:

strongly disagree 9 2%
disagree 16 4%
neutral 60 16%
agree 144 38%
strongly agree 146 39%

Obviously, this is an important question because students’ self-belief is often vital to their success.  Moreover, this self-belief is directly influenced by the messages students get during their interactions with faculty, staff, and administrators.

One of the more important interactions shaping this belief involves students and their advisers.  So we tested the relationship between these two items while holding constant several factors that we thought might also impact whether or not the student might indicate that (a) the student’s adviser helped him or her understand the SRS results, and (b) this  conversation influenced how the student approached the beginning of his or her freshman year.  These controls included measures that might account for the degree to which the student might already be fully prepared to succeed in their first year (and thus not really need the additional advice) or the degree to which the student might find guidance through other means like peer groups or faculty interactions (that would then “wash out” the impact of the SRS report and subsequent conversation).

Sure enough, we found a statistically significant and relatively large positive effect.  In other words, as students indicated a more positive response to the notion that their adviser made them feel like they could succeed at Augustana, students were also more likely (much more likely, to be frank) to report that their conversation with their adviser about the SRS results had influenced the way that they approached their freshman year.

This finding seems to validate the way that we designed the SRS report and the way that first year adviser training has emphasized using it as a formative conversation starter. Students often seem to respond particularly well to advice when they hear it as strategizing to increase their own likelihood of success rather than just something more that someone else has told them to do.  In addition, this finding suggests that a ready-made way for advisers to be successful in getting their students to act upon their good guidance is to use the SRS as a tool to start this conversation, revisit the strategies they discussed with the student on a later date, and continue to work with the student to build an actionable plan for success in college.

So even though it is almost the middle of the spring term and you might have long since forgotten about your students’ SRS reports, now might be just the time to get them out and revisit those results with your students.

Make it a good day,

Mark

But the story is so much more interesting than the truth!

A couple of weeks ago, the Delta Cost Project produced a report titled “Labor Intensive or Labor Expensive? Changing Staffing and Compensation Patterns in Higher Education.”  The authors examined several decades of IPEDS data to better understand the hiring and compensation trends that might have driven tuition increases across each sector of higher education.  Overall, the report concluded that higher education institutions’ workforces had increased on average by 28% in the last decade as college enrollments increased at a similar pace.

However, the sound bite that won the news cycle asserted that this report supported the “administrative bloat” meme – the claim that an explosion of non-faculty hires has driven increasing tuition costs and has eroded institutional support for (or as some folks would spin it – the supremacy of the) faculty.  The report did highlight several national trends over the last decade including increases in part-time faculty, increases in mid-level administrators, increases in the cost of benefits for all types of employees, and a drop in the ratio of faculty to administrators (i.e., there are more administrators per faculty member now than there were 20 years ago).

But all of these numbers in the Delta Cost Project report portrayed national trends.  A number of faculty and administrators asked me to examine our own Augustana data to compare whether our trends replicate these national data.  So I presented our local data to the Faculty Senate last week and have linked the power point for you to see here.  In order to make any sense of the rest of this post, you’ll have to click on the power point and have a look at the graphs in it.

I’d like to quickly point to a couple of take-aways and then ask the same question that I asked at the end of my presentation.

First, as you can see from the graphs in the power point, Augustana has not mirrored the national trends in the relationship between faculty and administrator positions.  In fact, we’ve gone the other direction.  Faculty positions have increased while administrator positions have declined.

Second, our own increasing use of part-time faculty parallels the national trends, although to a far smaller degree.  Similarly, albeit to an even smaller degree, we’ve increased the number of non-tenure track full time faculty in recent years.

Now I don’t expect for a second that presenting our local data will forever quiet the claim that administrative growth at Augustana is out of control.  But I would like to ask one question: What do any of these numbers have to do with student learning?  Do we know that more faculty, a lower student-faculty ratio, or a lower faculty-administrator ratio somehow improves our retention or graduation rates?  The little evidence we have would suggest that none of these changes produce any effect.  Likewise, there is little evidence to suggest that more administrators,  a lower student-administrator ratio, or a lower administrator-faculty ratio is a quick fix either.  The fact is that we have no idea what the ideal mix of faculty and administrators might be.  In fact, the answer might not be in the numbers themselves, but rather in how all of our faculty, administrators, and staff collaborate to create the best possible conditions for student acclimation, learning, and growth.

Make it a good day,

Mark

Supporting Students IN ORDER TO Challenge Them

The most fundamental of frameworks for successful student development, learning, and growth is the synergistic concept of challenge and support.  Essentially, this concept articulates the critical balancing of two approaches to facilitate learning.  First, If we want to help students grow in substantial ways, we have to challenge them to push themselves beyond where they are comfortable.  Then, in order to minimize the likelihood that they will quit in the midst of this discomfort, we must provide encouragement (support) to help them persist toward their goal. It is equally important to recognize that students need both types of interaction, no matter the ordering of them.  So if we want students to respond positively when we challenge them, we have to have already built a foundation of trust (by expressing a belief that they are capable of success) so that they will be willing to take the risk in responding to our challenge.  In this way, challenge and support function almost like Yin and Yang.  If we want our students to grow, and more importantly take responsibility for their own growth, neither one of these two concepts works without the continuous healthy presence of the other.

In the mid-year first year survey, we asked freshmen how often their instructors had pointed out something that they had done well.  We asked this question because we wanted to find out more about the degree to which students experienced support.  (Last week I discussed one of the questions that addressed the degree to which students’ experience challenge.) The responses were distributed like this:

  • Never – 3%
  • Rarely – 15%
  • Sometimes – 44%
  • Often – 29%
  • Very Often – 9%

Frankly, if you were to force me to pick an “ideal” response distribution, I’d say that I would like to see every student choose “sometimes” or “often.”  At the same time, I’d hope that this response was balanced by students’ indicating that they also experienced consistent levels of challenge. Furthermore, I’d hope that this response was a reflection of our students’ experiences in each course rather than the possibility that our students all had some professors that were uniformly critical and others who were uniformly encouraging.

It troubles me that 69 of the 375 respondents (about 60% of our freshman class completed this survey) answered “rarely” or “never.”  Of course these students may have also been classic screw-ups who rarely or never turned in work that merited a compliment.  But even if that were so, given that human beings need a combination of challenge AND support to successfully take on a challenge and persist through to overcome it, throwing our hands in the air and saying that these students’ work didn’t merit a positive word simply increases the chances that they won’t succeed.

Humbly, I would suggest that our job as educators isn’t to ensure failure.  Instead, I’d suggest that our job is to increase the likelihood of success, especially among those who don’t rise to the occasion on their own or who already had the tools before they got here.

One important detail to remember is that these questions asked students to indicate the degree to which they think they received compliments for something that they did well. That isn’t the same as trying to find out if their instructors actually gave them compliments. Sometimes students don’t recognize the words we say or write as compliments just because of where they are in their own development.  For example, students may well not understand the academic language we often use to describe their effort in a paper as a compliment.

So as you begin to provide feedback to students in discussion, on written work, in online fora, or on other assignments, find ways to provide enough support to gain their trust. For then, and only then, will you be in a position to really challenge them when it matters and push them to excel beyond what they originally thought was possible.

Make it a good day,

Mark

 

What if your sense of how hard your students work doesn’t match how hard they think they work?

There are so many times when I read or hear of a great idea that I know would help me in my work. But at the key moment when I could really put that piece of information to use, the little nugget might as well be circling a distant galaxy. So one of the things I am going to try to do better is write posts that are more relevant to the issues faculty and staff face when they face them. You’ve probably heard of Just-in-Time Teaching (it’s a great book, by the way). Think of this as just-in-time data.

Even though the spring term starts this week, many of you are probably still toying with the the details of your syllabus(es), thinking about what you might do to make your class just a bit better without blowing it up and creating an avalanche of work for yourself precisely when you are already swamped. A couple of weeks ago I wrote about evidence from our own data suggesting the potential benefits of adding an early assignment to your course. Two other items from our just-completed mid-year survey of freshmen put in mind one other important, and sometimes easily forgotten, issue that can also make a big difference in your student’s learning and your course’s success.

Freshmen were asked at the end of last term to respond to these two items.

My instructors set high expectations for my learning and growth.

never 3 1%
rarely 6 2%
sometimes 38 10%
often 163 43%
most or all of the time 165 44%

I really worked hard to meet my instructor’s expectations.

never 2 1%
rarely 2 1%
sometimes 38 10%
often 146 39%
most or all of the time 187 50%

On one level, the fact that these two items correlate so closely is a good thing (it would indeed be frightening if they didn’t!). However, as I thought more about these two response sets, I began to wonder: would 89% of our faculty (the proportion that matches our students’ response distribution) also say that first-year students “really worked hard” to meet expectations “often” or “most or all of the time”?

My guess is that there is something worth unpacking here. To be fair, it’s possible that the questions themselves are problematic.  Maybe students aren’t comfortable suggesting that their instructors didn’t push them all that much or that they “mailed it in” more often than not. Yet we have prior NSSE data to suggest that our first year students complete homework assignments and write multiple drafts of papers more often than students at comparable institutions. Because multiple data findings pointing in the same direction make it harder to challenge the validity of a general claim, I’m inclined to suspect that the data points outlined above might indeed suggest something worth considering.

That leaves us with the possibility (especially if you are one of those faculty who don’t think that your first year students “really work hard” to meet your expectations about 90% of the time) that our students either don’t always have a clear sense of what is expected of them or that maybe we aren’t always holding them to our own high expectations when we provide grades and feedback.  Moreover, and I mean this genuinely, more than a few of our students may not yet have had the kind of life experiences that teach one what it really means to work hard to accomplish something.

Of course we know from our daily work that learning is messy business. Human beings aren’t always so thrilled to be stretched outside of their comfort zone, nor are we always excited by the prospect of failure as a necessary precursor of real learning. This is why masterful teaching is a constant balancing act of pushing students beyond where they might want to go while at the same time supporting them by expressing a belief (even if it’s more theoretical than actual) that they can accomplish what you’ve ask them to do.

Most of us have probably been challenged at least once by a student who doesn’t think that they deserve the grade you gave them. That conversation is always more difficult if the student doesn’t grasp the nature of the standards you applied to their work.

I mention all of this in order to suggest that student responses to these two questions may represent the degree to which our students really (sometimes desperately) need clear, precise, and pointed guidance about faculty expectations for quality work.  Although we might all think everyone knows what it means to write clearly, students – especially freshmen – often have only the vaguest notion of what that actually looks like.

There may be lots of other things going on behind these responses.  In fact, if you’ve got an observation that you’d like to share, by all means add a comment below.  But if you want one fairly simple thing to insert into your course(s) that can pay dividends later in the term, take some extra time to clarify your expectations for your students in ways that they can understand.  Then you can truly hold their feet to the fire when you push them to “really work hard” in order to meet the expectations you set.

Make it a good day,

Mark

This week gonna need some laughs!

Since it’s finals week, since it’s snowing (AGAIN!), and since you all are going to be busy grading and shoveling for the next several days, I decided this this was as good a Monday as any to share some faux news stories that will hopefully make you laugh and momentarily forget the work piling up outside and inside.

An oldie but goodie from the Onion’s vault . . .

Professor Deeply Hurt by Student’s Evaluation

An Assessment Coordinator’s Dream from the Cronk News . . .

One Learning Outcome to Rule Them All

And finally, another Onion article that cuts it a little close . . .

University Implicated in Checks-for-Degrees Scheme

Hang in there everyone!  See you in a few weeks.

Make it a good day,

Mark

What if early feedback made your students work harder? (Spoiler Alert)

One of the ways that we come up with questions for our freshman and senior surveys is by asking advisers about the concerns that students express to them. If we start to hear a particular theme, we try to find a way to capture that issue in a single question. Then, with the data from that item, we can test to see (1) if it really is a common problem or concern, and (2) whether that issue correlates with other experiences or the broader learning outcomes that we know are important for student success.

This is how we developed the freshman survey item to which students agreed or disagreed on a five point scale, “I had access to my grades and other feedback early enough in the term to adjust my study habits or seek help as necessary.” Numerous students had claimed that they didn’t know that they were struggling in one or more of their classes until very late in the term. They suggested that it wasn’t as if they had known their grades early in the term but hadn’t sought out academic support, but rather that they had either not been assigned enough (or more often, any) graded work to know their academic standing or that their homework wasn’t graded and returned or posted until well after the middle of the term. By the time they received their first substantive grade it was often well past the drop or withdrawal date and there was little time to recover. Moreover, these students often felt the late term pressure and in most cases had emotionally “given up” on the class.

So we inserted this item into last year’s freshman survey. Of the 286 responses, the answers were distributed like this:

  • Strongly Disagree – 30 (10%)
  • Disagree – 64 (22%)
  • Neutral – 77 (27%)
  • Agree – 89 (31%)
  • Strongly Agree – 22 (8%)

After seeing an even more “normal” distribution of responses (i.e., a bar graph of the responses that looks like a bell curve) from this year’s new mid-year freshman survey, I decided that it would be worth looking in more depth at this data to see if this item might be predictive of any other important important student-faculty interactions.

In fact, there were several student-faculty interaction items that were significantly influenced by early access to grades and feedback.  In each case, the more strongly students agreed with having access to grades and feedback, the more strongly they agreed with each of the following statements.

  • My one-on-one interactions with faculty have had a positive influence on my intellectual growth and interest in ideas.
  • My professors were interested in helping students grow in more than academic areas.
  • Faculty and staff at Augustana treated me like an individual.

Moreover, these statistically significant relationships held even after we controlled for incoming ACT score, parents’ education, and the Student Readiness Survey scales for academic habits, academic confidence, and persistence and grit.

By itself, these findings are important. But we found one other result that seems particularly useful. Even after controlling for the effects of the pre-college characteristics listed above, early access to grades and other feedback produced a statistically significant effect to a question that asked, “How often did you work harder than you have in the past in order to meet your instructor’s expectations?”

In other words, students who had access to grades and other feedback early enough in the term to adjust their study habits also indicated that they worked harder than they had in the past to meet their instructor’s expectations.

Why might this be? It might be because the early feedback gives students a chance to check themselves and reflect on their current level of effort. Furthermore, if the feedback comes in the form of an actual grade, it’s data that is difficult to ignore (we all have seen students demonstrate impressive levels of denial or willful blindness). This finding might also reflect the likelihood of an educational environment that emphasizes a continual exchange between the teacher and the student. This kind of environment is much more likely to result in a deeper level of student engagement in a course.

Yes, this finding isn’t absolute proof.  And yes, this is a statistically significant finding from only one cohort of freshmen.  And no, you’ll never get smoking gun evidence that one pedagogical practice will transform your most curmudgeonly student into a singing apostle for your discipline. However, this finding does comport with a compellingly large body of research on effective pedagogy for engaged student learning.

So you want your students to work harder in your class? Insert a substantive graded assignment early in the term. Turn that assignment around quickly with some extended formative feedback. Students will use that feedback to better grasp the degree to which they are working hard enough to meet your expectations. With that information they can adjust their efforts or seek out that academic resources might help them increase their likelihood of success.

Make it a good day,

Mark

Refining the way we deliver the liberal arts

First of all, thanks to everyone who reads Delicious Ambiguity for putting up with a three-part series of posts.  Somehow it seems to me like it requires an extra dose of ego to think that any idea is so big or important that it deserves three separate posts, so I feel sort of sheepish for even trying to pull it off.  More likely, this endeavor could just mean that I’m so damn long-winded that I can’t say anything of substance within my self-imposed word limit.

With that said, this is the “refining” post and is to be the last of the three-part bit I promised on reframing, refocusing, and refining the way that we operationalize the liberal arts. Partly because I’m late with this post and partly because it would be ironic to write a painfully long post about the act of refining, I’m going to make an extra effort to be concise.

I think that we should refine the way that we deliver the liberal arts because the quality of any educator’s effort (i.e., the degree to which an educator influences learning) can’t be separated from the amount of time that an educator has to dedicate to this effort. We know that helping students develop the skills and dispositions necessary to succeed in life requires substantial personal interaction. Over the last decade, it appears that we’ve become a very busy college.  We’ve added a host of legitimate educational and administrative responsibilities at every level, revamped programs and curricula, and seem to constantly on the brink of deciding to change something big. Yet we have taken very little away even as we’ve taken on all of these entirely defensible programs, pedagogies, and policies.  During the last twelve months of strategic planning discussions, the question that many raised, “But what are we going to take away?” seemed sometimes to be as much a lament as it was a serious question.

So I offer a set of findings from one part of last year’s senior survey and a few musing on some potential implications of these findings in the hope that it will at least fuel continued discussion of the things that we should take away.  I know there will always be some who are personally vested in the things that we might decide to take away, but (1) if we are ultimately about the experience of the students, and (2) we know that our ability to provide the best learning experience for students requires us to be efficient in the time we allot to each educational aspect of that experience, then we have to be brave enough as a community to make these choices.

Last spring, after the faculty had approved our college-wide learning outcomes earlier in the fall, I thought it might be useful to get some sort of baseline sense of where students think they develop these skills – not as evidence of where they actually gain these skills, but rather some reflection of the degree to which the experience we offer is as holistic as we’d like to think it is.  Thus, I inserted a set of questions in the senior survey that asked students to identify the experiences where they think they developed each of these learning outcomes.  For each learning outcome, students were given a list of college experiences and were allow to choose as many as applied.  Here are the results.

Disciplinary Knowledge
Major(s) or Minor(s) Courses 93.7%
General Education Courses (AGES) 45.9%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 45.9%
Residence Life Experience 11.3%
Participation in Student Organizations and Clubs 28.5%
Working On or Off Campus 29.5%
Informal Interactions with Faculty and Staff 34.3%
Informal Interactions with Peers 25.4%
Volunteering in the Community 19.8%
Senior Inquiry 55.6%
None of the Above 0.4%
Critical Thinking and Information Literacy
Major(s) or Minor(s) Courses 91.3%
General Education Courses (AGES) 58.4%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 36.4%
Residence Life Experience 10.7%
Participation in Student Organizations and Clubs 23.0%
Working On or Off Campus 25.9%
Informal Interactions with Faculty and Staff 29.7%
Informal Interactions with Peers 26.7%
Volunteering in the Community 11.9%
Senior Inquiry 57.8%
None of the Above 0.4%
Quantitative Literacy
Major(s) or Minor(s) Courses 77.6%
General Education Courses (AGES) 50.5%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 15.2%
Residence Life Experience 3.2%
Participation in Student Organizations and Clubs 8.7%
Working On or Off Campus 16.0%
Informal Interactions with Faculty and Staff 10.9%
Informal Interactions with Peers 8.1%
Volunteering in the Community 3.4%
Senior Inquiry 39.0%
None of the Above 3.4%
Collaborative Leadership
Major(s) or Minor(s) Courses 62.0%
General Education Courses (AGES) 33.1%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 37.4%
Residence Life Experience 27.3%
Participation in Student Organizations and Clubs 65.5%
Working On or Off Campus 46.7%
Informal Interactions with Faculty and Staff 29.9%
Informal Interactions with Peers 43.2%
Volunteering in the Community 40.0%
Senior Inquiry 24.6%
None of the Above 1.4%
Intercultural Competency
Major(s) or Minor(s) Courses 47.1%
General Education Courses (AGES) 48.5%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 46.9%
Residence Life Experience 26.5%
Participation in Student Organizations and Clubs 38.6%
Working On or Off Campus 27.9%
Informal Interactions with Faculty and Staff 19.0%
Informal Interactions with Peers 40.0%
Volunteering in the Community 37.0%
Senior Inquiry 17.2%
None of the Above 5.1%
Communication Competency
Major(s) or Minor(s) Courses 84.4%
General Education Courses (AGES) 60.6%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 41.4%
Residence Life Experience 20.2%
Participation in Student Organizations and Clubs 41.6%
Working On or Off Campus 35.2%
Informal Interactions with Faculty and Staff 40.2%
Informal Interactions with Peers 37.2%
Volunteering in the Community 22.4%
Senior Inquiry 51.3%
None of the Above 1.2%
Creative Thinking
Major(s) or Minor(s) Courses 79.6%
General Education Courses (AGES) 59.2%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 36.6%
Residence Life Experience 13.3%
Participation in Student Organizations and Clubs 36.6%
Working On or Off Campus 26.1%
Informal Interactions with Faculty and Staff 23.0%
Informal Interactions with Peers 27.1%
Volunteering in the Community 17.4%
Senior Inquiry 49.3%
None of the Above 2.6%
Ethical Citizenship
Major(s) or Minor(s) Courses 59.4%
General Education Courses (AGES) 45.7%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 40.8%
Residence Life Experience 25.5%
Participation in Student Organizations and Clubs 42.0%
Working On or Off Campus 36.6%
Informal Interactions with Faculty and Staff 31.9%
Informal Interactions with Peers 39.6%
Volunteering in the Community 42.6%
Senior Inquiry 26.1%
None of the Above 4.8%
Intellectual Curiosity
Major(s) or Minor(s) Courses 87.1%
General Education Courses (AGES) 59.2%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 49.5%
Residence Life Experience 9.3%
Participation in Student Organizations and Clubs 27.7%
Working On or Off Campus 23.6%
Informal Interactions with Faculty and Staff 42.0%
Informal Interactions with Peers 34.9%
Volunteering in the Community 19.8%
Senior Inquiry 51.1%
None of the Above 2.2%

To some degree, the implications of these findings may lie in the eye of the beholder.  But what jumps out to me is the degree to which major and minor courses seem to do so much of the heavy lifting.  If this is really so, then it is no wonder that faculty would be pressed for time to do much else besides teach students.  However, if we would like to create a more holistic educational experience and do so in a way that allows all of us to be equally involved in the development of our students, then there might be some logic in considering ways for faculty to pass on some of the responsibilities of learning in specific areas.

But another way to read this data is to consider the degree to which students don’t indicate experiences outside of the academic realm more often.  And this might be a more useful way to think about designing a comprehensive college experience that is more effective and more efficient.  In an odd way, we might find that the refinement we need to consider is not that the responsibility for students’ learning across each of the learning outcomes is distributed more exclusively among individual experiences, but rather that each of us explicitly understands our role in contributing (1) a small but crucial building block to students’ development and (2) the sinew that connects that building block to another block that can only be gained through a different experience offered elsewhere on our campus (or an off-campus experience facilitated by someone on campus).

Someone once said that planning to do something is the easy part; the hard part was actually doing it.  I suspect for us this will also be true.  The strategic planning was the easy part.  The hard part will be implementing this plan in a way that accomplishes the learning goals and student success to which we aspire.

So let’s roll up our sleeves.  We’ve got work to do.

Make it a good day,

Mark