“Lean” in and learn something new

I think it’s fair to say that most educators cringe at the idea of applying practices from the world of business to education. So many times we’ve read or heard someone talk about education as if it were a cursory transaction where students or parents simply purchase a product as an investment toward future earnings. Of course, one only needs to spend a few days trying to get students to learn something that contests their prior assumptions to know that viewing education through such a transactional lens leads to a gross misunderstanding of what we do and how education works. I’d love to see a list of all the times when a business framework was misapplied to an educational setting with disastrous results.

So, does this mean that everything developed in a business setting is guaranteed to fail in an educational setting? It’s okay if you’re inclined to say “yes” (especially if you’ve been down that road a few times). When President Bahls suggested that we could apply principles of lean management to improve a variety of processes at Augustana College, I’ll admit that I shuddered. Maybe like you, I imagined an internal apocalypse: budget cuts and position reductions with no changes in expectations. But after reading up on the concept of lean management and spending last week as a member of the first Rapid Improvement Team, I have to admit that my shudder was merely emblematic of my own ignorance. While lean management has its own set of terminology that might seem foreign to educators, the values embedded in a lean management philosophy embody the same values that we aspire to uphold in a collaborative and transparent organization dedicated to educating students. I found the framework and the process to be deeply gratifying and potentially applicable to the range of domains in which we operate.

First, “lean” doesn’t mean thinner.  It’s not about losing weight, downsizing, or cutting out the fat. It’s not an acronym. The term refers to the degree to which processes are conducted efficiently while best serving the needs of the beneficiary (i.e., anyone who benefits from that process).

Second, lean management philosophy asserts that the people best positioned to make improvement happen are those who are intimately involved in that particular job or process. Not only do those folks know the ins and outs of that work better than anyone else; they also need to believe in the efficacy of any identified changes in order to give those changes the best chance of turning into demonstrable and lasting improvements. For these reasons, any attempt to improve a process must genuinely involve the people who do that work.

Third, lean management philosophy argues that improvement of a process is exemplified in those who benefit from that process. Although the beneficiaries of our work are often students, we often conduct operations that benefit more than just students. The beneficiaries of payroll are anyone who gets paid. The beneficiaries of the salad bar are anyone who eats a salad. As a result, the way to determine if we have improved a process is to identify clear means of demonstrating an improved impact on the beneficiaries of that process.

Fourth, lean management starts with the belief that the collective ability of an organization’s people can find and put in place substantial improvements to a process.  Effective lean management begins by collaborating to develop a shared understanding of the current state of a process or problem.  Only after the problem is fully understood as something worthy of improvement would an improvement team begin to consider potential solutions.

Fifth, lean management philosophy focuses on continual improvement, not perfection. There are simply too many external and unpredictable influences to expect perfection.  Furthermore (especially in the work that we do), just when we find that a particularly education practices works well, the students change and we have to continue to adjust.

Everything the Rapid Improvement Team did last week reflected all of these values.  I was impressed with the way the process was designed to keep them at the forefront while moving us toward a set of suggestions that were extremely likely to improve the process.

If you would like to see a recording of the presentation from the Rapid Improvement Team from last Friday, you can see it here.

Ultimately, the lesson I learned from this process was that it is possible (shocking, I know) for something that has been developed in the business world over the last several decades to be applied successfully in an educational institution in a way that actually strengthens our ability to enact the values we espouse.  In addition, I (re)learned that we have some amazing people at Augustana who are willing to put their hearts into doing what we do better. I’m lucky to be a part of it.

Make it a good day,

Mark

 

A taste of my own medicine

One of the fundamental tenets of Delicious Ambiguity has been that all of us who contribute to the development of students are at our best when we approach our work through a lens of “positive restlessness.” That phrase was introduced into the lexicon of higher education writers by George Kuh and his colleagues in their book Student Success in College, describing a pervasive philosophy that his research team saw at colleges that always seemed to be seeking out ways to get better no matter how successful they already were. Anyone who knows me recognizes that I relish the chance to look for ways to improve. But I think it is an entirely fair criticism to suggest that I might have an overly rosy view of change and that I should be forced to get elbow-deep in the down-and-dirty work of actually fixing a complicated and convoluted process.

So this week, if you’ve ever thought that I needed a dose of your version of reality, you are in luck. My “comeuppance” has appeared in the form of participation in a weeklong, immersive Rapid Improvement Event (RIE). I’ll be joining a team of Augustana employees trying to wrangle a portion of the payroll process and hopefully improve it. I don’t know much about payroll – so they tell me I’m “perfect” for the job.

So here goes!

Make it a good day,

Mark

 

Hey; what’s this I hear about the Winding Path Study?

Some of you have heard me mention a study that we (AKA our massive juggernaut of an IR office better known to most of you as Kimberly and Mark) started last spring called the Winding Path Study. In short, this study was designed to gather information from all living Augustana alumni (at least those for whom we had working email addresses) about the nature of their adult lives from the time they entered Augustana up until last spring.

During the twelve months of strategic planning discussions, one of the things that stood out to me was how much we really don’t know about the long-term impact of an Augustana education. Don’t get me wrong; we have lots of wonderful stories about Augustana graduates excelling in all sorts of professional and personal pursuits. But we don’t know nearly as much as we would like about the nature of our alum’s lives after college: the ways that they have handled success and failure, the adjustments they have had to make when life throws them a curveball, or the ways that their Augustana experiences might have influenced twists and turns in their life’s path right after graduation or much later in life. This information matters because, if we are preparing students to succeed throughout their adult lives, we need to know how those lives play out across personal and professional domains and as our alums grow and change over time.

After looking through all of the different ways that colleges have tried to survey their alumni, we couldn’t find any approach that matched our conceptual frame or addressed the questions we had constructed. So we rolled up our metaphorical sleeves and built a study from scratch based on the sociological theory of Life Course Perspective, a construct that describes the life course as a series of trajectories, transitions, and turning points.

In this post I’d like to share a few summary findings just to give you a flavor for what we’ve seen from the almost 2,800 responses we received last spring in the first stage of this project.

The first two questions explored the nature of our alum’s path when entering Augustana and moving through their undergraduate years.

  • Did you have a specific career goal or major in mind when you came to Augustana?
    • 53% – Yes; I was pretty sure I knew what I wanted to do
    • 33% – Somewhat; I had some ideas but wasn’t set on anything in particular
    • 12% – No; I didn’t know what I wanted to do at all
  • Did you change majors or career goals while you were an undergraduate?
    • 38% – Yes
    • 61% – No
  •  What path did you take right after graduation from Augustana?
    • 26% – Went to grad school in the same field that I studied
    • 8% –   Went to grad school in a different field than I studied
    • 42% – Took a job or volunteered in the same field that I studied
    • 15% – Took a job or volunteered in a different field than I studied
    • 2% –   Took time off to pursue other interests
    • 7% –   Other

The next set of questions explored the varied nature of our graduates’ adult lives. Although we couldn’t have possibly captured every facet of an adult life, our goal was to gather a first glimpse that could be explored in more detail later.

  • How many times have you changed jobs since you graduate from Augustana?
    • 16% – None
    • 27% – 1-2
    • 28% – 3-4
    • 16% – 5-6
    • 11% – 7 or more
  • How many of those job changes occurred because of a professional opportunity that you chose to pursue?
    • 22% – None
    • 36% – 1-2
    • 22% – 3-4
    • 11% – 5-6
    • 5% –   7 or more
  • How many of those job changes occurred because of a professional disruption (downsizing, bankruptcy, termination, etc.)?
    • 67% – None
    • 23% – 1-2
    • 3% –   3-4
    • 1% –   5-6
    • 0% –   7 or more
  • Were any of your job changes influenced by family considerations?
    • 39% – Yes
    • 56% – No
  • Were any of your job changes influenced by personal considerations?
    • 54% – Yes
    • 41% – No

(Please note that some folks didn’t respond to every question, resulting in some proportions equaling less than 100%).

These findings deepened our understanding of the variety of pathways that students pursue after college.  Almost one quarter of our graduates, immediately after college, entered graduate school or took a job in a different field than their major.  These findings also strengthened our belief that preparing students for successful lives after college goes far beyond one’s major or minor and extends long past the first job, first graduate school degree, or whatever the first thing a student chooses to do after college might be.

As you can also see, many if not most Augustana alumni have likely led adult lives that look more like winding paths than straight lines. These findings – even if they might seem fairly obvious to anyone who has lived through the reality of an unpredictable life – have shaped our thinking as we continue to design a college experience that prepares every student to carve through life after college – no matter what comes out of the woodwork.

The first stage of this study concluded with a question at the end of the survey asking if the respondent would be willing to participate in a half-hour interview.  Based on prior research experience, we expected 100-200 positive responses.  We received over 1,400 positive responses!  So the next step for us, after spending the last six months analyzing all of the open-ended responses, is to develop a framework for the interviews and how we will select potential interviewees.  We would like to interview as many as possible, but frankly, the specter of 1,400 interviews is a bit daunting! Moreover, because we also asked other categorizing information like the year the respondent graduated, their major, and their current profession, we have all kinds of ways that we can organize and analyze this data.

Over the rest of the academic year, I hope I’ll have another update on the results of the second phase of this study. In the mean time, enjoy the last week of the fall term!

Make it a good day,

Mark

Data doesn’t have to be numbers

As some of you might know, every once in a while I get asked to talk to other colleges or universities about my study abroad research. Yesterday I was fielding some questions at one such workshop, when a faculty member who takes students on study abroad trips told a story of her experiences talking to her students about what they had learned during their trip abroad. She talked at length about the students’ description of their own growth but ended her statement by saying, “Of course, I don’t have any data on this.”

I hear that line so often when talking with faculty or student affairs staff about their experiences with students.  And although I’m sure I’ll say this again at some point (and hopefully in a kind and caring way!), I just wanted to quickly say to anyone reading this blog today . . . Data is information.  Sometimes information comes in the form of numbers. Sometimes information comes in the form of comments.  Sometimes information comes in the form of student assignments.  Data does not have to be numbers. In fact, sometimes the worst data out there comes in the form of numbers. So if you have information about students that comes from data you have gathered, then you have data. It might be indicative of something that a broader swath of students experience, or it might just be illustrative of the small group of students that provided that data.  But it’s ALL data.

The subtitle of this blog is “using evidence to improve student learning.”  Evidence comes in all forms, and information that qualifies as evidence must go through a vetting process that doesn’t have to be numerical. So if you have data that you think qualifies as evidence – bring it!  Please don’t hesitate or worse, short-change yourself and your work, by thinking that it only qualifies if it comes in numbers.  That’s just not so.

Make it a good day,

Mark

When does a stereotype lose it’s margin of truth?

As we tumble down the back side of the fall term, I know that the potential value of a long and involved blog post drops like a stone in the face of the ten things that have to be solved right now(!).  So I’m gonna just roll out one big-picture data point and let you mull it over when you get a chance to breathe.

Remember that economic collapse that wrecked the economy and scared the pants off of tuition dependent college like us?  Yeah, sorry – not the best way to start the week.  Although there are a few reasons to think that the American economy might be slowly pulling out of its nose dive, we all know that the ripple effects haven’t abated much.

I’ve been trolling our three years of senior survey data lately to look for trends that might be worth noting (see last week’s post on a couple of general education items).  This morning one set of numbers really jumped out at me; the increasing proportion of our students who qualify for Pell Grants – a group of students who almost certainly wouldn’t be at Augustana if it weren’t for need-based financial aid from the federal government and the state of Illinois through the MAP grant program.

The table below shows the increase in this aspect of our student population over the past three classes.  I’ve included the actual numbers in parentheses to add some perspective.

Cohort % Pell Recipients as entering freshmen % Pell Recipients as graduating seniors
Fall of 2008 – Spring of 2012 12.5% (80/641) 10.7% (51/476)
Fall of 2009 – Spring of 2013 17.5% (108/616) 14.2% (66/465)
Fall of 2010 – Spring of 2014 24.6% (185/753) 22.4% (119/532)

You can see that over a relatively short period of time, we’ve roughly doubled the proportion of students who receive Pell Grants (and therefore also received MAP Grants from the state of Illinois).  If you look closely, you can see a hint of this lower socioeconomic status on retention and graduation, since the proportion of these students shrinks over the course of four years.

Just in case you were wondering, 26.2% of our student body (655/2500) of our current overall student body is receiving a Pell Grant.  And among our newest class of freshmen, 28.1% are receiving Pell Grants.

The headline of this blog referred to the stereotype that we like to throw around about our students coming from the Chicago suburbs.  Sometimes I hear that stereotype dressed up with healthy dose of wealth, homogeneity, and entitlement.  I’m not trying to suggest that those students don’t exist at Augustana.  Of course they do.  But the thing about these numbers that hit me was this:

  • We have a trend in our data that suggests a growing proportion (now over a quarter of our student body) of our students that, at least on one critical dimension, don’t conform to that stereotype.
  • Are we adapting our expectations and interactions with our students to match what we know about from whence they come?

Make it a good day,

Mark

What is the role of general education? Some ominous shadows in the data

Despite a genuine commitment to a liberal arts mission, at times it seems easier said than done. On one hand, public fretting (some of it well founded) about unemployed and apparently unemployable college graduates has made some suggest that a college education should focus more of its coursework on preparation for a specific career. On the other hand, the proliferation of knowledge and sub-disciplines within many academic fields translates into more knowledge that faculty often believe (sometimes rightly) need to be added to the range of concepts covered within a particular major. Both of these tangible pressures bolster the argument for expanding the footprint of the major. By comparison, the counter-arguments for maintaining a robust general education program tend to be more abstract and sadly, rarely stand a chance.

Two trends (one macro and one micro) highlight the declining clout of general education. First, the number of U.S. colleges classified as liberal arts colleges has dropped substantially in the last several decades (from 212 to 130). Most of this change involves institutions that expanded their educational offerings into more vocational and pre-professional programs. Closer to home, the proportion of Augustana students who earn at least two majors continues to increase (over 45% of graduates in 2014). In the case of Augustana students, our double majors don’t stay in college longer than everyone else, they just concentrate more of the credits they earn in specific areas.

During last year’s conversations about the relative impact of general education and potential improvements that could be made, some seemed to suggest that our general education program was not in need of revisions. One of the questions posed was whether or not our senior survey data might provide evidence to inform this conversation. Now that we have a third year of senior survey findings, I thought it might be useful to explore the responses to the survey’s general education items and look for any patterns or hints of trends. I’m not sure that the findings below provide definitive answers, but I hope they will further inform the discussion and direction of the general education conversation at Augustana.

The Augustana Senior Survey includes six questions intended to assess the nature of students’ experiences in their non-major or general education courses. Interestingly, the lowest average response score over the last three years came from the 2014 seniors on five of the six questions. Further analysis indicated that the drop from highest to lowest score was statistically significant for four of those questions. They are listed in the table below.

Senior Survey Gen Ed Question 2012 2013 2014
The skills I learned in my general education courses helped me succeed in my major courses. (response options – strongly disagree, disagree, neutral, agree, strongly agree) 3.55 3.43 3.38
My classes outside my major(s) challenged me to produce my best academic work. (response options – strongly disagree, disagree, neutral, agree, strongly agree) 3.57 3.53 3.44
In your non-major courses, about how often were you asked to include different perspectives (different races, religions, genders, political beliefs, etc.) in class discussions or writing assignments? (response options – never, rarely, sometimes, often, very often) 3.50 3.52 3.41
About how often did you discuss ideas from your non-major courses with faculty members outside of class? (response options – never, rarely, sometimes, often, very often) 2.88 2.82 2.76

I fully admit that three years of data is not nearly enough to make predictive claims or produce some sort of smoking gun. However, it is enough data to begin triangulating these findings with others (everything from observational to rigorously quantitative data) and look for evidence of multiple findings moving in the same direction. This can be a particularly effective way to identify early “shadows” in the data and give us time to consider their implications in a less reactive environment.

It would be entirely reasonable to expect some fluctuation on average response scores for individual items across multiple years. But it struck me as curious that the responses to so many of the general education items – questions that I think represent the way that we imagine our general education courses functioning at a liberal arts college – moved together in a negative direction.

What might explain this phenomenon? Is it a function of our students feeling an increased pressure to focus on career preparation? Could it be a function of our own subtle leanings toward areas of our own expertise? Or could it be that we lack a clear sense of exactly how our general education requirements link together to form the kind of integrated breadth of understanding that would ultimately produce the ideal liberally educated student?

I don’t know the answer to any of these questions. But these findings did make me think again about our discussion of the role of general education and the degree to which we may need to revisit our commitment to 1) the role of general education and 2) the way we ensure that our general education program helps students develop all of the knowledge, skills, and dispositions that we know are critical to their success after graduation.

Make it a good day,

Mark

 

Retention, part deux: Freshman survey data to the rescue!

In last week’s Delicious Ambiguity post we dove into the deep end of the data pool regarding retention and the complexity of the problem. Our institutional data shows that the factors shaping our student’s decisions to withdraw or persist are influenced by characteristics that students bring with them to college as well as experiences that they have during their first year. Moreover, it’s clear that addressing this issue requires “all hands on deck” if we are to make any demonstrable progress.

But knowing that information by itself leaves us far short of actually knowing what to do differently. For us to improve our retention rates we need to know which student experiences matter most in shaping their decision to persist. We need to identify specific experiences over which we have substantial and concrete influence. Information about more general experiences, even if they are specific to one aspect of the college experience, is not enough. For example, both of the items below predict our students’ general sense of belonging on campus.

  • “My day to day experiences in my residence hall have helped me feel like I fit in at Augustana.”
  • “I know that my Community Adviser (CA) cares about how I am doing at Augustana.”

Although both findings might appear interesting, the item addressing our CA’s impact on students’ sense of well-being is more specifically prescriptive, providing tangible guidance for designing the role of CAs as well as the way that we select, train, and assess their efforts.

Similarly, if we can collect and link granular experience data to bigger picture retention data, we will be more likely to glean specific direction from our data analyses that ultimately helps us improve retention. This was our big aspiration when we altered our freshman survey’s design last year, gathering more specific experience data about academic acclimation and social integration midway through the first year. After analyzing last year’s responses, I’m excited to share several specifically actionable findings that appear to increase the likelihood of persistence.

After last week’s examination of some retention trends involving race, gender, socioeconomic status (SES), and pre-college preparation, we applied those findings to run some statistical models that combined data from our institutional data, our student readiness survey, and our mid-year freshman survey. All of the findings I share below hold true after taking into account race, gender, SES, pre-college preparation (ACT). We then added items from the freshman data that might influence retention above and beyond those four pre-college characteristics. In the end, two items produced statistically significant effects.

The first item producing a statistically significant positive effect on retention addressed a very specific aspect of the LSFY/Honors experience.

  • My LSFY/Honors instructor helped me develop at least one specific way to be a more successful college student.

Just as so many other researchers have found previously, students often need guidance in figuring out how to successfully navigate college. This means so much more than just knowing how to react to trouble in a class or with a roommate. Instead, this means knowing how to take control of the experience in order to make the most of it in preparation for life next month, next term, or next year.  Our finding suggests that LSFY and Honors instructors who taught students at least one specific way to proactively engage college as a student actually contributed directly to student persistence. And this finding held regardless of incoming ACT score, suggesting that this kind of learning is valuable for all students no matter their pre-college academic preparation.  As LSFY continually explores ways to make that course more effective, this finding seems well worth incorporating into that discussion.

The second item that produced a statistically significant effect addressed a more general sense of social integration.

  • I feel like I belong on campus.

Although this is interesting, we needed to dig further to come up with more concrete guidance toward future improvement. After peeling back another layer of the onion, we found that very kind of guidance.

This time, in addition to accounting for race, gender, SES, and ACT score, we decided to add comfort with social interaction to the mix. Interestingly, in the end comfort with social interaction still produced a statistically significant positive effect, suggesting that despite everything that we might do to influence students’ sense of belonging, more reserved students are likely to still feel less of a sense of belonging than more outgoing students at the midpoint of the first year. However, this effect appears to vanish by the end of the first year, supporting the contention that more reserved students may simply need more time to find their niche on campus.

Under these analytic conditions, we found granular experience guidance for faculty, both as instructors and as advisers, and for student affairs professionals that appear to influence student’s sense of belonging. The two items addressing faculty interactions with students were:

  • My first year adviser made me feel like I could succeed at Augustana.
  • How often have your instructors pointed out something you did well on an assignment or in class?

Again, these findings held even after accounting for incoming ACT score. In other words, regardless of a student’s academic “ability,” faculty communicating to students that they can succeed and pointing to something that they have done well appears to contribute to a student’s sense of belonging on campus. This doesn’t mean that faculty should pull punches or tell students that they are doing well when they are not. Instead, this suggests to me the faculty play a critical role in contributing to student’s belief that they can succeed and then finding positive reinforcement to show them the way.

We found two items predicting a sense of belonging on campus that provide some concrete guidance for working with students outside of class.

  • Fall Connection provided the start I needed to succeed academically at Augustana.
  • I know that my Community Adviser cares about how I am doing at Augustana.

Although the item addressing Fall Connection (now called Welcome Week) seems fairly general, I think it further emphasizes the importance of the changes introduced this year to increase the emphasis on academic preparation. Based on last year’s data (prior to this elevated academic emphasis), this aspect of Fall Connection mattered significantly. In addition, I am particularly intrigued by the nature of the CAs impact on students’ sense of belonging. This kind of guidance provides pretty clear direction in designing the nature of CAs conversations with students.

All of these findings together simply confirm that we all play a significant role in shaping our students’ decision to persist at Augustana College. I hope we can find ways to further convert these findings into concrete action. As with so many other aspects of college students’ experience, it’s not what they do; it’s how they experience what they do.

Make it a good day,

Mark

Let’s talk retention (a.k.a., how to start a fight on campus)

Ok, so that headline might sound a little dramatic. Yet even on the most collegial of campuses, a serious conversation about retention rates – especially if that number has gone in the wrong direction over the last year or two – can quickly devolve into wave of finger pointing and rekindle a litany of old grudges.

We’ve all heard the off-handed comments. “If admissions would just recruit better,” “If faculty would just teach better,” “If student affairs would just help students fit in better,” “If financial aid would just award more money.” Lucky for me, all of these assertions are testable claims. And since we have the data . . . (cue maniacal laughter).

Yet using our data to find a culprit would fly in the face of everything that we are supposed to be about. We say that our students learn because of the holistic nature of the Augustana experience. And analyzing our student data by individual characteristics implies that our students are somehow one-dimensional robots. Most importantly, if we want to improve, then we have to assess with the specific intent of starting a conversation – not ending it. That means that we have to approach this question with the assumption that we are all critical contributors to retaining students.

This year’s first-to-second year retention rate isn’t great.  At 82.9%, it’s the lowest it’s been in five years. In the context of the Augustana 2020′s target retention rate of 90%, there is certainly reason for raised eyebrows. To understand what is going on underneath that overall number, it’s worth looking at our data in a way that mirrors the students’ interaction with Augustana up until they decide to stay or leave. So let’s organize these trends into two categories: students’ pre-college demographic traits and the students’ first year experiences.

Scholars of retention research generally point to five pre-college demographic traits that most powerfully impact retention: gender, race, socioeconomic status, academic preparation, and first generation status. Below is five-year trend data across these categories.

2009 2010 2011 2012 2013
Overall 87.8% 87.6% 84.4% 84.9% 82.9%
Female 91.2% 89.6% 85.7% 90.1% 82.7%
Male 83.6% 85.0% 82.8% 78.6% 83.2%
White 88.1% 89.0% 84.4% 85.8% 84.2%
Multicultural 87.0% 82.6% 85.6% 81.3% 78.4%
Pell grant recipient 78.4% 83.3% 84.0% 81.3% 80.8%
Only qualified for loans 88.7% 87.4% 83.5% 83.3% 81.2%
Did not qualify for need aid 90.6% 90.6% 86.2% 89.5% 86.7%
First generation data not collected until 2012 83.0% 80.8%
ACT <= 22 77.8% 82.1% 83.3% 75.0% 78.6%
ACT 23-25 90.5% 89.5% 84.3% 87.7% 84.9%
ACT 26-27 92.4% 88.2% 83.0% 90.4% 83.8%
ACT >=28 91.0% 90.9% 89.0% 87.0% 82.2%
Test Optional 76.7% 75.0% 75.8% 81.3% 84.8%
ACT top three quartiles 91.0% 89.9% 85.3% 88.3% 83.9%

As you can see, our own data suggests a more complicated picture. Although nationally women persist at higher rates than men, our data flipped last year when persistence among men actually eclipsed persistence among women. Our retention rate for multicultural students (our euphemism for non-white students) has trended steadily downward, a fact made more pressing by a steady increase in the number of multicultural students. Although we haven’t tracked first-generation status for more than a few years, this retention rate has also dropped while the number of first generation students has increased. While our retention rate of Pell Grant recipients (those students with the highest need) has increased slightly, the retention rates of students who only qualified for loans has dropped steadily. At the same time, the retention rate of students from the highest socioeconomic status has dropped a bit.

Finally, academic preparation retention rates paint an interesting picture. The national data would suggest that our worst retention rates should be among those students who come from the lowest ACT quartile. At Augustana, those students’ retention rates are also lower and haven’t changed much. By contrast, the retention rates of students from each of the other three quartiles, although they are still higher than the lowest quartile, dropped substantially between 2012 and 2013. Interestingly, the retention rate of the students who applied test-optional has gone up almost 8 points over the past five years.

But students’ likelihood of persisting to their second year is not etched in stone before they start college. Another way to look at some of these trends is to examine the characteristics of the students who leave against those traits that might indicate an experience that differs from the mainstream in some important way – especially if we know that this difference in experience might affect the calculus by which the student determines whether it is worth the time, money, and emotional investment to stay. So as you look through this table, remember that these percentages are the proportion of departed students who fit each category.

2009 2010 2011 2012 2013
cumulative GPA was below 2.5 57.3% 50.5% 47.3% 43.4% 39.3%
Male 60.0% 51.6% 47.3% 63.6% 45.8%
Female 40.0% 48.4% 52.7% 36.4% 54.2%
White 68.0% 73.1% 78.2% 71.7% 69.2%
Multicultural 9.3% 21.5% 16.4% 28.3% 30.8%
Pell grant recipient 29.3% 32.3% 25.5% 31.3% 31.8%
Only qualified for loans 36.0% 38.7% 47.3% 46.5% 42.1%
Did not qualify for need aid 34.7% 29.0% 27.3% 22.2% 26.2%
First Generation 24.2% 30.8%
ACT <=22 32.0% 28.0% 20.9% 30.3% 25.2%
ACT 23-25 21.3% 23.7% 28.2% 23.2% 26.2%
ACT 26-27 12.0% 19.4% 20.9% 11.1% 15.9%
ACT >=28 21.3% 19.4% 18.2% 24.2% 24.3%

Much of this data corroborates what we saw in the examination of retention rates by pre-college characteristics in the case of gender, race, socioeconomic status, first generation status, and academic preparation. However, one new trend adds some interesting nuance to the impact of the first year experience on retention. If we look at the proportion of departing students who are also in academic difficulty when they left, there is a clear differences of almost 20 percentage points among those who had less than a 2.5 GPA when they left. In other words, far fewer of our departing students are in a position where their grades might be the primary reason for their departure. This suggests to me that, if they aren’t departing because of grades, then there are other key elements of the first year experience that would be primary contributors to the decision to depart.

We aren’t going to answer the question of what is negatively impacting retention today. Even if we were to pinpoint a significant factor in a particular year, because the nature of each class differs, evidence from one year might be entirely useless the next. My point today is simply to highlight the degree to which all of us impact retention together.

I’d like to think that on some level we know that finger pointing is foolish. Yet in an environment where we are simultaneously immersed in our own silos and entirely dependent on the efforts of others (e.g., faculty don’t have a job if admissions doesn’t recruit anyone), it doesn’t seem all too surprising that such behavior (especially if budgets are under threat) might surface despite the best of intentions. So maybe if you hear someone grouse about retention rates and “rounding up the usual suspects,” you’ll remind them that we are in this together. If we fail to improve, it won’t be because someone didn’t do their job – it will be because we all didn’t pull together.

Make it a good day,

Mark

 

 

 

 

Swimming in the 2014 Senior Survey data!

I think I’ve resigned myself to the fact that it is nearly impossible to explain to someone who hasn’t experienced Augustana’s 10-week terms what it feels like to go from standing still less than a month ago to flying by the seat of your pants at week four. But here it is verging on mid-term time, and I’m hurtling through space trying everything I can to get my bearings!

So, to show that the Institutional Research and Assessment staff (AKA Kimberly and I) doesn’t just sit around dreaming up ways to collect more data, I thought I’d share with you . . . more data. (I guess this doesn’t really debunk my assumptions about your assumptions, does it.)

Every spring, we ask our graduating seniors to complete a survey that asks all sort of questions about their experiences at Augustana. In addition, we ask a few important questions that we’ve found to be useful outcome questions (would you choose Augustana again, is your post-graduate plan a good fit for who you are and where you want your life to go, and do you already have a job or grad school place).

It takes a lot of work to process this data into a readable report, but it’s finally finished and posted on the IR web page.  Here is the direct link to the 2014 Senior Survey results.

2014 Senior Survey Report and Findings

Now you could jump on that link right away and start swimming in the data – percentages, averages, and standard deviations (oh, my!). And you might survive the experience, although your eyes will probably start to glaze over as you look at mean score after mean score and your brain will likely start to go soft wondering (rightly so) what exactly each average score means – is it good? is is bad? is it just right? (sort of like Goldilocks in the house of the three bears . . . replacing the bowls of porridge with excel spreadsheets, of course).

So if I may, let me make a suggestion that I hope will make some of this data more meaningful.  Instead of looking at the numbers first, put a sheet of paper over the numbers and look at the questions first.  Reflect on why each question might matter for students and what might be the “about right” distribution of responses.  Pick out 3-5 questions that seem particularly interesting to you.

THEN take away the sheet of paper covering the numbers. Do your musings match up with the average score or the distribution of responses?  What more would you like to know that might help you get a better handle on what we could do to improve, if the difference between your reflections and the actual score suggests that institutional improvement might be valuable?

This data isn’t of much use if it doesn’t help us get better at what we do. And you – the people on the ground floor who are working with students every day – are the ones who are ideally suited to tackle this data, jump into this process, and benefit from the results of your efforts.

If you have any questions, comments, suggestions, or criticisms (I prefer to think of it as constructive feedback!) about the senior survey, PLEASE PLEASE PLEASE contact us – Kimberly or me – in the IR office. Nothing we have built is so important that it can’t be changed . . . especially if those changes make the survey better.

Make is a good day,

Mark