This week gonna need some laughs!

Since it’s finals week, since it’s snowing (AGAIN!), and since you all are going to be busy grading and shoveling for the next several days, I decided this this was as good a Monday as any to share some faux news stories that will hopefully make you laugh and momentarily forget the work piling up outside and inside.

An oldie but goodie from the Onion’s vault . . .

Professor Deeply Hurt by Student’s Evaluation

An Assessment Coordinator’s Dream from the Cronk News . . .

One Learning Outcome to Rule Them All

And finally, another Onion article that cuts it a little close . . .

University Implicated in Checks-for-Degrees Scheme

Hang in there everyone!  See you in a few weeks.

Make it a good day,

Mark

What if early feedback made your students work harder? (Spoiler Alert)

One of the ways that we come up with questions for our freshman and senior surveys is by asking advisers about the concerns that students express to them. If we start to hear a particular theme, we try to find a way to capture that issue in a single question. Then, with the data from that item, we can test to see (1) if it really is a common problem or concern, and (2) whether that issue correlates with other experiences or the broader learning outcomes that we know are important for student success.

This is how we developed the freshman survey item to which students agreed or disagreed on a five point scale, “I had access to my grades and other feedback early enough in the term to adjust my study habits or seek help as necessary.” Numerous students had claimed that they didn’t know that they were struggling in one or more of their classes until very late in the term. They suggested that it wasn’t as if they had known their grades early in the term but hadn’t sought out academic support, but rather that they had either not been assigned enough (or more often, any) graded work to know their academic standing or that their homework wasn’t graded and returned or posted until well after the middle of the term. By the time they received their first substantive grade it was often well past the drop or withdrawal date and there was little time to recover. Moreover, these students often felt the late term pressure and in most cases had emotionally “given up” on the class.

So we inserted this item into last year’s freshman survey. Of the 286 responses, the answers were distributed like this:

  • Strongly Disagree – 30 (10%)
  • Disagree – 64 (22%)
  • Neutral – 77 (27%)
  • Agree – 89 (31%)
  • Strongly Agree – 22 (8%)

After seeing an even more “normal” distribution of responses (i.e., a bar graph of the responses that looks like a bell curve) from this year’s new mid-year freshman survey, I decided that it would be worth looking in more depth at this data to see if this item might be predictive of any other important important student-faculty interactions.

In fact, there were several student-faculty interaction items that were significantly influenced by early access to grades and feedback.  In each case, the more strongly students agreed with having access to grades and feedback, the more strongly they agreed with each of the following statements.

  • My one-on-one interactions with faculty have had a positive influence on my intellectual growth and interest in ideas.
  • My professors were interested in helping students grow in more than academic areas.
  • Faculty and staff at Augustana treated me like an individual.

Moreover, these statistically significant relationships held even after we controlled for incoming ACT score, parents’ education, and the Student Readiness Survey scales for academic habits, academic confidence, and persistence and grit.

By itself, these findings are important. But we found one other result that seems particularly useful. Even after controlling for the effects of the pre-college characteristics listed above, early access to grades and other feedback produced a statistically significant effect to a question that asked, “How often did you work harder than you have in the past in order to meet your instructor’s expectations?”

In other words, students who had access to grades and other feedback early enough in the term to adjust their study habits also indicated that they worked harder than they had in the past to meet their instructor’s expectations.

Why might this be? It might be because the early feedback gives students a chance to check themselves and reflect on their current level of effort. Furthermore, if the feedback comes in the form of an actual grade, it’s data that is difficult to ignore (we all have seen students demonstrate impressive levels of denial or willful blindness). This finding might also reflect the likelihood of an educational environment that emphasizes a continual exchange between the teacher and the student. This kind of environment is much more likely to result in a deeper level of student engagement in a course.

Yes, this finding isn’t absolute proof.  And yes, this is a statistically significant finding from only one cohort of freshmen.  And no, you’ll never get smoking gun evidence that one pedagogical practice will transform your most curmudgeonly student into a singing apostle for your discipline. However, this finding does comport with a compellingly large body of research on effective pedagogy for engaged student learning.

So you want your students to work harder in your class? Insert a substantive graded assignment early in the term. Turn that assignment around quickly with some extended formative feedback. Students will use that feedback to better grasp the degree to which they are working hard enough to meet your expectations. With that information they can adjust their efforts or seek out that academic resources might help them increase their likelihood of success.

Make it a good day,

Mark

Refining the way we deliver the liberal arts

First of all, thanks to everyone who reads Delicious Ambiguity for putting up with a three-part series of posts.  Somehow it seems to me like it requires an extra dose of ego to think that any idea is so big or important that it deserves three separate posts, so I feel sort of sheepish for even trying to pull it off.  More likely, this endeavor could just mean that I’m so damn long-winded that I can’t say anything of substance within my self-imposed word limit.

With that said, this is the “refining” post and is to be the last of the three-part bit I promised on reframing, refocusing, and refining the way that we operationalize the liberal arts. Partly because I’m late with this post and partly because it would be ironic to write a painfully long post about the act of refining, I’m going to make an extra effort to be concise.

I think that we should refine the way that we deliver the liberal arts because the quality of any educator’s effort (i.e., the degree to which an educator influences learning) can’t be separated from the amount of time that an educator has to dedicate to this effort. We know that helping students develop the skills and dispositions necessary to succeed in life requires substantial personal interaction. Over the last decade, it appears that we’ve become a very busy college.  We’ve added a host of legitimate educational and administrative responsibilities at every level, revamped programs and curricula, and seem to constantly on the brink of deciding to change something big. Yet we have taken very little away even as we’ve taken on all of these entirely defensible programs, pedagogies, and policies.  During the last twelve months of strategic planning discussions, the question that many raised, “But what are we going to take away?” seemed sometimes to be as much a lament as it was a serious question.

So I offer a set of findings from one part of last year’s senior survey and a few musing on some potential implications of these findings in the hope that it will at least fuel continued discussion of the things that we should take away.  I know there will always be some who are personally vested in the things that we might decide to take away, but (1) if we are ultimately about the experience of the students, and (2) we know that our ability to provide the best learning experience for students requires us to be efficient in the time we allot to each educational aspect of that experience, then we have to be brave enough as a community to make these choices.

Last spring, after the faculty had approved our college-wide learning outcomes earlier in the fall, I thought it might be useful to get some sort of baseline sense of where students think they develop these skills – not as evidence of where they actually gain these skills, but rather some reflection of the degree to which the experience we offer is as holistic as we’d like to think it is.  Thus, I inserted a set of questions in the senior survey that asked students to identify the experiences where they think they developed each of these learning outcomes.  For each learning outcome, students were given a list of college experiences and were allow to choose as many as applied.  Here are the results.

Disciplinary Knowledge
Major(s) or Minor(s) Courses 93.7%
General Education Courses (AGES) 45.9%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 45.9%
Residence Life Experience 11.3%
Participation in Student Organizations and Clubs 28.5%
Working On or Off Campus 29.5%
Informal Interactions with Faculty and Staff 34.3%
Informal Interactions with Peers 25.4%
Volunteering in the Community 19.8%
Senior Inquiry 55.6%
None of the Above 0.4%
Critical Thinking and Information Literacy
Major(s) or Minor(s) Courses 91.3%
General Education Courses (AGES) 58.4%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 36.4%
Residence Life Experience 10.7%
Participation in Student Organizations and Clubs 23.0%
Working On or Off Campus 25.9%
Informal Interactions with Faculty and Staff 29.7%
Informal Interactions with Peers 26.7%
Volunteering in the Community 11.9%
Senior Inquiry 57.8%
None of the Above 0.4%
Quantitative Literacy
Major(s) or Minor(s) Courses 77.6%
General Education Courses (AGES) 50.5%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 15.2%
Residence Life Experience 3.2%
Participation in Student Organizations and Clubs 8.7%
Working On or Off Campus 16.0%
Informal Interactions with Faculty and Staff 10.9%
Informal Interactions with Peers 8.1%
Volunteering in the Community 3.4%
Senior Inquiry 39.0%
None of the Above 3.4%
Collaborative Leadership
Major(s) or Minor(s) Courses 62.0%
General Education Courses (AGES) 33.1%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 37.4%
Residence Life Experience 27.3%
Participation in Student Organizations and Clubs 65.5%
Working On or Off Campus 46.7%
Informal Interactions with Faculty and Staff 29.9%
Informal Interactions with Peers 43.2%
Volunteering in the Community 40.0%
Senior Inquiry 24.6%
None of the Above 1.4%
Intercultural Competency
Major(s) or Minor(s) Courses 47.1%
General Education Courses (AGES) 48.5%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 46.9%
Residence Life Experience 26.5%
Participation in Student Organizations and Clubs 38.6%
Working On or Off Campus 27.9%
Informal Interactions with Faculty and Staff 19.0%
Informal Interactions with Peers 40.0%
Volunteering in the Community 37.0%
Senior Inquiry 17.2%
None of the Above 5.1%
Communication Competency
Major(s) or Minor(s) Courses 84.4%
General Education Courses (AGES) 60.6%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 41.4%
Residence Life Experience 20.2%
Participation in Student Organizations and Clubs 41.6%
Working On or Off Campus 35.2%
Informal Interactions with Faculty and Staff 40.2%
Informal Interactions with Peers 37.2%
Volunteering in the Community 22.4%
Senior Inquiry 51.3%
None of the Above 1.2%
Creative Thinking
Major(s) or Minor(s) Courses 79.6%
General Education Courses (AGES) 59.2%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 36.6%
Residence Life Experience 13.3%
Participation in Student Organizations and Clubs 36.6%
Working On or Off Campus 26.1%
Informal Interactions with Faculty and Staff 23.0%
Informal Interactions with Peers 27.1%
Volunteering in the Community 17.4%
Senior Inquiry 49.3%
None of the Above 2.6%
Ethical Citizenship
Major(s) or Minor(s) Courses 59.4%
General Education Courses (AGES) 45.7%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 40.8%
Residence Life Experience 25.5%
Participation in Student Organizations and Clubs 42.0%
Working On or Off Campus 36.6%
Informal Interactions with Faculty and Staff 31.9%
Informal Interactions with Peers 39.6%
Volunteering in the Community 42.6%
Senior Inquiry 26.1%
None of the Above 4.8%
Intellectual Curiosity
Major(s) or Minor(s) Courses 87.1%
General Education Courses (AGES) 59.2%
Augie Choice Programs (Study Abroad, Internship, or Undergraduate Research) 49.5%
Residence Life Experience 9.3%
Participation in Student Organizations and Clubs 27.7%
Working On or Off Campus 23.6%
Informal Interactions with Faculty and Staff 42.0%
Informal Interactions with Peers 34.9%
Volunteering in the Community 19.8%
Senior Inquiry 51.1%
None of the Above 2.2%

To some degree, the implications of these findings may lie in the eye of the beholder.  But what jumps out to me is the degree to which major and minor courses seem to do so much of the heavy lifting.  If this is really so, then it is no wonder that faculty would be pressed for time to do much else besides teach students.  However, if we would like to create a more holistic educational experience and do so in a way that allows all of us to be equally involved in the development of our students, then there might be some logic in considering ways for faculty to pass on some of the responsibilities of learning in specific areas.

But another way to read this data is to consider the degree to which students don’t indicate experiences outside of the academic realm more often.  And this might be a more useful way to think about designing a comprehensive college experience that is more effective and more efficient.  In an odd way, we might find that the refinement we need to consider is not that the responsibility for students’ learning across each of the learning outcomes is distributed more exclusively among individual experiences, but rather that each of us explicitly understands our role in contributing (1) a small but crucial building block to students’ development and (2) the sinew that connects that building block to another block that can only be gained through a different experience offered elsewhere on our campus (or an off-campus experience facilitated by someone on campus).

Someone once said that planning to do something is the easy part; the hard part was actually doing it.  I suspect for us this will also be true.  The strategic planning was the easy part.  The hard part will be implementing this plan in a way that accomplishes the learning goals and student success to which we aspire.

So let’s roll up our sleeves.  We’ve got work to do.

Make it a good day,

Mark