campus quad

The Importance of Institution-Specific Predictions

Civitas Learning Civitas Learning Space Data Science Research

Share this Post

There’s a lot to be excited about when you work at Civitas Learning. One of my favorite parts of the work is traveling to our partner institutions – In the past year I visited the faculty, staff and administrators at more than 50 of our partner institutions. What strikes me most is how very unique each institution is. In many ways, they appear so similar on the surface, and the practice in the U.S. has certainly been to treat them all similarly with regard to best practices and generalized suggestions for improving outcomes. But they are not alike, and when you actually take the time to dive into the data and understand them as individual institutions the differences are fascinating, and important to understand.
Two Similar Looking Institutions have Different Top Predictors
This really came to life when I spent a week sharing my time between two large institutions serving predominantly online student populations. The work we were doing focused on using Illume™ on the Student Insights Platform™, so we were able to see what each model calculated as the top Powerful Predictors of student persistence and success at the institution level. These institutions had similar student profiles, and similar retention rates, but what came up in their top 10 Powerful Predictors was radically different. University A’s top 10 predictors were dominated (9 out of 10) by LMS features – Average Grade (per day), Average Discrete days of LMS Activity, Average Number of Discussion Posts, etc. However, working with University B – which would appear to be serving a similar student body in a similar construct – I found LMS features did not rise into the top 10 predictors (though they were present in the top 25). Instead, University B had heavy academic and behavioral predictors in the top 10. These institutions, so similar in so many ways, had entirely different drivers for student persistence and success. If University B were to use University A’s plan for student success it would be unlikely to move the needle because what is most impactful for students is dramatically different between these two institutions. whiteboard images of two institutions Rather than trying to focus on everything and not know for certain what ten things matter most to student success, we can tell from behavior (with the help of data science) what will be most impactful. We can, and are, using this knowledge to create better, more specific programming and communication to help students. Powerful predictors tell a story. They tell us what kind of demographic, academic, behavioral, and financial factors will most influence students. No amount of telling the story changes the ending though. We have to drive these insights to informed action. The needle is moving and they know why.
Basing Strategy on the Differences
Accepting that we are different as institutions is important, and a critical first step in this work. Understanding how to act on these differences is equally as important. Seeing a model so heavily balanced on LMS features leads us toward a strategy that heavily engages students in their online classroom. The LMS data helps provide us with the information we need to be able to tell students how to succeed better. Imagine how great it would be to tell students “We know at our institution, that students like you, who are really successful in their first term, login four out of seven days in the first week of class to become acclimated with the class materials, meet the professor, and post their first discussion post. Can you do that this week?” Illume’s powerful predictors are an indication of the institutional culture that is strongest for that segment of students. Let’s walk through an example of top predictors from two community colleges known for their successful work with analytics. They both serve a combination of working adults and recent high school graduates. We filtered Illume to only examine the predictors for female students on an associate’s degree track. The differences are astounding. In this example, at Institution A the most important watch points are standard deviation in term GPA (from prior terms), the term season and the financial aid per attempted credit hour. If we were watching for these at the other college we would broadly miss the most important signals in the data. For Institution B, the predictors that will let us help their female associate’s degree candidates are credits attempted, credits earned and LMS activity.
Cultural Differences
The footprints of the culture they have for the women at these institutions is different, and this is the first piece of evidence that there is a difference in culture, the drivers of success, and the most effective interventions that could improve the experience for these women. We have to become sociologists of our own students – Observe and talk to these students to help bolster our understanding of the footprints we followed. Illume Dashboard | Institution A Illume Dashboard | Institution B When we started this work at Civitas Learning we were thinking we’d find silver bullets – certain big indicators and actions that ring true across every institution. While some carry in the top ten from institution to institution, the data are truth – each institution is unique. And diving into the data lets us know who to reach out to, how, and when. The good news is silver bullets aren’t important to success if you’re willing to do the work. I look forward to diving into some of the interesting findings we are seeing with Illume that will affect policy and practice. Go look at your own Illume today – What’s driving your culture? Does that surprise you? What did you expect to see? What is most powerful for different groups of students? In this work it’s not all about giant “aha” moments. It’s about optimizing the data and exploring it, gaining insights and understanding, taking informed action that results in marginal gains (which add up significantly for an institution’s bottom line as well as the success of your students) and learning from it.

Top Banner Photo: University of Washington: Campus Quad by Joe Wolf used by permission CreativeCommons BY-ND 2.0.

Mara Richard

Mara is Director of Partner Services for Civitas International, and works in our UK Office. At the time this article was written she worked with our partners here in the U.S. as a partner success consultant.