Taking Action with Confidence

Civitas Learning Civitas Learning Space Data Science Outreach

Share this Post

In Part 1 of this series, we looked at the power of our predictions to detect persistence risk and showed you how these predictive models were detecting, on average, 82 percent of the at-risk students on Day One of the semester, compared to just 21 percent detection with the traditional trigger of a cumulative GPA. DOWNLOAD So now that we know that the predictions are trustworthy, how do we use them to take action with confidence early in the semester? In my work, I sometimes see partners get caught up looking for the one big thing causing student risk – the one big unicorn insight – but the reality is it’s not going to be one thing.  There are many different factors that affect student success, and they often vary by student and by time.  The prediction itself is the most actionable insight for optimizing outreach and intervention.  While we can combine the prediction with a data-informed insight to help narrow focus, simply using the prediction can be a powerful tool.  We can use the prediction to open the door to understand the “why” of student risk and create the human connection that helps students persist. dashboard view of prediction score

Try a nudge campaign

One of the most effective ways to use the predictions is to run nudge campaigns.  Nudge campaigns are a communication strategy that an institution designs to prompt a specific group of students to achieve a specific, measurable outcome.  So, what is a nudge?
“Nudges are small pushes in the right direction that do not require prescribed actions, but encourage certain behaviors. When students are presented with a nudge sent from a trusted person at your institution, they have the freedom to make their own choices with information about behaviors we know are more strongly associated with positive persistence and graduation outcomes.” – George D. Kuh
In the publication, Student Success in College: Creating Conditions that Matter, George Kuh and team define nudges as small pushes in the right direction. We have found that nudge campaigns that use the prediction in combination with mindset messaging and are targeted by student segment are very efficient and effective at creating lifts in persistence.
Example Nudge Campaign
For example, one institution found that registering less than 40 days before the start of the term was a top Powerful Predictor. At this institution, the later you register, the larger the risk. In addition, they found this was more predictive for African American students. Increasing persistence and success among their African American population was a key strategic goal, so they decided to use this insight in combination with the prediction and conduct outreach. They ran a nudge campaign focused on African American students, segmented across prediction scores. They differentiated the message based on each student’s predicted likelihood to persist. For example, the message for low and very low likelihood to persist focused on encouragement and empathy, but made clear registration was the student’s responsibility and ultimate success was in each student’s hands. This nudge campaign used messages such as ‘we are here to support you’’we know this can be hard’’it’s in your hands.’ The partner ran the campaign using Illume and Inspire and saw a 1.6%(pp) lift in persistence. This equated to 46 additional persisting African American students and $38,474.31 in retained revenue, just by trusting the prediction and acting on it
What We’re Learning
As our partners are digging in, trusting the predictions, and taking action, we are learning with them.  Here are a few of the key take-aways from this work thus far:
  1. Don’t overthink it – start simple. Start by using the prediction in combination with a student group that is already identified or top-of-mind at the institution. Use mindset-based messaging and a do-no-harm approach.
  2. Focus on the human-to-human connection. A prediction doesn’t tell you exactly why a student is at risk, but that they are at risk. The prediction helps you to open the door and facilitate a human connection to better understand the reason for each student. Whether it is financial, psycho-social, life or logistics – the human connection is what will make the difference.
  3. Language matters – be creative! Be empathetic, conversational, and human. One partner used the prediction score to reach out to high achieving students who had low likelihood to persist. Their subject line was ‘We are proud of you!’ and the email congratulated them for working hard, while acknowledging that even strong students face challenges. Our research had shown this partner that an average of 44% of the students leaving higher education have GPAs of 3.0 or higher. The quick two-email campaign over 30 days saw a 9.5%(pp) lift in persistence and a retained revenue of approximately $172,000 for the institution.
  4. Go wide. Send nudges to the biggest groups possible so you can both have a larger impact and know whether it worked by getting to a measure of statistical significance. (The bigger the group, the smaller the lift needed to reach statistical significance.)
Getting to action is the most important step in getting to student outcomes. There is incredible opportunity in trusting the prediction, taking thoughtful, do-no-harm action, measuring the results, and iterating.  In doing this, together we are moving to a new approach of rapid learning and increasing aggregate student success gains. Increased student opportunity and improved outcomes await. Read Part 3 of this series focused on achieving outcomes through tailored nudges.

Laura Malcolm

Laura Malcolm is Senior VP, Outcomes & Strategy, for Civitas Learning. She has 18 years of education and product design experience building technology products to serve partner institutions, faculty and students in the attainment of their educational goals. At Civitas Learning, Laura leads the design and delivery of solutions and applications that leverage insight analytics to improve student outcomes. Laura is a two-time CODiE Award recipient for product design.