In this episode of 3 Big Questions we visit with Dr. Marie Cini, Provost and Senior Vice President, UMUC. 3 Big Questions is a recurring feature in the Civitas Learning Space.
UMUC has been in the Business Intelligence (BI) work for a long time…in the build vs. buy decision why did you choose to partner with Civitas Learning?
As a University we have been engaging in data-based decision making for a long time now. For example, in the late 1990s UMUC began engaging in research related to effective learning in the online environment, and we used our findings to design our faculty training for online teaching. In our traditional institutional research work we track the impact of various policies on student success and make changes accordingly. As another example, we now limit students to registration four days before the start of a course. We found that letting students register later than that resulted in far higher failure and withdrawal rates. So, as I said–we have been data-driven for a long time.
But with the advent of our new data warehouse and the ability to access student behaviors in the online classroom, we have a far richer set of data to discover what predicts student success, and we are able to engage in more effective interventions. Civitas Learning contacted us early in their history and offered to demonstrate the power of predictive modeling on anonymized data. We essentially “kicked the tires” and found the Civitas modeling very helpful in our understanding of course completion rates.
What have been some of the significant learnings from your work in predictive analytics so far?
I said at EDUCAUSE that I thought developing the model would be difficult and designing effective interventions would be easy once we knew who to target. Instead, we found that the predictive models, while not exactly easy to develop, came together pretty quickly. Right now on day zero of an online course we can predict with about 84 percent accuracy those students likely to pass the course, those who may struggle, and those at high risk of failure. At day 7, this rate goes up to 86 percent. We have been working on a series of interventions starting with email messaging to see what will be most effective in helping more students succeed–which is our goal in this work. One recent finding, which we have to more fully test, is that students seem more responsive to communications from advisors than from program administrators. This makes sense since the program administrators are often unknown to the students, whereas they are more used to speaking with advisors.
We have more variables to test, based on a framework for student success that we are using to test our hypotheses. Interventions in the future will test the source of the messages, the type of messages (texts versus emails, for example), as well as more overarching interventions such as different instructional methods and even course designs.
Rather than assume that the student is at fault for being in a high risk category, we want to design interventions that are targeted to specific segments of our students. We believe that as long as students are motivated, we should continuously develop and test new ways to help them succeed. We have recently opened the Center for Innovation in Learning, which is an applied research and development unit at UMUC to continuously develop, test, and then implement the most successful methods or technologies that will help students learn.
What were the decisions you made, and are making, regarding the balance between privacy and intervention, and creating a culture of data democracy?
We are very sensitive to the privacy concerns that students might have about the data being used in these models. We don’t want this to be a deterministic exercise; we want to use the data to help students be successful, not use it to filter them out. We also don’t want to simply tell faculty which students may be at risk until we can help faculty develop methods to help those students succeed. The time of big data is here. Higher education is working to harness the power of big data to improve student success, but we have many questions to ask and answer so that we have the right systems, processes, and interventions in place.