Civitas Learning is on a mission to help a million more students learn well and finish strong through insight and action analytics applications such as Degree Map™. Degree Map is specifically designed to elevate the advising conversation by providing a clean, clear snapshot of a student’s degree progress and remaining journey ahead. Through these more informed and personal conversations come better-informed decisions by students. Degree Map’s goal is to provide a path for students to successfully persist semester-over-semester and complete with a credential or transfer pathway in hand.
Austin Community College was Civitas Learning’s pioneer beta partner in the development of Degree Map. Read about the development and roll out of the app on campus in the ACC Learning Brief.
Posting Retention Gains
Civitas Learning recently measured term-over-term persistence improvements contributed by Degree Map at Austin Community College to determine the efficacy of the app in practice. By accounting for terms completed and utilizing propensity score matching over a three-semester period, Degree Map contributed to a statistically significant persistence improvement of 2.42 percentage points overall. Additionally, analysis surfaced a dosage effect in which greater Degree Map usage results in higher persistence. For example, using Degree Map more than 5 times results in a statistically significant 7.3 percentage point increase in persistence.
Because correlation does not imply causation, it’s conceivable that students with Degree Map plans were predisposed to persist due to underlying characteristics that resulted in Degree Map usage (correlation) but were not fostered by Degree Map (causation). To eliminate such selection bias and control more appropriately for causation, pilot and control groups were established using propensity score matching where students between the pilot and control were first matched according to number of terms completed. Since number of terms completed is highly correlated with persistence, matching pilot and control groups based on terms completed controlled for persistence contributions caused by academic experience.
Next, propensity score predictions were used to match pilot and control students based on demographics, census data, transfer information, financial aid, credit load, academic performance, school experience, and major. By using these predictors in propensity score matching, the analysis controlled for demographic, academic, and personal characteristics in an effort to extract Degree Map’s actual contribution to persistence.
Lastly, persistence was measured over a three-term period beginning in Fall 2013 and ending in Fall 2014. Persisting students were defined as those students enrolled in Fall 2013 who graduated or continued in the Spring of 2014 and/or those enrolled in Spring 2014 who graduated or continued to the Fall of 2014.
Results & Analysis
Overall, the term-to-term persistence improvement for students with Degree Map plans was 2.42 percentage points with a p value of 1.01E-10. Further analysis also yielded a dosage effect in which greater usage of Degree Map resulted in higher persistence rates.
|Number of sessions on Degree Map||Persistence Improvement||P Value|
|2||3.28 percentage points||1.8E-5|
|3||5.54 percentage points||1.2E-6|
|4||5.43 percentage points||0.0012|
|5 or more||7.26 percentage points||1.8E-6|
While using Degree Map more increases persistence, repeat sessions in the same semester are not necessary. Most students engaged in Degree Map sessions no more than twice per term with the largest group of users being first-term students. These first-term students using Degree Map experienced a term-to-term persistence improvement of 2.30 percentage points with a p value of 0.00062.
Fidelity to Plan
Overall, the persistence improvement is highest for students who adhere most closely to their plan at the time of registration, adding an additional 1.9 percentage point increase in persistence on average
In summary, students who engage with Degree Map for planning their degree persist at higher rates than students who do not engage with Degree Map. Persistence is further improved when students engage with Degree Map more, and adhere to their plan during registration.