Beyond the Plateau: Designing Retention Experiments While the System Is Still Moving

Subscribe: Spotify | Apple Podcasts | Youtube Music | Transcript

Hitting your retention goal feels like progress. But at Slippery Rock University, it’s also a signal to ask harder questions.

In this episode of Next Practices, John Rindy, Assistant Vice President for Career and Academic Progress, and Emily McLain, Associate Director for Academic Success Design, share how they’ve built a culture of continuous experimentation — not just goal-setting — on their campus. 

Slippery Rock has sustained first-year retention above 80% since 2008. And rather than treating that as a finish line, John and Emily treat it as a starting point for the next set of questions.

Their conversation is a practical look at what student success work actually looks like once the obvious fixes are behind you.

When Positive Outcomes Make the Work Harder

Most institutions celebrate retention improvements. Slippery Rock does too. But John and Emily describe something most student success teams don’t talk about openly: the moment your metrics improve is also the moment your work becomes more precise and more difficult.

“The low-hanging fruit changes,” Emily explains. Where they once focused on undeclared students — a population that historically left at rates around 45% in Pennsylvania’s state system — they now see those students persisting at closer to 85%. That win is real. But it means the next opportunity is harder to spot, and easier to miss if you’ve stopped looking.

John and Emily don’t stop looking. They hold a monthly 90-minute data meeting with their director for analytics and decision support. No agenda other than writing research questions on a board and digging into the data — including the question: should we design a retention experiment around this?

That discipline is what led them to their next focal point: second-semester drop-off. Slippery Rock loses roughly 5% of its first-year class in the first semester. In the second, that number climbs to 9%. “To me,” John says, “we’ve just uncovered a low-hanging fruit.” The difference is, finding it required someone still paying attention after the headline numbers looked good.

Experiments, Not Initiatives

One of the clearest ideas in this conversation is the distinction between running an initiative and designing an experiment. Initiatives get launched and evaluated. Experiments get iterated.

Emily puts it directly: “We are prototyping things, we are continuously assessing, and then we are building and changing them. We allow an iterative process to just be how we work.” That mindset shapes everything from how they use data to how they handle setbacks — including when data challenges a faculty member’s assumptions about their own course.

A concrete example: John noticed through Civitas Learning’s Initiative Impact Analysis that fraternity and sorority participation had a measurable positive effect on semester-to-semester persistence. Descriptively, of roughly 70 students who joined Greek life in one fall, 69 returned the following semester. That pattern raised a new question — were they missing an opportunity to partner with student organizations more broadly on retention?

That question led to a designed experiment with Women in Business (WIB), a new organization in Slippery Rock’s College of Business. Rather than waiting to see how the club developed, John brought a specific proposition to its student leaders: let’s build deliberate mentoring, executive-in-residence programming, and exclusive engagement into your organization’s first year — and then look at what happens to the persistence rates of women in those majors. Faculty advisors are on board. The data infrastructure is in place. And John already sees the bigger possibility: if this works, Slippery Rock has more than 200 clubs and organizations that could become retention partners.

Moving from Insight to Action

John and Emily are candid about something that stalls progress at many institutions: you can measure everything and still not act on it.

Data is most useful when it’s attached to a question you’re genuinely trying to answer. When Emily brought course-level signals to a faculty member — flagging a pattern where a grade of C was functioning as a persistence tipping point — the initial reaction was defensive. Her approach was to pause, put herself in the faculty member’s position, and reframe: this isn’t about your teaching. It’s about getting to students who are struggling before they disappear.

That reframe is possible because John and Emily have spent years building the kind of campus relationships where those conversations can happen. “People know us,” John says. “They know that we have good hearts for the institution.” Trust, built over time, is what makes it possible to bring data into uncomfortable conversations and come out with a partner rather than a detractor.

Challenging the Assumptions Underneath the Numbers

One of the most thought-provoking moments in the episode is when John asks why some institutions retain 98% of their students — and what assumptions Slippery Rock might be carrying that explain the gap between their numbers and that ceiling.

It’s the kind of question that’s easy to dismiss. But John and Emily take it seriously. They’ve already seen assumptions shape outcomes in unexpected ways. Female students traditionally persist at higher rates than male students. But recently, Slippery Rock flipped that pattern — and Emily and John are running experiments specifically designed to understand and respond to that shift. The data didn’t confirm what they expected. So they changed course.

That willingness to be surprised by data — and to use the surprise as a design prompt rather than an anomaly to explain away — is what makes their approach distinct.

What This Looks Like in Practice

A few practices that stand out from this conversation:

• Take support to where students already are. Tutoring doesn’t wait for requests. At Slippery Rock, tutors set up outside high-DFW courses on the first day of class and introduce themselves directly to students walking in. The goal is to normalize help-seeking before the stigma around it has a chance to take hold.
• Use quartiles instead of broad risk categories. As persistence rates improve, generic “at-risk” labels become less useful. Emily and John shifted to quartile-based analysis — which always surfaces a bottom group, regardless of how strong the overall numbers are. The goal line moves, but the need for precision doesn’t disappear.
• Treat campus partners the way you treat students. Emily draws on a framework from Relationship-Rich Education: students need genuine welcome, inspiration to learn, a web of meaningful relationships, and space to explore questions of meaning and purpose. Her insight is that the same principles apply to faculty and staff partners. When she approaches campus colleagues with that orientation, the quality of collaboration changes.

The work doesn’t settle down

“The moment we stop learning about ourselves, learning about our students, learning about our data — we are going to freeze where we are.” That’s John’s framing, drawn from philosopher Eric Hoffer. It’s a clear articulation of what makes Slippery Rock’s approach sustainable: not a tool, not a metric, but a refusal to treat student success as a problem you solve once.

If your institution is approaching a retention plateau — or already sitting on one — this episode is worth your time.

Links & Resources:

• Slippery Rock University
• John Rindy (LinkedIn)
• Emily McClaine (LinkedIn)
• Relationship-Rich Education: How Human Connections Drive Success in College