summit photo

Creating a Truly Data-Informed Institutional Culture

Civitas Learning Change Management Civitas Learning Space Data Science Trends

Share this Post

When Alice asked, “Would you tell me, please, which way I ought to go from here?” the Cheshire Cat replied “That depends a good deal on where you want to get to.” It is not just in Wonderland that this holds true. Much of the current literature about higher education speaks to the requirement for building a “data-informed culture”. Examples of institutions recognized as truly having a data-informed culture are often highlighted in the media and publications that cover higher education and often include St. Petersburg College, University of South Florida, Rio Salado, ASU and more. Countless conference sessions and consultants are dedicated to helping institutions move toward this aspirational organizational model. The benefits of being a data-informed organization are clear: Sound decision-making, increased efficiency of operations, ability to focus resources on the right student-success initiatives – all powered by a deep contextual understanding of what the data are saying.
The Measures of a Data-Informed Institution
What is the measure of a truly data-informed institution? When asked this question by my colleagues in higher education administration, I frequently respond by asking them how often they hear the following in their meetings: “what do the data tell us?” The measure of a truly data-informed institution is it has grounding discussions in trusted and well-understood data.Granted, there is value in bringing in the personal anecdote and individual student stories such as, “I was just talking to a freshman and they told me our orientation process is lacking.” This is a clear marker of something worth investigating for improvement. But moving from anecdote to action without data is ill informed. Does the orientation have a measurable impact on freshman persistence? The data can address whether or not the freshman orientation is achieving its goals; the students can address the more subjective question of whether they enjoyed it – and that is best assessed via deeper data collection. Asking the question “what do the data tell us” is a powerful signal that your institution is becoming data-informed. Yet, getting to the point where all meetings start with the data question will require both technical and cultural work. The predicate conditions here are around data that are trusted, that are relevant, and that are insightful.
Data-Driven vs. Data-Informed
Words matter. How we describe a goal can have a big impact on how we approach that goal. The frequently referenced “data-driven decision making” (DDDM) is falling out of favor for a more nuanced concept of “data-informed decision making” (DIDM). The seemingly small difference is surprisingly significant when put into practice. DDDM carries an implication that the data are conclusive and absolute, and therefore that the conclusions drawn and actions to be taken are almost a foregone conclusion. The concept removes the agency of people from the equation. On the other hand, DIDM allows for nuance, context, discussion, and multiple interpretations. This is the on-ground reality – data will generally provide us with correlation not causation. People will parse, process, and pow-wow over those correlations to move to toward causation, and therefore ground a plan of action. See: Forbes: Be Data-Informed, Not Data-Driven, For Now and Andrew Chen: Data-informed Versus Data-driven
Being able to rapidly access the data that meet those criteria can itself be a significant undertaking. As we frequently encounter, many institutions’ data are, let’s say, “sub-optimal.” Many institutions have ‘messy’ data. But it’s important to note, what we now know with certainty is data sources do not improve until they are put into use. There are many assertions about what constitutes a data-informed culture. In one of the seminal business works on data analytics, Davenport and Harris (2007) list five key attributes of a mature, data-informed organization:
  • The right data
  • The right amount of enterprise/integration/communication
  • The right leadership
  • The right targets for analytics
  • The right analysts
In a similar vein, an EDUCAUSE survey, (2012), crafted a metric for analytics maturity specific to higher education based on respondent themes. Notably, their list of 12 themes reflects the above list of attributes:
  • Accessibility of data
  • Centralization/integration of data
  • Quality of data
  • Data capacity
  • Number and expertise of analytics professionals
  • Communication between IR and IT
  • Data-based decision making as a part of the overall culture
  • Analytics as part of the strategic plan
  • Documented “wins” using analytics
  • Budgeting for analytics
  • The availability of appropriate tools/software
  • The existence of policies regarding data security and privileges
Baer and Norris (2013), posit a three-level maturity model, with the top-level key indicator being “Strong, committed leadership makes analytics a strategic imperative for the institution.” The top level exhibits these attributes:
  • An emerging culture of performance measurement and improvement
  • Mature predictive analytics and LRM
  • Over time, link to new developments in learning analytics, big data, and workforce analytics
  • Pervasive analytics for everyone—elevates job descriptions
  • Leadership focuses on optimizing student success and institutional effectiveness using analytics
SAS, as reported in the MIT Sloan Management Review, (2014) has also created a what-is-by-now-recognizable framework to assess analytics maturity, keying on elements of Leadership, People, Culture, Infrastructure, and Process. I have seen several institutions add the key metric of “Transparency” to their maturity schemas. ECAR Index whiteboard
Analytics Maturity Assessment
In our practice at Civitas Learning, we have created an amalgam of many of these items in an Analytics Maturity Assessment rubric consisting of five key subject areas:
  • Leadership
  • Organization and culture
  • Governance, policy, and processes
  • Funding
  • Data, technology, and tools
This Analytics Maturity Assessment helps an institution understand where their areas of strength lie, and where they can benefit most from focusing energy and resources toward the goal of improving as a truly data-informed institution. Becoming a more data-informed institution is a worthy goal. But it is one that begins by understanding your starting point, regardless of the framework you use. It is important and imperative work, for as the Queen said, “Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!”
References
Davenport, Thomas H.; Harris, Jeanne G. (2007). Competing on Analytics: The New Science of Winning. Harvard Business School Press. Bichsel, Jacueline, Analytics in Higher Education: Benefits, Barriers, Progess, and Recommendations (Research Report). EDUCAUSE Center for Applied Research, August 2012, available from http://www.educause.edu/ecar Baer, Linda; Norris, Donald, Building Organizational Capacity for Analytics , (EDUCAUSE Report). February 2013, available from https://net.educause.edu/ir/library/pdf/pub9012.pdf Kiron, David; Prentice, Pamela K; Ferguson, Renee B., The Analytics Mandate (Research Report). MIT Sloan Management Review, May 2014. Available from https://sloanreview.mit.edu/analytics-mandate

Top Banner Photo: Summit – Civitas Learning 2015

Dr. Rob Robinson

Rob is the Senior Director of Strategic Services at Civitas Learning where he works with colleges and universities to help them leverage data analytics to further their mission of student success. Prior to coming to Civitas Learning, Rob was a Principal Strategist at Blackboard, an Assistant Vice Provost for Educational Technology at the University of Texas at San Antonio and a key leader with the University of Texas TeleCampus, a large-scale, System-wide organization supporting fully online degree programs, many of which were inter-institutional collaborative programs.