Transformation is a journey, consisting of changes, that allows companies to stay competitive and continue to grow. These changes are created through human behavior, which is often difficult to measure. How do you measure culture? Or motivation? Even though change measurement can seem ambiguous, this shouldn’t stop change management practitioners from continuing to use data to define change results. Propeller consultant, Kyle Crawford, and Propeller alumna, Rachel Crocker, through years of experience partnering with large companies going through transformation, believe that a focus on the right data will measure the success of a change more effectively. They recently spoke about the best way to marry data and change efforts at an event that Propeller co-hosted with change management professionals in the San Francisco Bay Area. We interviewed Rachel and Kyle to get more insight into how data should be better utilized in managing change.
Business is being redefined by data. Why do you think change management hasn’t caught up?
Rachel: I think one of the inherent challenges with measuring change management efforts in general is that it tends to be an embedded, enabling function within larger project initiatives. The work change practitioners do tends to be in the execution state and tactics tend to be linked to the flow of the overall project. Even when we talk about overall adoption or utilization, we typically speak in terms of overall prevalence instead of speed to adopt; a critical measure in the effectiveness of our application of change principles. One of the best ways to add value in this space is by identifying key leading indicators of success and decoupling and measuring efforts to understand the result of our actions.
Kyle: While we are definitely in the middle of the data revolution, I don’t think it’s just change management that hasn’t caught up. Everyone is struggling to understand data and how to compete in this space. The earliest data adopters are those who created business models centered on data. Examples include Google, Netflix, and Airbnb. The next adopters are those that generate enormous amounts of data and have realized that they can gain a competitive advantage if they use that data effectively. The last groups to undergo the data revolution are those whose work has traditionally been more qualitative, human, and personal - change management is a perfect example of this. There often just isn’t much data out there to analyze and these groups are going to have to collect data before they can benefit. Change managers are going to have to work through both the generation and analysis of data before they realize the benefit of the data revolution.
Data can sometimes feel overwhelming. What is the best way to align everyone on what data to focus on?
Rachel: Alignment can be a challenging outcome to achieve in any situation. Typically, departments, internal organizations, and even business units have different key performance indicators (KPIs) that are tracked. Within this, there may be different data sources, varying collection and storage methods, and variation on management and quality. To overcome the alignment challenge, change leaders must break down barriers and access to the source of the data, create a common language and dialogue around measuring success, and establish transparency.
Kyle: Building off what Rachel said, there is clearly an issue with competing data and that’s partly because the quantity of data available is overwhelming. A few year ago, it was estimated that over 4 zettabytes of data is stored worldwide. That’s 4+ trillion gigabytes! Your smart phone has less than 100 gigabytes of storage. That number is expected to increase tenfold over the next decade. To use the right data, a leader should ask questions in their domain or area of focus and then narrow down the relevant data from there. As an example, agreeing that an increase in productivity is the key objective of an implementation can better align which data will indicate whether productivity is being increased. This requires both domain-specific knowledge from the experts within an area and data know-how from the data people within an organization. Partnership between these two groups is necessary to make progress.
What are you most excited about measuring within the change management space?
Rachel: I’m most excited about not just measuring the success of the implementation but also the achievement of the overall outcome or intent that precipitated the need for the change in the first place. Change management operates at the tactical level, executing plans and assessments to ensure the change is a success. Practitioners do a great job at finding ways measure user adoption, sentiment and utilization but there is an opportunity to link the result of the project to the intended outcome of the change itself. Change leaders will need to flex and challenge themselves to be more data savvy and to understand how to measure and categorize leading indicators that point towards outcome achievement.
Kyle: The fact that change management has traditionally had a qualitative focus is what makes data in this area so interesting. When we’re talking about hard numbers like revenue, click through rate, or social media posting trends, the science is relatively straightforward and clean with regard to how to analyze, visualize, and report. Things get tricky when we start trying to quantify decidedly qualitative things like how a resistant a group is to change or whether we are getting the right level of engagement to ensure success. These areas require more nuance and let us engage in a newer area of study, like survey design and natural language processing, to try to understand what is happening.
Tell us what is meant by stating that change adoption isn’t always the best measure of success.
Kyle: Adoption is a great way to determine whether we have finished rolling out a new process or tool but focusing solely on adoption misses the bigger picture focus. Did we achieve the outcome that we were pursuing from the change we implemented? The key to understanding success is relating it to an overall outcome. Why did we engage in a specific project in the first place? Outcomes, and whether we achieved them, is where energy should be focused to ensure that project sponsors are getting the return they expect. Examples of outcomes include revenue increases, cost savings, or efficiency gains. Adoption, in and of itself, isn’t an outcome. It’s just an interim measure along the change journey.
Rachel: I think this unique data and outcome focused perspective opens an opportunity for the field of change management to help leaders and organizations understand how we think about success, what exactly makes a change implementation successful and, how we can continue to be agile and smart about data to demonstrate project success.
Kyle Crawford knows that the best way to lead is by example. His motivation for doing great work not only propels him forward, but also inspires those around him. As an intelligence officer with the United States Marine Corps, Kyle led a platoon of 35 Marines and served as an executive officer for a company of 200 officers. He’s also a data junkie, constantly gathering and analyzing the best intel to make sound decisions and take calculated risks when the odds are in his favor. He holds bachelors degrees from Rensselaer Polytechnic Institute.
Hands-on and fearlessly curious, Propeller alumna Rachel Crocker values engagement above all else. Her love of storytelling (perhaps inspired by her affinity for TED Talks) is evident in her business communications, and she’s able to bring even the most formidable concepts down to earth. From anticipating operational snafus to experimenting at home in the kitchen, Rachel is living a life that constantly keeps her challenged and stimulated. She holds an MBA from Marylhurst University and a bachelor’s degree from Oregon State University.
