BACKGROUND AND OBJECTIVE
Our client is a technology company whose flagship product is its Change Experience Platform (CxP) which assists corporates with the design and implementation of organizational change initiatives a large scale. The CxP provides both behaviour-based and survey-based assessments of how a particular organizational change initiative is performing with regard to behaviours that needed to be changed to achieve the initiative’s end goals.
The client developed a survey-based assessment framework that is organized around 4 dimensions of successful change management, each measured by a battery of scale questions. The client wanted Phronesis-Partners to validate the measurement scales for each of the dimensions and develop a complete long-form questionnaire with additional supporting questions that can potentially impact change success. Besides this, the data from an actual survey exercise was to be analysed to validate the change assessment model and provide actionable insights.
HOW WE HELPED
- EXPERT VALIDATION OF MEASUREMENT SCALES:
Phronesis-Partners’ internal survey and Organizational Behaviour experts undertook a vetting of the 4-dimensional change assessment scale. Our internal experts evaluated the questionnaire items on the basis of “clarity”, “relevance” to the defined dimension, and “completeness”. Wherever required, we suggested changes in phrasing to either ensure clarity or to better capture a particular aspect of the relevant dimension that the scale items were expected to measure. We also looked at whether the items designed to measure a particular dimension were conceptually distinct enough to measure different aspects of the dimensions. Our expert team also suggested the addition of some scale items.
- QUESTIONNAIRE DEVELOPMENT:
The project team discussed the expert team’s suggestions with the client and finalized the items to be used for each measurement. The client team also added some additional items. Our project team then undertook a review of recent academic and practitioner literature on Organizational Change Management and designed a structured questionnaire with additional items that might impact employee responses to organizational change.
- DESCRIPTIVE ANALYTICS:
Our client then used to questionnaire to conduct an employee survey in an organization for which it was implementing a Change Initiative aimed at achieving higher levels of Equity, Inclusion, and Diversity within the organization. We used the data collected by the client to a) validate the model and b) devise actionable insights. Descriptive analysis was conducted using frequency analysis, contingency tables, statistical descriptive measures, and ANOVA mean comparisons to get an understanding of the data and basic group-wise differences between different employee groups defined by responses to some of the psychometric and behavioural profiling variables. Correlation analysis was used to understand the relationships between different measures at an elementary level.
- MODEL VALIDATION AND PREDICTIVE ANALYTICS:
The Phronesis Team used Structural Equation Modelling with Partial Least Squares (PLS-SEM) to validate the model and test relationships between different constructs including the 4 dimensions of successful change assessment. PLS-SEM is a distribution-free technique that can work with small sample sizes and is especially suited to the study of inter-relationships between dimensions measured using a set of scale items where predictive validity and actionable insights are required. The SmartPLS-3 tool was used for the analysis. Our project team developed a set of hypotheses involving direct and interaction effects on the final endogenous construct that stood as a proxy for successful change. Usual checks on construct validity, effect sizes, and multicollinearity (for formative constructs) were carried out. The various coefficients of the structural model were assessed using bootstrap sampling (5000 bootstrap samples) and significance tests. Blindfolding was used to assess the predictive relevance of the model. Most of the most important hypothesized relationships were supported by our model.
Explained variance of the endogenous variables and predictive relevance were found to be satisfactory.
SEM Model (Adapted)*
- PRESCRIPTIVE ANALYTICS:
The results of the structural model were used to construct an Importance-Performance map identifying areas where the final client needs to improve. The improvement areas were prioritized in terms of their derived importance.
HOW THE CLIENT GAINED FROM THE ENGAGEMENT:
The client was able to fully develop and rigorously validate the survey measures that would be used as part of their change management platform. The end client for whom the change initiative was implemented could use the results of the analysis to act on the priority improvement areas that are likely to have the greatest impact on the desired endpoints until the next assessment.
CLIENT TESTIMONIAL:
“The collaborative approach and deliverables from Phronesis were excellent. I felt a strong partnership as we worked through the process, and I appreciated the willingness to provide expert research and counsel while always hearing and addressing my ideas and concerns. The statistical analysis went beyond my expectations and has set the stage for Rali to provide something truly ground breaking for our customers and partners.”
– Larry Mohl, Founder and Chief Transformation Officer
Larry Mohl is an experienced innovator and change management executive. A self-described “intrapreneur”, Larry is an expert at driving change from within, disrupting and re-shaping deeply-rooted company cultures. In Chief Learning Officer positions at Motorola Cellular and later American Express and Children’s Healthcare of Atlanta, Larry saw an opportunity to create a new learning and change initiative solution that would be different from anything he had seen on the market. The proof has been undeniable. There’s nothing that Larry enjoys more than hearing from customers and partners about how Rali is contributing to their success.
PHRONESIS TEAM:
Yusof has more than 15 years of experience in the Market Research Industry, more specifically, in the design of MR studies, analysis of data and derivation of actionable insights from the same. Yusof has deep experience of Market Research Analytics and is Well versed in Quantitative Techniques including multivariate analysis of MR data – Cluster (segmentation) analysis, MaxDiff, Choice-Based Conjoint, CHAID, Structural Equation Modelling (PLS and CB), derived importance analysis etc. He holds an MA in Economics from Delhi University and has also been a Teaching and Research Assistant at The National University of Singapore.
