Understanding Variation
Deming's Profound Knowledge for School Leaders - Part Two
In this post:
Using the Ladder of Inference to avoid reactionary thinking
Common mistakes to avoid when addressing variation in results and outcomes.
Leadership Dispositions that will help school leaders navigate complexity and think critically about the variation in results and outcomes in their schools.
Essential Questions for school leaders and their teams that will deepen conversation around Understanding Variation.
Access to a comprehensive discussion tool that can be used to guide development with Systems Thinking and deeper understanding of Deming’s Profound Knowledge among teams of school leaders.
Decision Making and the Ladder of Inference
School leaders are faced with a wide range of decision making responsibilities in their roles. Decisions range from the simple to the complex on a daily basis. For example, on the same day (or even in the same hour), a school leader may need to decide if the color scheme for the new office furniture is acceptable, then determine if it is appropriate to move ahead with the removal of a student from the school for disciplinary reasons, and then approve a budget allocation for a professional development program that will require a year-long contractual commitment. When working at a high level, all these decisions and the scores of others the school leader will make over a given week must be made in the context of the school's organizational goals and mission.
As a school leader, decision-making depends on how one understands a given situation. This can feel self-evident based on experience and familiarity with similar situations experienced in the past. However, those impacted by a situation and any associated decisions that need to be made often focus on different aspects of the situation and interpret it in their own ways, leading to different conclusions about what should be done or what the correct decision is based on what they see. The Ladder of Inference is one way that Systems Thinkers can work with teams to make better decisions and communicate effectively. Better decisions are usually made when the team or group can intentionally stay in the lower rungs of the ladder before going all the way to the top and making hasty decisions.
Most decisions that leaders need to make, big and small, are essentially responses to variations in results, outcomes, human behavior, etc. Knowing the difference between special cause and common cause variation will help a leader stay on the lower rungs of the Ladder of Inference in the early stages of decision-making while preventing the damage often results in jumping to conclusions or making hasty decisions without understanding causes, seen and unseen, behind a situation that requires a decision. Understanding the differences between special cause and common cause variation helps the leader avoid making two crucial mistakes: reacting to common cause variation as if it were special cause variation and treating special cause variation as if it were common cause variation.
Knowledge about Variation
In schools, we constantly deal with variation—differences between students, classrooms, teaching methods, test scores, and overall school performance. Edwards Deming, a pioneer in systems thinking and quality management, emphasized the importance of understanding variation to improve outcomes. This insight is critical for school leaders when making decisions about instruction, assessment, and school improvement.
Deming explained that variation is inevitable. There will always be differences in output and results in student performance, program effectiveness, or teaching quality in schools. These differences are best understood as one of two primary types of variation: common causes and special causes.
Common Causes of Variation: These are the everyday, routine fluctuations within a system. In schools, this might be differences in students' backgrounds, varying levels of teacher experience, or even the resources available to classrooms. These variations are part of the normal functioning of the system. Other examples in a school setting might include:
Slight Variations in Standardized Test Scores Between Years. Over time, fluctuations in standardized test scores may be due to common causes such as the natural variation in students' prior knowledge, the annual variation in the rigor of test questions, or changes in student motivation. Reacting to these minor shifts as though they were special causes could lead to unnecessary overhauls of teaching methods or curricula.
Fluctuations in Class Participation. Daily variations in student participation levels can be attributed to common causes like mood, sleep, or external stressors (e.g., exam seasons). A school leader might expect this natural variation and would avoid over-interpreting slight decreases in engagement as the result of a problem with the teacher’s style or curriculum.
Minor Attendance Variations Over the Year. School-wide attendance might naturally ebb and flow based on factors like weather, common seasonal illnesses (such as the flu), or family vacations. These common causes don’t typically require intervention beyond routine policies and procedures.
Special Causes of Variation: These occur due to unusual, unpredictable factors that are not part of the system's regular functioning. In education, special causes might include unexpected events like a sudden change in curriculum, a specific student's family crisis, or the impact of a new teaching method introduced mid-year. Other examples in a school setting might include:
Sudden Spike in Student Anxiety Levels. If a school sees a sharp, unexpected increase in students reporting anxiety or seeking counseling services, it could be a reaction to a special cause, such as sudden policy changes, dramatic increases in homework loads, or external pressures, such as university admissions changes. This warrants deeper investigation and targeted intervention.
Dramatic Drop in Teacher Morale. If teacher morale suddenly declines, it may be due to a special cause, such as a recent leadership change, new policies that increase workload without support, or an external event, such as a public scandal or crisis in the school. This type of variation demands immediate attention to identify and mitigate the underlying issues.
Sudden Decline in Enrollment Applications. A school that typically maintains a steady stream of enrollment applications might experience a sudden drop due to a special cause, such as negative press coverage, a major policy shift in the admissions process, or increased competition from nearby schools. This requires a focused response to address the root cause.
Common Leadership Mistakes
Deming cautioned against two frequent mistakes that leaders commonly make when addressing variation in results and outcomes:
Mistake 1: Reacting to Common Cause Variation as if it were Special Cause Variation.
Mistake 2: Treating Special Cause Variation as if it were Common Cause Variation.
It is not uncommon to hear school leaders lament that their days are spent putting out fires as they address various issues across the school with parents, students, and faculty. Deming would say that the best way to put out fires is to prevent them in the first place. Constantly making the two mistakes noted above prevents a school leader from moving from fire fighting to fire prevention, which is an essential shift when looking to move from a management stance to a leadership stance in the organization. In The New Economics for Industry, Government, Education., Deming writes:
“Certainly we need to put out a fire, no matter what be the cause, but our aim should be to reduce the number of fires in the future. To go about the reduction of fires, treating every fire as it arose from a special cause, an accident, is totally different from regarding it as a product of a stable process. The supposition that every fire is an accident may well block the road to reduction in the number of fires.”
According to Deming, a system can either be in statistical control or out of control. When a process is in statistical control, its variation is predictable. In schools, this means knowing that test scores, attendance, or other performance concerns may fluctuate within a certain range due to common causes and that sudden outliers may indicate special causes. When leaders understand this distinction, they can make more informed decisions about what action is needed—if any.
Deming’s insights remind us that without understanding the nature of variation, efforts to improve school performance can lead to frustration and unintended consequences. School leaders who are able to distinguish between common and special causes of variation are better equipped to guide meaningful and sustainable improvement.
By seeing patterns in variation, leaders can focus on system-wide changes that improve outcomes for all students rather than reacting to isolated incidents or anomalies on a short-term, reactionary basis.
Understanding Variation - Leadership Dispositions
Interprets data with an understanding of variation: Recognizes that random fluctuations occur in performance and avoids making decisions based on normal variation.
Avoids overreacting to short-term changes: Resists the temptation to take immediate action based on single data points or small shifts, understanding the natural ebb and flow of system performance.
Focuses on improving systems, not individuals: Prioritizes systemic improvements rather than blaming or over-praising individuals for results driven by normal variation.
Recognizes the limits of intervention: Understands that frequent interventions based on short-term results can be counterproductive and may hinder progress.
Emphasizes long-term process improvement: Concentrates efforts on enhancing the overall system and processes rather than reacting to short-term performance spikes or dips.
Supports team productivity through system improvements: Rather than micromanaging, focuses on actions that make it easier for staff to succeed, such as improving tools, resources, and processes.
Exhibits calm leadership during fluctuations: Maintains composure during performance changes, knowing that the most important work happens in long-term preparation, not in momentary corrections.
Understanding Variation - Essential Questions
How can we develop awareness of the inevitable variation within our school system, and how should this influence our decision-making?
What is the importance of having a stable system, and how do we assess the capability of our school to achieve consistent results (that are intentional)?
How can we distinguish between special causes (individual, unique events) and common causes (systemic issues) of variation in our school’s performance?
What are the risks and costs of tampering with systems that are experiencing normal variation, and how can we avoid this costly mistake?
How can school leaders use data effectively while accounting for different sources of uncertainty and variation?
What is the difference between using data for enumerative studies (to understand the current system) versus using it for analytic problems (to make predictions about future performance)?
How can we ensure that our data-driven decisions are informed by a deep understanding of the system’s variation and not based on incomplete or misleading data?
What strategies can we employ to predict future outcomes more effectively based on the data we collect and analyze today?
(Stop) Jumping to Conclusions
Classic rock fans will remember Joe Walsh, a member of the Eagles and a solo artist on rock radio in the early 1980s. As leaders working to Understand Variation and make better decisions for schools, Walsh’s 1981 song called “A Life of Illusion” provides a fun, concise, and on-point reminder that variation is ever present in our lives and organizations and that jumping from one conclusion or assumption to another perpetuates a cycle of crisis and stress.
Deming tells us that we can do better. Understanding the differences between Common Cause and Special Cause Variation prevents us from jumping to conclusions and the poor decisions that follow.
Profound Knowledge
This post is the second in a series of four that will introduce the leadership principles of Deming’s Profound Knowledge as adapted for school leaders and school environments. In summary, Profound Knowledge has four parts: Appreciation for a System, Knowledge about Variation, Theory of Knowledge (Action), and Psychology.
Once learned, the concepts of Profound Knowledge position the leader to make decisions and lead with a constancy of purpose that focuses on continually improving teaching and learning in the school. Simply stated, the system of Profound Knowledge provides a foundation that helps school leaders understand HOW to lead transformation and build culture in their schools.
The GFL Profound Knowledge Discussion Guide
School leaders can access a PDF download of the Profound Knowledge Discussion Guide here or in the downloads section of the GFL Website.
- Dr. Steven Lyng / gofourthlearn.com
References:
Baker, Edward Martin . The Symphony of Profound Knowledge. iUniverse, 5 Dec. 2016.
Carder, Ph.D., Brooks , and Marilyn Monda, MA. “Deming’s Profound Knowledge and Leadership “We Are Still Not out of the Crisis” .” ASQ Human Development and Leadership Division, 2013.
Deming, W. Edwards. The New Economics for Industry, Government, Education. 3rd ed., Cambridge, Massachusetts MIT Press, 2018.
Moen , Ronald D. , and Clifford L. Norman . “Always Applicable - Deming’s System of Profound Knowledge Remains Relevant for Management and Quality Professionals Today.” Quality Progress, June 2016, pp. 47–53, www.qualityprogress.com. Accessed 10 Dec. 2023.
The Systems Thinker. “The Ladder of Inference - the Systems Thinker.” The Systems Thinker, 10 Jan. 2016, thesystemsthinker.com/the-ladder-of-inference/.