Transparency & Accountability

Special Edition: The Book Review        

It seems like every time there is a leadership change in a Local Educational Agency (LEA) the profile of the desired next leader is designed by the community of stakeholders is to that the new leader is transparent with the community and holds everyone accountable.?This has become one of the major platforms that board members use to get elected to office.?Meeting the requirements of being transparent and accountable then requires reporting of data in formats that are understandable to all stakeholders.

LEAs and their individual schools are judged by numerous factors by external stakeholders: safety of the area, the appearance of the facilities, stability of the staffing, and academic performance. ?When marketing resale and new housing the realtor will highlight the local schools as a selling point.?The weight that is put on test results has become very high stakes in education.?In a new book Mismeasuring Schools’ Vital Signs: How to Avoid Misunderstanding, Misinterpreting, and Distorting Data released on September 29th authors Steve Rees and Jill Wynns discuss the complexities of gathering, reporting, and utilizing data.

Any educational leader, board member, or community advocate wishing for transparency and accountability for their LEA must read this book!?In the past decade-plus, I have watched how assessment data has become weaponized in the highly political constructs that have become LEA board meetings.?When data is not fully reviewed and validated or misrepresented faulty conclusions can quickly occur which has led to many tumultuous changes in LEA leadership that ultimately slow down the goal of reform as the organizations lack stability and consistency.

In 2000, the level of accountability was extremely high stakes in California with the relatively new Academic Performance Index (API) that rated every school on a scale of 200 – 1000 (800 target) and then went further to provide bonuses to educators for the biggest gain.??This was unfortunately not an appropriate method (the why is a longer story) because the scale system was weighted in such a way that small movements at the bottom of the scale had more value than sustaining or growing at the top.?At that time, I was working as the Director of Student Assessment for a large urban school district in Southern California and it was during that period that I met Steve Rees through School Wise Press who shared my passion for using data appropriately.

In Mismeasuring Schools’ Vital Signs: How to Avoid Misunderstanding, Misinterpreting, and Distorting Data the authors have focused on specific areas of data with deeper looks into the science of assessment pedagogy (Chapter 2); The key indicators in graduation data (Chapter 3); The critical foundational block of proficiency (Chapter 4); the conundrum of English language learners (Chapter 5); The economics and financing of education (Chapter 6); and a discussion of how to reduce barriers (Chapter 7).?Each chapter is an excellent stand-alone lesson that brought together provides a very transparent big picture.

Many who read the title of the book and the initial heading who think that this might be an anti-assessment focus, but nothing could be farther from the truth.?The opening part of the book explained it best when the authors stated, “We welcome a climate surrounding school accountability that is not so hot, not quite so polarized, and more open to discussion.”?When California announced that they were delaying the release of Smarter Balance scores, the authors we among those that decried the error in that decision.?Within the book, they help outline how essential it is to have dedicated individuals working in education that are skilled in data disaggregation and analysis.

The book begins by looking at how often data is looked at in the format of a snapshot in time.?Growth models of reporting are often misleading as they are not utilizing year over year data from common students and do not account for the initial starting point of entering students.?The explanations of the human cost of mismeasurement are personalized down to the student level in a way that the reader quickly determines and understands the danger in inadequate utilization or analysis of data.

The authors take the reader through the entire progression with an understanding that the goal is not to determine whether to test, but rather to investigate the how, who, when, and what of the assessment to be used.?When discussing the errors of omission, errors of judgment, and errors of commission, the authors quickly validated much of what I have been writing about regarding a focus on lost learning and the need for discriminator analysis to determine the foundational thinking of students to ensure that they are not misplaced due to an assessment.?The entire chapter 2 is an excellent introduction to test analysis mechanics written in a way that is digestible to those who are not psychometricians.

The goal of public education is for the student to graduate.?I have dedicated my entire focus in education to get students to graduate as shown in my origin story and utilization of the research in my work with SI&A.?The authors help illustrate factors such as student mobility, varying graduation requirements across LEAs and States, and a true look at the bias that is built into some adopted policies that harm students.?The author builds upon the graduation data chapter by moving back to the foundation and the variables of environment and starting position for students.?Imagine running a 440-yard dash race and your opponent getting to start from the second turn.?All that runner will ever see is the back of the person in front and soon be discouraged to stay in the race.?Labeling students incorrectly and not focusing on the foundation before finishing can create this scenario.

Chapter 5 focuses on English language learners and takes a deeper look at how these students are initially identified and then the service models moving forward.?Borrowing from Star Trek the authors use “Vulcan logic” to ask the question “aren’t all students initially English language learners??The chapter delves into academic English versus conversational English and the pitfalls of reclassification.?This aligns with the key strategies from AVID and the concept of how to introduce academic language to all students to ensure higher achievement.

One area that will be the focus of many more articles from me in future months looking at the disjoint between the educational and business services of an LEA.?Too often the two do not align in thinking or use of data, and often the data in the management information system and student information system do not dissect.?With districts facing enrollment loss and a misunderstanding of the importance of indirect support services that are lost when the political “only focus on the classroom” is misinterpreted the importance of position control and connectedness is explored.?The concept of people analytics to provide better support models for students is another area that good LEAs have embraced and one that all stakeholders should internalize.

The researcher John Hattie calls education an “immature profession.”?His meaning is that the educational system has not evolved in the past century in areas of analytics and proper uses of data which were requirements for survival in other private sector industries.?Being reliant on human resources, with over 90% of every dollar spent by LEAs going to people, there is a significant challenge in implementing consistent systems and interpretations of data that have contributed to the extreme divisiveness of many school board meetings in recent years.?The authors do a good job of trying to help the reader understand the need to shift from opinion to evidence.

I encourage anyone that is reading this article to go out and purchase Mismeasuring Schools’ Vital Signs: How to Avoid Misunderstanding, Misinterpreting, and Distorting Data and take the contents to begin discussions with your local entities.?After each chapter, the authors have included multiple talking points aimed at both LEA leadership and LEA board members and community members to get all stakeholders involved and utilize similar toolkits to embrace data.

?In 2004, I was deeply involved in utilizing assessment data to make a difference in student achievement.?I printed a slogan on numerous pens “What is the data telling us…Improving Student Achievement Together.” ??The first part of that saying is not a question, but rather a challenge to take time to truly look at, analyze, and synthesize the data to then collaborate with others to implement systems to support student learning.

Attendance is the Gateway to Student Achievement and the First Line of Defense Against Lost Learning.?We must first invest to ensure that student is present every day, which is still an area of need for most school districts, and then we must better utilize the tools we have to create the appropriate support for all students.?I was honored to be able to read the book before it was released and I hope that others who read it will come to understand much of how I have looked at data and join me in supporting public education to move forward with this generation of students who need our solidarity and cohesive commitment to help them grow.