Reviewing for Sensitivity: A Call to Action
The world has changed so rapidly over recent years it makes one’s head spin. Our K-12 schools are not immune from the politically charged topics and violence in surrounding communities, such that our educators and school leaders are having to contend with a variety of intense challenges. While bias in curriculum, banning of books, and critical race theory have garnered considerable recent media attention, these are only the tip of an iceberg of cultural shifts surrounding our schools.
One action school and/or district leaders should take now is to evaluate all locally-developed assessments for sensitivity considerations.
Locally-developed tests can go by many names, but are typically designed to be used for internal monitoring of academic progress throughout the school year. The content of these assessments may be completely teacher-developed or may also include some adapted test items from adopted curriculum packages. The subjects assessed on locally-developed tests may also go far beyond the typically required tested subjects of reading, writing, mathematics, and science.
It is imperative for schools and/or districts using locally-developed assessments to evaluate these tests for potential sensitivity concerns owing to the world changing around the tests. Similar sensitivity reviews are conducted in large-scale assessment programs because state educational agencies and assessment development companies have staff dedicated to maintaining test content and continually updating development processes, including bias, fairness, and sensitivity review criteria. But locally-developed assessments far too often are developed using limited (or temporary) funding and may not have dedicated assessment-focused staff maintaining the test content.
The World is Always Shifting Around our Tests
Even with review processes in place in large-scale assessment programs, sometimes problematic test content is left on tests as the world changes around them. During my days working at a state education agency, I received a report from a teacher that there was an objectionable item on a publicly available practice test. Immediately the issue was obvious. When the item assessing English proficiency had originally been written about 4 years prior, it had been appropriate to ask a student to tell a story based on illustrations of various familiar activities, such as planting a flower, watering a garden, or building a wall with bricks. However, by 2016 when I received the call, the idea of building a wall, particularly for students who were receiving English language development instruction because English was not their first language, was more than problematic, it was offensive.
The item was promptly removed from all future use and a potentially politically explosive situation was avoided because the teacher understood the item had been written years before and reported the issue directly to those who could make the change. In short, we got lucky! Schools and districts must evaluate their locally-developed assessments for new sensitivity considerations as soon as possible because we can’t rely on luck to uncover offensive test content without undue media attention or offending student sensibilities. Additionally, I firmly believe in responsible stewardship of any test content put in front of students. Regular Sensitivity Reviews are required maintenance to ensure students get the best, every year.
A Process to Conduct a Sensitivity Review
These four steps may be conducted in a very short period of time as a sensitivity review should be done quickly, efficiently, and with regularity. Since the world will not stop changing around us, a sensitivity review should be conducted as often as once every two years.
First, identify who will do the work. A Sensitivity Review produces the best outcomes when a purposeful selection of educators who have different backgrounds are included because the intention is to consider as many different points of view as possible when evaluating the test content. Selection of who will conduct this review, especially if being conducted at the district level, should also include some individuals who are working directly with students at the grade level(s) of the test(s) being evaluated. Students year-over-year are becoming more politically aware and more informed about current events at younger ages, so some sensitive topics may be falsely dismissed with “they are too young to understand” if there is no one currently working directly with the students of that age group included in the group of reviewers.
Second, establish the topics that are too sensitive to be included on the tests. Just as any skilled editor knows, when one reviews a document for everything, they catch nothing. The group of reviewers should brainstorm then agree upon the sensitive topics that are relevant to their local context and will be the focus of the intentional inspection. What is and is not a sensitive issue needs to be hashed out separate from the test content for more efficient and productive discussions later in this process. Additionally, owing to local context influencing what is and is not sensitive, there is no comprehensive list of current topics that will serve all communities. There may be some overlap nationwide, such as topics relating to the pandemic and recent global events, but assumptions at this step will bog down (if not totally derail) the rest of the review process.
Third, each reviewer conducts an independent evaluation of all test content. It is important for this first pass to be independent to allow each individual their own opinion of what they see as potentially problematic test content. Any and all concerns should be noted with as much detail needed to support later discussion and may be kept on record to further expedite future sensitivity reviews. Additionally, sampling test content will not do this process justice, every test and every item must be scrutinized.
Fourth, the group of reviewers discuss the findings of their independent evaluations. Any and all test content found to be potentially problematic needs to be openly discussed. Then, as a group, determine if the content may remain on the test unchanged or not. For all content determined to present consequential sensitivity concerns, then a recommendation to replace or edit should also be agreed upon by the group of reviewers. It is important to remember that not all sensitivity concerns are of equal magnitude and the recommendation should be derived from the nature of the concern. Sometimes a minor adjustment can salvage the item without compromising the construct being measured and remove the sensitivity concern.
Revisions and replacement of problematic test content begins a different process that may or may not include the same individuals who conduct the Sensitivity Review. If substantial alterations to the local assessments are necessary, this may initiate a much larger process of revamping, or developing and validating new tests.
If no problematic test content is found, it is still a success, and should be documented for future reviews and the outcomes shared with collective leadership upon completion. Sensitivity Reviews are a part of important maintenance that contributes to the quality and integrity of a local assessment program.
