Is a Reduction in State Assessments a Reality?

In June of 2015, I attended the National Conference on Student Assessment hosted by the Council of Chief State School Officers.  At that conference, I went to a session where two of my assessment heros were presenting.  Those heros are Rick Stiggins and James Popham.  I was happy to get a quick picture with both of them!!

Rick Stiggins

James Popham

At the time, they both proposed that there was an abundance of state-level summative assessments and that our students were being over tested.  Moreover, they asserted that state-level summative assessment programs were of little use to teachers.  In other words, it was very difficult for classroom teachers to use these types of assessments to improve instructional practice or provide intervention for students.  Oddly, a year before I heard John Hattie make the same assertion at the National Visible Learning Conference.  In fact, Dr. Hattie spoke about assessment reforms in New Zealand where state assessment was virtually eliminated as we know it in the United States.  Instead of a centralized, state summative assessment system where teachers have little/no control over item construction or test configuration; teachers in New Zealand select questions from an item bank, build their own assessments and the government uses those results to measure student achievement.

John Hattie

Personal discussions with Popham, Hattie, and Stiggins suggests that greater teacher control over state-level assessments,  certainly makes more sense.  First, placing teachers at the center of state-level assessment provides ownership over the whole process.  In fact, one could assert that teachers would view assessment as an integral part of the instructional cycle.  For example, teachers may be more inclined to use assessment to drive and modify their instruction, as is called for in formative assessment.  Finally, I would posit that placing teachers in control of state-level assessments would contribute to their agency and responsibility for increasing student achievement.  In other words, teachers would feel a greater sense of responsibility to the assessment process.

While the US Department of Education might view New Zealand's assessment system as radical, there is a movement to reduce testing for US students and teachers.  John King, US Secretary of Education recently unveiled a plan to reduce unnecessary and redundant assessments.  All the details of the plan can be found here.   There are seven steps to the plan that include the following:

• Worth taking: Assessments should be aligned with the content and skills a student is learning, require the same kind of complex work students do in an effective classroom and the real world, and provide timely, actionable feedback. Assessments that are low quality or redundant should be eliminated.
• 
  
• High quality: Assessments should measure student knowledge and skills against the full range of State-developed college- and career-ready standards in a way that elicits complex student demonstrations of knowledge, and provide an accurate measure of student achievement and growth.
  
• 
  
• Time-limited: States and districts must determine how to best balance instructional time and the need for high-quality assessments by considering whether each assessment serves a unique, essential role in ensuring all students are learning.
  
• 
  
• Fair – and supportive of fairness – in equity in educational opportunity: Assessments should provide fair measures of what all students, including students with disabilities and English learners, are learning. As one component of a robust assessment system, States should administer key assessments statewide to provide a clear picture of which schools and students may need targeted interventions and supports.
  
• 
  
• Fully transparent to students and parents: States and districts should ensure that students and parents have information on required assessments, including (1) the purpose; (2) the source of the requirement; (3) when the information about student performance is provided to parents and teachers; (4) how teachers, principals, and district officials will use student performance information; and (5) how parents can use that information to help their child.
  
• 
  
• Just one of multiple measures: No single assessment should ever be the sole factor in making an educational decision about a student, an educator, or a school.
  
• 
  
• Tied to improved learning: In a well-designed testing strategy, assessment outcomes should be used not only to identify what students know, but also to inform and guide additional teaching, supports, and interventions. 
• 
  

I believe this plan is a step in the right direction and predict that a movement will continue to grow where teachers have far greater influence in state-level summative assessment systems.

The Application of Selected Program Evaluation Standards on the Implementation of Web 2.0 Tools in Classrooms

As a superintendent who has spent nearly 25 years in public school education, it seems that K-12 public school administrators and teachers are on a constant mission to improve student outcomes. One of the latest efforts to increase student achievement is rooted in the implementation of instructional technology.  Many schools and districts are feverishly building infrastructure and adding devices.  As schools and districts race to implement iPads, Chromebooks, laptops, and WiFi to take advantage of Web 2.0 tools and personalized learning, there are program evaluation standards that may assist district and school stakeholders in making sound implementation decisions.

According to Yarbrough, Shulha, Hopson, and Caruthers (2011), there are Utility Standards developed by the Joint Committee on Standards for Educational Evaluation (JCSEE) that can be used to guide quality program evaluation.  Program evaluation is designed to assist stakeholders in determining if their organization is situated to take advantage of an opportunity to implement a new initiative that leads to improved outcomes.

Although there are eight Utility Standards, I will focus on U2 Attention to Stakeholders.  Again, the full list of Utility Standards can be found here.  

U2 Attention to Stakeholders 

According to Yarbrough, Shulha, Hopson, and Caruthers (2011), stakeholders are individuals who have a connection to a program or are served by a program. Evaluators can gain valuable insight from internal and external stakeholders with respect to program implementation.  For example, a principal might be excited about implementing personalized learning for all students in his/her school.  However, teachers may be hesitant about personalized learning.  Parents and Board of Education members will also take a stance on personalized learning.  Finally, students will also have a perspective on personalized learning.  In fact, Headden (2013), wrote that shrinking school budgets has served as a catalyst for school systems to explore technology as a means of offering individualized instruction.  On the other hand, Headden (2013), also wrote that a criticism of personalized learning is that is lacks the human element of teachers.  As one can imagine, each stakeholder group from above will likely support one statement or the other with respect to personalized learning.

Attending to the needs of all stakeholders can lead evaluators to a variety of perspectives that can either contribute to, or detract from a successful program implementation (Yarbrough, Shulha, Hopson, and Caruthers, 2011).  Rather than spending millions of dollars on another failed attempt at improved student outcomes, skilled evaluators (using the utility standards) can help organizations make collaborative decisions that can lead to greater success in implementing new initiatives.

So, as one attempts blast off with implementing instructional technology, the lesson here is to engage all stakeholders in meaningful dialogue and allow people to express their fears and opposing opinions. Often times, there is much to be learned from the sharpest critics.  I recently heard the following quote at a CUE Rockstar event that summed it up perfectly:

The cave you fear to enter  holds the treasure you seek ~Joseph Campbell

Reference

HEADDEN, S. (2013). The Promise of Personalized Learning. Education Next, 13(4), 14-20.

Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Thousand Oaks, CA: Sage.

Social and Ethical Responsibilities in AEA

Brief Biography

Michael R. McCormick is proud to serve as the Superintendent for the Val Verde Unified School District. Most recently, he served for six years as the Assistant Superintendent for Education Services. He has been an administrator for 19 years and an educator for 25 years. Michael “is a dedicated educator who truly cares about children…he is always working on ways to improve the community and motivate students.” Mr. McCormick was named Administrator of the Year by the Riverside County Office of Education in 2012 and by the Association of California School Administrators (ACSA) in 2015.

Under his leadership, VVUSD has been recognized by Education Trust West as, “tops among California districts Closing the Achievement Gap.”, named College Board AP District of the Year in 2012 and was featured in the National Center on Educational Outcomes publication Moving your Numbers.

Mr. McCormick participated as a member of the Riverside County contingent of educators summoned by the White House to a meeting at the U.S. Department of Education. The work completed there was highlighted in President Obama’s speech to attendees at the White House College Opportunity Day of Action held on December 4, 2014, in the White House Executive Office of the President “College Opportunity Commitment Report”, and in a press release from the White House Press Secretary.

Mr. McCormick traveled to China as part of the College Board 2014 Chinese Bridge Delegation. Michael was selected as ambassador, speaking for the group, exchanged ceremonial gifts on behalf of the delegation, met with Ministers of Education, and gave television and print interviews for local Chinese media.

Michael is a well-respected and sought after speaker, giving presentations both statewide and nationally. He has presented original research the last three years at the California Educational Research Association (CERA) conference. He has also presented for the College Board, the California School Boards Association, the National School Boards Association, the Curriculum & Instruction Steering Committee of the California County Superintendents Educational Services Association (CISC) and the California Educational Technology Professionals Association.  Mr. McCormick is proud to have delivered multiple keynote presentations at Google Symposiums focusing on the successful implementation of instructional technology in a K-12 school district.

Michael is passionate about student assessment and educational technology.  His educational career has focused on serving students by creating systems that ensure access and opportunity to high-quality curriculum, instruction, and assessment.  As a matter of fact, his guiding principle is that ALL students are provided an equal opportunity to graduate with the greatest number of post-secondary choices from the widest array of options. 

Social and Ethical Responsibilities in Assessment, Evaluation, and Accountability

The American Evaluation Association (AEA) published a public statement entitled Cultural Competence in Evaluation (2011) to bring awareness to the value of cultural competence in evaluation.  According to the American Evaluation Association, cultural competence is a process of "learning, unlearning, and relearning" (2011, p.3).  Moreover, that cultural competence requires researchers to have a very high degree of self awareness, an awareness of their own cultural assumptions, an awareness of the assumptions of others, and the ability to interact with others in a respectful and genuine manner (American Evaluation Association, 2011).

Valuing the position of the American Evaluation Association, with respect to cultural competence in evaluation, prompted questions about the importance of cultural competence in item construction for student assessments.  I began to wonder about the processes assessment companies use to create assessments that accurately measure student knowledge.  I chose to focus on the Smarter Balanced Assessment Consortium since they have been contracted to develop statewide summative assessments for all students in grades 3-8 and 11 in the state of California. Thus, I generated the following questions to explore:

1. How does the Smarter Balanced Assessment Consortium ensure assessment items are designed in a culturally competent way?
2. Are students, as evaluation participants, made aware of their rights and potential risks of participation by taking annual summative assessments to evaluate their academic progress?

Through my research, I discovered that the Smarter Balanced Assessment Consortium does make every effort to construct assessment items that are culturally competent.  I was very pleased to discover website content, a slide presentation, and a video detailing the processes of creating items that are free from bias, universally accessible, and sensitive.  Based on a review of the information presented in the link above, I was satisfied that the Smarter Balanced Assessment Consortium is creating items that meet the expectation of cultural competence prescribed by the AEA.

Please allow me to share one specific example of cultural competence used by the Smarter Balanced Assessment Consortium in item construction.  There is an emphasis placed on the use of language when creating assessment items.  According to the American Evaluation Association (2011), language is very powerful as it represents the code people use within their culture.  I believe there is a social and ethical responsibility to create assessment items that are free from language bias.  In my professional experience, paying attention to language bias in assessment items ensures, to the greatest extent possible, that student content knowledge is accurately assessed.  On the other hand, assessment items that include language bias, can result in lower student scores because of the student's lack of knowledge of a culturally biased word or phrase.  In this case, the student may know how to solve the problem, but be distracted from the correct answer because of faulty item construction.

The answer to the second question is more alarming.  According the the American Evaluation Association (2011), cultural competence in evaluation includes an ethical obligation to guarantee that all participants understand their rights and potential risks of participating in the evaluation.  With respect to statewide public school assessment, I would posit that students are not made fully aware of their rights.  More importantly, are students made aware of the risks of participation?  In fact, state law requires every 3-8 and 11th grade student to participate.  It should be noted that there is a waiver process that is also part of California law that allows parents to remove their child/ren from participation in the statewide assessment program.

The second question certainly causes me to think about social and ethical responsibility of ensuring third grade students understand the potential risks of participating in statewide summative assessments on an annual basis to measure their academic growth.  It is no secret that assessment results will be used to judge their academic performance.  Moreover, those results may be used to determine class placement and program placement.

Cultural competence among assessment developers is essential.  The expectation from stakeholders is that student assessment results are valid.  Validity is critical because of the way that educators use student results.  In my opinion, public school educators and assessment developers must pay attention to cultural competence.  According the the American Evaluation Association (2011), evaluators are challenged to increase their self-awareness and seek opportunities to improve systems with respect to awareness and sensitivity to all cultures.  While there is evidence to suggest we have addressed cultural competence in the development of assessment items, I believe public school educators must pay greater attention to ensuring students understand the "risks" of participating in statewide summative assessments.

References

American Evaluation Association. (2011). Public statement on cultural competence in evaluation. Fairhaven, MA: Author.  Retrieved from http://www.eval.org

Smarterbalanced.org,. (2015). Item writing and review. Smarter Balanced Assessment Consortium. Retrieved 6 December 2015, from http://www.smarterbalanced.org/smarter-balanced-assessments/item-writing-and-review/