The following information is provided to assist districts in meeting the posting requirements stipulated in PA 173 Section 1249(3)(c). It is worth noting that MASB’s instrument is intended for use by school board members in the evaluation of superintendents. As such, effort has been invested to ensure that the language in the rubrics and the recommended process is easy for noneducators to understand and implement.

Research Base

National Policy Board for Educational Administration (2015). Professional Standards for Educational Leaders 2015. Reston, VA: Author.

The 2015 Standards are the result of an extensive process that took an in-depth look at the new education leadership landscape. It involved a thorough review of empirical research (see the Bibliography for a selection of supporting sources) and sought the input of researchers and more than 1,000 school and district leaders through surveys and focus groups to identify gaps among the 2008 Standards, the day-to-day work of education leaders and leadership demands of the future. The National Association of Elementary School Principals, National Association of Secondary School Principals and American Association of School Administrators were instrumental to this work. The public was also invited to comment on two drafts of the Standards, which contributed to the final product. The National Policy Board for Education Administration, a consortium of professional organizations committed to advancing school leadership (including those named above), has assumed leadership of the 2015 Standards in recognition of their significance to the profession and will be their steward going forward.

Midcontinent Research for Education and Learning (2006). School District Leadership That Works: The Effect of Superintendent Leadership on Student Achievement. Denver, CO: Author.

To determine the influence of district superintendents on student achievement and the characteristics of effective superintendents, McREL, a Denver-based education research organization, conducted a meta-analysis of research—a sophisticated research technique that combines data from separate studies into a single sample of research—on the influence of school district leaders on student performance. This study is the latest in a series of meta-analyses that McREL has conducted over the past several years to determine the characteristics of effective schools, leaders and teachers. This most recent meta-analysis examines findings from 27 studies conducted since 1970 that used rigorous, quantitative methods to study the influence of school district leaders on student achievement. Altogether, these studies involved 2,817 districts and the achievement scores of 3.4 million students, resulting in what McREL researchers believe to be the largest-ever quantitative examination of research on superintendents.

Authors

The Michigan Association of School Boards has served boards of education since its inception in 1949. In the decades since, MASB has worked hands-on with tens of thousands of school board members and superintendents throughout the state. Evaluation of the superintendent has been a key aspect of that work – MASB developed superintendent evaluation instruments and trained board members in their use nearly half a century before the requirements.

MASB staff and faculty involved in creating the MASB Superintendent Evaluation instrument Include:

• Rodney Green, Ph.D., Superintendent of Schools (retired), Consultant, MASB
• Olga Holden, Ph.D., Director of Leadership Services (retired), MASB
• Donna Oser, CAE, former Director of Executive Search and Leadership Development, MASB
• Debbie Stair, M.N.M.L., former school board member, Assistant Director of Leadership Development, MASB

New York Council of School Superintendents staff and leadership involved in creating the Council’s Superintendent Model Evaluation (which significantly influenced MASB’s instrument):

• Jacinda H. Conboy, Esq., New York State Council of School Superintendents
• Sharon L. Contreras, Ph.D., Superintendent of Schools, Syracuse City SD
• Chad C. Groff, Superintendent of Schools
• Robert J. Reidy, Executive Director, New York State Council of School Superintendents
• Maria C. Rice, Superintendent of Schools, New Paltz CSD
• Dawn A. Santiago-Marullo, Ed.D., Superintendent of Schools, Victor CSD
• Randall W. Squier, CAS, Superintendent of Schools, Coxsackie-Athens CSD
• Kathryn Wegman, Superintendent of Schools (retired), Marion CSD

Validity

Validity refers to how well an instrument measures what it is supposed to measure. Construct validity was established for the MASB Superintendent Evaluation instrument. Construct validity ensures the assessment is actually measuring superintendent performance. Validity was established using of a panel of experts familiar with the research base and work of the effective school superintendent. The experts examined the research, identified performance indicators for measure and refined the scale for measurement.

Panel members included:

• Rodney Green, Ph.D., Superintendent of Schools (retired), Consultant, MASB
• Olga Holden, Ph.D., Director of Leadership Services (retired), MASB
• Mary Kerwin, former school board member, former Senior Consultant, MASB
• Debbie Stair, M.N.M.L., former school board member, Assistant Director of Leadership Development, MASB

Efficacy

Efficacy refers to the capacity of the evaluation instrument to produce the desired or intended results. The MASB Superintendent Evaluation instrument has three intended outcomes:

1. To accurately assess the level of a superintendent’s job performance
2. To improve the superintendent’s professional practice and impact on student learning
3. To advance the goals of the school district

MASB will seek to establish efficacy of the MASB Superintendent Evaluation instrument by surveying school board members and superintendents from a representative sample of school districts (see details below). An electronic survey instrument will be used to ascertain the extent to which:

1. The district followed the prescribed process for conducting the evaluation, and
2. The evaluation instrument and prescribed process supported the stated outcomes  

Reliability

Reliability is the degree to which an evaluation instrument produces stable and consistent results. While there are several types of reliability, MASB will seek to establish the test-retest reliability of the MASB Superintendent Evaluation instrument. Test-retest reliability is a measure of reliability obtained by administering the same instrument twice over a period of time to a group of individuals. To accomplish this, a representative sample of school districts using the MASB Superintendent Evaluation instrument will participate in a reliability study. A minimum of 15 school districts (with low board member turnover and no transition in the superintendency) will conduct an evaluation at the midpoint of their evaluation cycle (T1) and again at the end of their evaluation (T2). Scores from the two assessments will then be correlated in order to evaluate the test for reliability. A coefficient of 7.0 or higher will indicate acceptable stability.

Evaluation Rubric

The complete MASB Superintendent Evaluation instrument is available in the following formats.

Evaluation Process

Planning: At the beginning of the year in which the evaluation is to occur, the Board of Education and superintendent convene a meeting in public and agree upon the following items:

• Evaluation instrument
• Evaluation timeline and key dates
• Performance goals (if necessary beyond performance indicators outlined in rubric, district-wide improvement goals and student growth model)
• Appropriate benchmarks and checkpoints (formal and informal) throughout year
• Artifacts to be used to evidence superintendent performance 
• Process for compiling the year-end evaluation
• Process and individual(s) responsible for conducting the evaluation conference with the superintendent
• Process and individual(s) responsible for establishing a performance improvement plan for the superintendent, if needed
• Process and individual(s) responsible for sharing the evaluation results with the community

Checkpoints: The Board of Education and superintendent meet at key points in the evaluation year as follows:

• Three months in – Informal update – Superintendent provides written update to the board. Board president shares with the superintendent any specific concerns/questions from the board.
• Six months in – Formal update – Superintendent provides update on progress along with available evidence prior to convening a meeting in public. Board president collects questions from the board and provides to superintendent prior to meeting. Board and superintendent discuss progress and make adjustments to course or goals, if needed.
• Nine months in – Informal update – Superintendent provides written update to the board. Board president shares with the superintendent any specific concerns/questions from the board.
• 11-12 months in – Formal evaluation – Superintendent conducts self-evaluation; presents portfolio with evidence to Board of Education (made available prior to meeting). Board members review portfolio prior to evaluation meeting; seek clarification as needed. Board president (or consultant) facilitates evaluation. Formal evaluation is adopted by Board of Education.

Evidence

Validity, reliability and efficacy of the MASB Superintendent Evaluation instrument relies upon board members using evidence to score superintendent performance.

• Artifacts to serve as evidence of superintendent performance should be identified at the beginning of the evaluation cycle and mutually agreed upon by the Board of Education and the superintendent.
• Artifacts should be limited to only what is needed to inform scoring superintendent performance. Excessive artifacts cloud the evaluation process and waste precious time and resources. 
• Boards of education and superintendents should establish when artifacts are to be provided, i.e., as they originate, at designated checkpoints, during self-evaluation, etc. 

A list of possible artifacts that may be used as evidence is provided at the end of each professional practice domain rubric. Appendix D of the evaluation instrument offers additional artifacts that may serve as evidence of performance.

Conducting the Formal Evaluation and Conference

Prior to meeting:

1. Superintendent prepares self-evaluation, compiles evidence and provides to Board of Education.
2. Board members seek clarity as needed regarding self-evaluation or evidence provided.
3. Board of Education members receive blank evaluation instrument and make individual notes about their observations.

During meeting:

4. Superintendent presents self-evaluation and evidence. Superintendent remains present throughout the meeting.  
5. Board president reviews with Board of Education superintendent’s self-evaluation and evidence provided for each domain and facilitates conversation about performance.
6. Score is assigned for each performance indicator via consensus of the Board of Education.
7. Upon completion of all performance indicators within all domains, board president calculates overall professional practice score and identifies the correlating rating.
8. Board president reviews with Board of Education evidence provided related to progress toward district-wide goals.
9. Score is assigned for progress toward district-wide goals via consensus of Board of Education.
10. Board president reviews with Board of Education evidence provided related to district’s student growth model.
11. Score is assigned for student growth via consensus of Board of Education.
12. Board president calculates overall evaluation score based on professional practice, progress toward district-wide improvement goals and student growth ratings.
13. Board president makes note of themes/trends identified by the Board of Education during the evaluation.
14. Board president calls for vote to adopt completed year-end evaluation for superintendent.
15. Superintendent notes his/her comments on evaluation.
16. Board president and superintendent sign completed evaluation form. 

After the meeting:

17. Completed evaluation form reflects Board of Education’s assessment of superintendent’s performance.
18. Board president works with superintendent to coordinate public statement about superintendent’s performance.

Contingencies:

If a superintendent is rated as minimally effective or ineffective, the Board of Education must develop and require the superintendent to implement an improvement plan to correct the deficiencies. The improvement plan must recommend professional development opportunities and other actions designed to improve the rating of the superintendent on his/her next annual evaluation.

If a superintendent is rated as highly effective on three consecutive annual evaluations, the Board of Education may choose to conduct an evaluation biennially instead of annually. However, if a superintendent is not rated as highly effective on one of these biennial evaluations, the superintendent must again be evaluated annually.

Developing an Individual Development Plan

Individual Development Plans are an excellent way of helping employees develop their skills. Boards of education should encourage superintendents to develop an IDP in order to foster professional development.

In the event that a superintendent receives a rating that is less than effective, the law requires the creation of an IDP. The following process is a framework for creating and implementing an IDP for the superintendent:

• During the evaluation conference, the Board of Education provides clear feedback to the superintendent in the domain(s) in which he/she received a less than effective rating.
• A committee of the Board of Education is established to support and monitor the superintendent’s development.  
• The superintendent drafts an IDP and presents it to the committee for feedback and approval. The IDP outlines clear growth objectives, as well as the training and development activities in which the superintendent will engage to accomplish objectives. The committee reviews, provides feedback and approves the IDP.
• The committee meets quarterly with the superintendent to monitor and discuss progress.
• The superintendent reports progress on his/her IDP with his/her self-evaluation prior to the formal annual evaluation.   

Training

MASB provides training on its Superintendent Evaluation instrument to board members and superintendents via a cadre of certified trainers. Training is as follows:

Fundamentals of Evaluation: This training covers the fundamentals of evaluation including legal requirements, essential elements of a performance evaluation system and processes for establishing superintendent performance goals and expectations. This session may not be necessary for participants who have attended Board Member Certification Courses (CBAs) 300 and 301, or who have documented participation in in-district workshops focused on superintendent evaluation conducted by MASB trainers. It is offered at various locations on an individual registration basis or as requested in cooperation with intermediate school districts.

Instrument-Specific Training: This training covers the use of the MASB Superintendent Evaluation instrument including the cycle and processes of evaluation, rating superintendent performance on the rubric, as well as the use of evidence to evaluate superintendent performance. This training fulfills the requirement of evaluator training for board members as well as evaluatee training for superintendents whose districts are evaluating their superintendent with the MASB Superintendent Evaluation instrument. It is conducted on-location in districts with board members and superintendent present.