THE MISSISSIPPI LEGISLATURE

The Joint Committee on

Performance Evaluation and Expenditure Review

Report # 539

Opportunities for Improving the Accountability of the Mississippi Department of Education

Executive Summary

Introduction

The Mississippi State Board of Education requested a third-party review to seek opportunities on how to hold the State Department of Education (MDE) accountable to the board. The board’s Chair noted that the board needs assistance in helping to improve the accountability of MDE under the new State Superintendent, who began serving in his position in January 2010.

In performing this review, PEER sought to answer the following questions:

• What factors affect the board’s ability to hold MDE accountable for its use of resources?
• How could MDE’s accountability be improved?

Background

Mississippi commits more of its general fund budget to K-12 public education than to any other budget category. The Mississippi Department of Education has not been exempt from the effects of the budget crisis, however. The 2011 Budget Report shows a decrease of ten percent in funding to public education for FY 2011.

A nine-member State Board of Education oversees the Mississippi Department of Education, sets public education policy, and is responsible for appointing the State Superintendent of Education.

MDE receives revenues from federal grants, state general funds, and other funds. From FY 2007 to FY 2009, funding for general education programs decreased by 7%, primarily as a result of a decrease in federal funding.

Although recent improvement has been reported, results on national assessments and on the American College Test indicate that Mississippi students lag behind the rest of the nation. Also, many students are not demonstrating proficiency on the most recent, more rigorous state assessments.

Conclusions

What factors affect the Board of Education’s ability to hold the Department of Education accountable for its use of resources?

The Board of Education’s ability to hold the department accountable is affected by federal and state mandates, how accountability tools are used, and changes in educational standards and programs.

Federal and State Mandates

Federal and state mandates reflect the complex nature of the educational system and the difficult task the board has in holding MDE accountable.

Broad, vague, and sometimes immeasurable mandates in state law regarding public education affect the board’s ability to hold MDE accountable. MDE shares responsibility with many other parties for fulfilling the education mandates in state law. These other parties include the federal government, the state board and state superintendent of education, other state-level boards, councils, task forces, local school boards, communities, parents, and children. Disjointed accountability mandates account for a significant amount of MDE staff time and ultimately could impede the ability of MDE to focus on the effectiveness of MDE programs.

How Accountability Tools are Used

By adopting a performance management approach, MDE would greatly increase its ability to align its resources and operations to desired results. Also, the board could increase its ability to make sound, results-oriented policy decisions. In adopting this approach, the board and the department should utilize tools already available (e. g., strategic planning).

In the context of performance management, the department has certain tools available to it that are essential for the board to hold the department accountable. These are listed below, along with PEER’s conclusions regarding each.

• The Office of Educational Accountability—The Office of Educational Accountability currently focuses more on financial auditing than on program accountability.
• Strategic planning and departmental performance measures—By incorporating performance measures, time frames for achievement, programmatic responsibility, and associated resource allocations (both financial and staffing) into MDE’s strategic plan, the board would greatly increase its ability to hold the department accountable for the efficient and effective utilization of its resources. It would further help the board for the department to differentiate in its strategic plan between those activities for which outcomes are under its direct control versus those activities that require implementation by external entities, such as school districts, to achieve success.
• Resource allocation and organizational structure— Because the Department of Education’s budget is not linked to its strategic plan, the board is unable to make fully informed decisions regarding the commitment of departmental resources. Also, periodic assessments of the organizational structure and staffing patterns would allow the board and the department to use these as a management tool to achieve long-term goals.
• Measurement and reporting—MDE’s current departmental performance measures are not appropriate to establish accountability because they focus solely on measuring the activities performed rather than measuring the results achieved. Performance measures should ultimately reflect accountability of a person, group, or organization for the results of an activity and should provide managers with information on how resources and efforts should be allocated to increase effectiveness.
• Data systems and research—Although the Department of Education collects and reports a significant amount of data, the department is not using this data to its fullest extent. Because Mississippi’s existing education data systems (i. e., K-12, post-secondary, workforce) were originally designed to meet annual reporting requirements, not to track progress over time, these systems have not been linked. Thus presently it is difficult, if not impossible, for the board or the department to examine student progress and outcomes over time and to determine whether students are prepared for college and for the workforce.
  
  Also, the board does not have a visual instrument that synthesizes the data collected by MDE and that presents the information contained in this data in a form that is accessible to non-data experts. Thus the board is not able to evaluate fully the effectiveness of the programs and offices at MDE or to determine the status of progress toward the educational goals that it has set for MDE.

Changes in Educational Standards and Programs

The effectiveness of MDE’s programs and activities is difficult to gauge over time when educational standards and ways of thinking are in an almost constant state of change and when funds for specific programs are eliminated or reduced before effectiveness can be demonstrated.

The state has experienced significant changes in recent years regarding its educational standards, assessments, and accountability model. Some of these are listed below, along with PEER’s conclusions regarding each.

• Changes in curriculum and assessments—In 2006-07, MDE increased the rigor of curriculum and assessment standards in response to poor performance on the national assessment and on the ACT. In June 2010, the board adopted a “national common core” curriculum that will require Mississippi to revise its curriculum again. State assessments are likely to change also.
• Changes in the state accountability model—In 2007, state law mandated inclusion of graduation and dropout rates in the state accountability model. Beginning in the 2008-2009 school year, MDE changed its state accountability model for the fourth time since 1988 in response to the more rigorous curriculum and assessment system implemented in 2007-08. In 2005, all fifty states agreed to begin implementing a standard method for calculating graduation and dropout rates (i. e., a cohort rate). As of 2010, MDE will only use a cohort rate for reporting.
• Discontinued funding for High School Redesign—In 2006, the previous Superintendent of Education announced a shift in education and termed it Redesigning Education for the 21st Century Workforce, also known as High School Redesign. This focus on integrating academic and technical content in education was intended to prepare students for the workplace more effectively, lower the dropout rate, and make college an option for all Mississippi students. However, after funding a total of thirty-two school district pilot sites for implementation of the program, the Legislature did not fund High School Redesign for the 2009-10 school year. The effects on schools that have not received funding are unknown due to the lack of research on the effectiveness of the program.
• Discontinued funding for Mississippi’s Reading Reform Model—Mississippi has abandoned parts of the Mississippi Reading Reform Model because most of its funding came from sources that are no longer available. Currently, Mississippi has no statewide literacy plan.
• Reduced funding for the Children’s Progress Academic Assessment—In 2007, the Legislature passed a law requiring MDE to implement early literacy and numeracy screening assessment instruments. MDE selected a contractor to administer the assessments, but this contract will be eliminated in the 2010-11 school year due to budget cuts. The level of effectiveness of this program will not have been established due to the short period in which to monitor program results.

PEER does not believe that all of these changes are necessarily negative; in fact, some appear to move MDE in a positive direction in accomplishing its mission. The goal for the board and MDE should be to monitor the changes and measure progress over time to demonstrate the effectiveness of the changes.

Opportunities for Improving Accountability of the Department of Education

How could the accountability of the Department of Education be improved?

PEER provides the following list of twenty opportunities for improving the accountability of the Department of Education. Pages 85 through 104 of the report provide details on implementing each of these proposed opportunities.

• Define Mandates—The Board of Education, working through MDE, should operationally define as many mandates as possible.
• Advocate for Unified Reporting Mandates Based on Meaningful Outcome Measures—After developing logic models for its major activities (as discussed on page 94 of the report), the Board of Education, working through MDE, should present the Legislature with suggestions for improving statewide reporting mandates.
• Refocus the Office of Educational Accountability—The recent reorganization of the Office of Educational Accountability would allow for a shift from financial accountability to increased accountability of MDE programs and resources. The office could then play a critical role in helping the board fulfill its role to hold MDE accountable for programs and resources.
• Goal Setting—The Department of Education could improve its strategic planning process by establishing realistic goals based on research data.
• Goal Setting—The Department of Education could improve its strategic planning process by establishing interim progress goals for all three system goals.
• Goal Setting—The Department of Education could improve its strategic planning process by refraining from tying goals and outcome measures to national averages.
• Strategies, Outcome Measures, and Action Steps—The Department of Education could improve its strategic planning by converting outcome measures to measurable terms.
• Strategies, Outcome Measures, and Action Steps—The Department of Education could improve its strategic planning by improving consistency in designation of objectives as goals, strategies, or outcome measures.
• Strategies, Outcome Measures, and Action Steps—The Department of Education could improve its strategic planning by developing specific strategies for achieving each of the three system goals.
• Identify What is Under the Department’s Control—The Department of Education should differentiate between outcome measures and activities that are under its direct control versus those under the control of external entities.
• Link Resource Allocation to the Strategic Plan—The Board of Education should align resource allocation (both financial and staffing) with its strategic plan.
• Periodically Assess Staffing Patterns—To help ensure efficient use of staff in accomplishing educational goals, the Department of Education should periodically assess staffing patterns in relation to departmental workload and long-range plans for educational improvement, particularly in the area of improving instruction.
• Improve Reporting of Expenditures for Contract Staff—To establish accountability for contract staff, the Department of Education should improve its reporting of contractual expenditures for contract staff, annually compiling a summary report of categorized contractual service expenditures that provides the contract purpose, the expenditure amounts, the number of contracts, and retiree status of contractors.
• Apply Principles of Performance-Based Contracting—To enhance accountability for contract staff by focusing on end results, the Board of Education should familiarize itself with the elements of performance-based contracting and apply those principles when reviewing contract expenditure information presented by the department’s staff.
• Improve the Quality of Performance Measures—The Department of Education should improve the identification and use of its performance measures.
• Utilize Grant Funding for Development of a Statewide Longitudinal Data System—In June 2009 and May 2010, the U. S. Department of Education awarded Mississippi grants—one for $3.4 million and one for $7.6 million—to design and implement a statewide longitudinal data system. By the end of the second grant period in 2013, MDE expects to have a relational database linking all education (K-20) and workforce data.
• Use Research on Outcome Measures to Demonstrate Overall Effectiveness of Programs—The federal government has established meaningful outcome measures that the Department of Education could use as models for indicators of overall program or system effectiveness.
• Produce Multi-Year Trend Reports—To provide for better analyses of student performance data, the Department of Education should produce multi-year trend reports for districts. MDE could connect staff from the Office of Research and Statistics to district staff in order to increase the level of analysis and interpretation of student performance data.
• Use a “Data Dashboard”—The Department of Education’s implementation of a visual instrument that contains key indicators of performance (i. e., a “data dashboard”) would enable the board to see, at a glance, whether current efforts toward meeting its goals are on track and to respond quickly and appropriately when problems arise.
• Measure and Manage Change—In order to mitigate the challenges of changing standards and programs, both the Board of Education and the Department of Education could benefit from focusing additional effort on developing performance management capacity supported by a full complement of sound measurement tools (i. e., longitudinal data systems, creative use of multiple indicators, a data dashboard).

PEER Home Page         Full Text PDF (3,759K)