A Review of the Accountability Standards of the Mississippi Department of Education
Executive Summary
Introduction
In response to a legislative request, PEER conducted a review of the Mississippi Department of Education’s accountability standards to address concerns of whether the standards adequately measure school performance.
Background
State accountability standards must be designed in such a way that they effectively demonstrate actual school performance. If standards do not reflect actual student performance, education stakeholders and decisionmakers cannot make the appropriate decisions or necessary adjustments to improve schools’ and districts’ performance.
MDE’s accountability standards were created in order to communicate how well Mississippi’s schools and districts are performing, to identify schools and districts that need improvement, and to advise decisionmakers on necessary adjustments. Although college and career readiness was not included in the original purpose of the standards, as Mississippi shifts toward what will likely be more rigorous standards, college and career readiness will begin to shape the overall purpose of the state accountability standards.
The Accountability Standards Task Force, the membership of which is approved by the Mississippi Board of Education, makes accountability standards recommendations to the Commission on School Accreditation. Once recommendations are approved by the commission, the Board of Education provides the final approval before new standards or changes in standards go into effect. Selected staff at the department provide information necessary for the task force to make accountability standards recommendations.
According to the Mississippi Department of Education, changes in Mississippi state law, federal requirements, and the desire to make the accountability standards equitable for all schools and districts and easier to understand led to the adoption of MDE’s current accountability standards.
Characteristics and Components of a School’s Accountability Grade
MDE uses five different assessments to determine schools’ and districts’ accountability grades. These assessments are administered at various grade levels within schools and districts. After students take the assessments, MDE uses each student’s scale score to determine that student’s placement within one of four achievement categories (advanced, proficient, basic, or minimal).A
MDE then uses the percentage of students that a school or district has in the top two achievement categories (i. e., advanced and proficient) to determine that school’s or district’s accountability grade. MDE uses seven components (i. e., 700 possible points) to determine a grade for a school with no twelfth gradeB or a district with no high school and nine components (i. e., 900 possible points) to determine a grade for a school with a twelfth grade or a district with a high school. The components and their possible points are illustrated in Exhibit A.
MDE then uses cut-points established by the Accountability Standards Task Force to determine what total score must be achieved for a school to receive an A, B, C, D, or F accountability grade. MDE’s current cut-points are shown in Exhibit B.
How MDE Calculates a School’s Grade
The components of each school’s or district’s accountability grade contain three types of calculations: proficiency, growth, and graduation rate (see Exhibit A). Although MDE uses a graduation rate calculated in accordance with federal requirements (see page 26 of the report), the department has its own methods for calculating proficiency and growth, as follows:
MDE uses the growth components only for math and reading/language arts because math and reading/language arts are tested every year in grades three through eight and once in high school.
Conclusions
Because of the way in which Mississippi’s accountability standards are currently calculated, the standards do not provide stakeholders and the public with a clear picture of how Mississippi schools and districts are performing. Not only does the calculation of the current standards make it impossible to compare one school or district to another, but also to compare a school or district to itself over time. Mississippi’s standardized tests are carefully constructed to ensure that a student has mastered a certain level of competency; those tests alone should provide the criterion/standard for measuring school performance.
The Effectiveness of MDE’s Accountability Standards in Measuring School Performance
Achievement Categories Obscure Student Score Data
MDE’s use of achievement categories obscures actual student test score data because all scores in an achievement category are basically considered to be equal, despite the wide range of scores within a category. Determining proficiency by calculating the percentage of students whose scores are in the top two achievement categories, described in MISS. CODE ANN. Section 37-17-6 (5) (c) (i) (1972), compounds the problem because the range of scores deemed “proficient” is even wider, indicating an insensitive measurement instrument.
Combining Proficiency and Growth into a Single School Grade
Due to the way MDE’s accountability grade components are structured, combining proficiency and growth to determine a school’s or district’s accountability grade may not present the most accurate picture of actual student performance. PEER believes that growth is a very important factor in school performance, but if the way growth is calculated affects a school’s or district’s grade in such a way that it no longer demonstrates true student performance at that school or district, MDE’s overall purpose of the accountability standards is not being fulfilled. If the purpose of the accountability standards is to improve student achievement and increase the level of accountability of schools and districts, then more emphasis should be placed on proficiency--how a student actually performs on the assessments.
Emphasis on Growth Fails to Demonstrate Actual School or District Performance
MDE emphasizes growth in order to ensure that lower performing schools or districts that are improving positively contribute to their school’s or district’s accountability grades and, as required by state statute, to emphasize the progress of the lowest twenty-five percent of students in the school or district.
However, because of the way MDE has structured its accountability standards, in certain situations (such as a student whose score places them in the low 25% of scores), a student’s growth from one achievement category to another could be counted up to three times in the determination of the school’s or district’s accountability grade for a given year. Additionally, a school or district could appear to have made substantial growth gains, which might actually be inaccurate.
If proficiency scores are accurate, comparing proficiency scores from one year to the next or reporting scale scores divided by total possible scale score points would be other ways of showing whether a school or district improved from one year to the next.
Growth Multipliers Appear to be Arbitrary
MDE’s assignment of weights for learning gains appears to be arbitrary and results in the obfuscation of data, which impedes MDE from reaching its goal of improving student achievement and increasing school and district accountability.
While it might seem beneficial to provide incentives for schools and districts to encourage them to reach a higher level of achievement, if those incentives obfuscate data regarding actual student performance, the ultimate goal of improving student achievement and increasing school and district accountability has not been reached.
The Clarity and Accuracy of the Accountability Standards’ Presentation of Schools’ and Districts’ Performance
“Better of Two Years” and “Pausing” of Schools’ and Districts’ Grades
Although MDE developed its current accountability standards for use in the 2013-2014 assessment year, because of the implementation of college-and career-readiness standards that year, MDE has used “better of two years” or “pausing” adjustments to schools’ and districts’ accountability grades.
The “better of two years” adjustment meant that after having calculated the actual accountability grades for each school and district, MDE could decide, for each school and district, to apply the calculated grade based on the 2013-2014 assessment results or to retain the previous year’s grade. “Pausing” means that rather than calculating actual accountability grades for each school and district for that assessment year and choosing the “better of two years,” if approved by the U. S. Department of Education, MDE may automatically apply the previous year’s accountability grade.
These practices obscure the actual performance of students on assessments, therefore preventing MDE from making accurate comparisons among schools or districts to each other or to themselves over time. Further, accountability grades could reflect the accountability standards as they were calculated in a previous year rather than as they should be calculated in the current year.
How MDE Determines Accountability Grades for Six-Component Schools
Rather than determining cut-points for the accountability grades for schools without a twelfth grade and without a science assessment (i. e., six-component schools), MDE determines these accountability grades based on the actual distribution of grades for seven-component schools. MDE takes the A-F distribution of the actual grades of schools that have seven components and applies that distribution to the six-component schools. MDE then applies, or “links,” that distribution (i. e., the percentages for each A-F grade) to the six-component schools. This method forces the six-component schools into the seven-component distribution, reflecting the performance and growth of those schools rather than their own performance and growth.
The staff at MDE is aware of this problem and according to MDE, in May 2015 the Board of Education approved a rule that would address this problem.
The Method of Creating Assessment Benchmarks and Cut-Points is Not Criterion-Based
MDE’s current process for determining accountability grades is not being driven by student performance; rather, a Mississippi teachers’ group determines the benchmarks for student performance. MDE, the task force, and the Technical Review Committee, with the help of a consultant, determine the cut-points for establishing the accountability grades each year, maintaining significant control over the outcome of accountability grades.
Thus the processes used to determine achievement category benchmarks, A-F cut-points, and the number of possible points for each accountability component are subjective rather than criterion-based. Moreover, the placement of benchmarks and cut-points can affect the magnitude of trends, possibly giving some schools and districts an advantage in their accountability grades.
Changes in Graduation Requirements
In 2013, the federal government began requiring that the graduation component had to account for twenty percent of a school’s or district’s accountability grade. At that time, a student could not graduate high school in Mississippi unless he or she passed each subject area test (i. e., English II, Algebra I, U. S. History, and Biology I).
In January 2014, the State Board of Education voted to allow students to graduate if they failed one or more of their subject area tests but met certain other requirements. In March 2015, the board amended this action to allow additional options. The perception is that MDE has made graduation more easily attainable, thus allowing schools and districts to have better graduation rates.
Recommendations
To accomplish this, the Legislature should amend MISS. CODE ANN. 37-17-6 (5) (c) (i) (1972).
(Note: When proficiency is referenced in other recommendations in this report, it is with the assumption that an accurate proficiency measure will be utilized.)
MDE could do so by assigning a letter grade (A thru F) for proficiency, followed by another indicator to represent growth. The department could use a letter grade to demonstrate proficiency and an arrow that indicates direction to reflect whether a school has made adequate learning gains. For example, a school that made learning gains and earned a B in proficiency would have a grade of B↑. However, a school that earned a B in proficiency, but did not make adequate learning gains, would have a grade of B↓.C
For the separation of scores to take place, the Legislature would need to amend MISS. CODE ANN. Section 37-17-6 (4) (g) (1972) to allow for separate proficiency and growth indicators.
Further, to ensure that schools’ and districts’ grades can be reliably compared to those of other schools or districts for that year and that a single school or district can analyze its performance over a period of time, MDE should report schools’ and districts’ grades using the same accountability standards (as opposed to a previous year’s standards or a previous year’s grades).
Exhibit A: Components of a School’s or District’s Accountability Grade, as of 2013-2014 Assessment Year
| Components | Without 12th Grade | With 12th Grade |
| 700 Possible Points | 900 Possible Points | |
| Reading Proficiency | 100 | 100 |
| Reading Growth-All Students | 100 | 100 |
| Reading Growth-Low 25% of Students | 100 | 100 |
| Math Proficiency | 100 | 100 |
| Math Growth-All Students | 100 | 100 |
| Math Growth-Low 25% of Students | 100 | 100 |
| Science Proficiency | 100 | 50 |
| U.S. History Proficiency | 50 | |
| Graduation Rate-All Students* | 200 |
*MDE uses a federally approved four-year graduation rate calculation (MISS. CODE ANN. Section 37-17-6 [1972]). See page 26 of the report.
NOTE: MDE does not currently use “college and career readiness” and “acceleration” to calculate a school’s or district’s grade. However, according to MDE, these components will be included beginning with school year 2015-2016 results. See pages 52-53 of the report for more information on these components.
SOURCE: MDE.
Exhibit B: MDE Cut-Points for Schools and Districts, as of 2013-2014 Assessment Year
| Letter Grade | Cut-Point Range | |
| Without 12th grade | With 12th grade | |
| A | 518 or higher | 695 or higher |
| B | 455-517 | 623-694 |
| C | 400-454 | 540-622 |
| D | 325-399 | 422-539 |
| F | 324 or lower | 421 or lower |
SOURCE: MDE.
A Scaling refers to the process of converting a student’s raw test score to a common score that allows for comparison between students.
B An example of a school with no twelfth grade would be an elementary school.
C The growth component is not a measure of performance; it seeks to communicate where a school or district stands relative to current performance. An A school or district that earns an A in proficiency would not have much (if any) room for growth and would not necessarily have an arrow indicator following the school’s or district’s grade.
D Components of the MAEP funding formula process are defined in MISS. CODE ANN. Section 37-151-5 (1972). Currently, districts receiving a grade of C are classified as “successful” and if other statistical requirements are met, their expenditures form the base of the MAEP funding formula. Expenditures from districts receiving higher grades (A or B) or lower grades (D or F) impact the statistical calculations used in the MAEP formula, but expenditures from these districts do not otherwise impact the funding formula. The MAEP funding formula is calculated every four years, with adjustments for inflation during the intervening years. The most recent recalculation was for FY 2015. (A full recalculation of the MAEP funding formula will be completed for FY 2019.)
As noted above, MDE uses the MAEP formula to determine the amount of funding necessary to fund all schools at the funding level of the schools used in the formula that met the “successful” level of student performance. However, if the classification of student performance is flawed, as is illustrated in this report, the assumptions underlying the selection of schools to be used in the computation of funding are also flawed from a performance perspective.