Texas

College and Career Readiness for All Students

"The higher the standards that are targeted, the clearer it becomes that the only way to take the students to those standards is early intervention and a long-term focus on improving the fundamentals of teaching and learning. Setting lower standards and targeting short-term incremental test score gains, on the other hand, often tempts educators into shortsighted 'quick-fix' practices."

—Chrys Dougherty, Lynn Mellor, and Nancy Smith, National Center for Educational Achievement

2008 NCEA Methodology for Identifying Just for the Kids Higher Performing Schools: Lower Elementary Grades

General Approach

NCEA's analysis included student test results from spring 2006, 2007, and 2008. For Grades 4 through 11, NCEA used an academic growth analysis that examines whether the students in each school perform above predicted based on their prior year's scores. Since students are not tested prior to third grade, NCEA looked at achievement levels of students from third grade who have been continuously enrolled in the school or district for three years or more, rather than prior test scores. In all cases, student and school demographics were taken into account in determining whether a school's students performed above predicted. For elementary schools, rather than aggregating the results of two different models, NCEA identified separate lists of higher performing schools in third grade and upper elementary grades.

The commonly used statistical technique of Hierarchical Linear Modeling (HLM) was used in both analyses. This technique makes a statistical adjustment to take into account the greater uncertainty of school results measured for smaller groups of students. The analysis was conducted separately for each subject, so that schools are ranked separately for reading/English Language Arts, mathematics, writing, science, and social studies.

Schools with many students close to the ceiling on the state test can't be measured well on growth. For this reason, and to include schools whose students are the highest performing in absolute terms, NCEA added to the list the top 5% of schools in the state based on the percentage of students scoring at or above NCEA's college readiness benchmark in reading, mathematics, or science. The state's advanced standard was used in any subject where no college readiness benchmark has been established by NCEA or ACT.

The steps in the analysis are as follows:

Step 1: Dataset creation

Merge each student's test results for each year in the analysis with the same student's fall enrollment records from the same year and the two prior school years. Use this information to separate out test results of students who were continuously enrolled at the same school or district for at least three years ("continuously enrolled students"). If the school's grade configuration does not allow for three years of continuous enrollment, as in the case of a school with only Grades 3-5, then the students must be enrolled in that school for the maximum number of years that the grade configuration will allow, and they must be in the same district for three years.

Step 2: Performance measure

For each subject, regress each student's converted scale score (z-score) in each grade and year on the following variables: the student's low-income and English Language Learner status; whether the student was African American or Hispanic; the percentage of continuously enrolled tested students in the student's grade; the percentage of tested students who are low-income; the school-wide percentages of English Language Learners, African American, and Hispanic students; and whether the school was a magnet school.

HLM software is used to calculate, for each school in each third grade subject, a separate set of school effects by year and a combined school effect across years.

Step 3: Identification of eligible schools

Identify those schools that were eligible to be ranked for a given subject based on their having an adequate number of students in the analysis in each grade and year in that subject. A school is eligible if it has no missing grades in the most recent school year and no more than the maximum allowable number of missing grades shown in the table below for the previous two school years.

Number of grade-year combinations in the school in previous two years 2-4 5-6
Maximum allowable number of missing grade-year combinations in the previous two years 0 1

A grade is missing for a given school year if either a) test score records were available for fewer than ten continuously enrolled students; or b) more than 20% of the records in that grade are deleted based on masking rules applied by the state.

Step 4: School rankings

Divide the eligible schools into four low-income groups (0–25, 25–50, 50–75, and 75–100% low-income students). Rank the schools within each of these low-income groups by their combined school effect across grades and years - the performance measure calculated in Step 2.

Step 5: School selection

Select schools that meet all of the following criteria:

  1. The school was in the top 10% of schools in its low-income group based on the combined school effect across grades and years.
  2. The school had better-than-predicted performance in all grades in the current year.
  3. If the school didn't meet Criteria 1 and 2 but was in the top 5% of the state based on the percentage of students meeting NCEA's college readiness benchmark, then the school was treated as if it had met Criteria 1 and 2. The website will inform users about which set of criteria the school made the list.
  4. In the case of reading/English language arts and mathematics, the school met federal Adequate Yearly Progress (AYP) requirements in the current year.
  5. In the case of writing, science, and social studies, at least 90% of the students enrolled in each tested grade in current year took the state test.
  6. The school was not on the state's "watch list" based on the state accountability system in the current year.

A school meeting all of the requirements in Step 5 is deemed to be higher performing in the subject in question.