Texas

College and Career Readiness for All Students

"The higher the standards that are targeted, the clearer it becomes that the only way to take the students to those standards is early intervention and a long-term focus on improving the fundamentals of teaching and learning. Setting lower standards and targeting short-term incremental test score gains, on the other hand, often tempts educators into shortsighted 'quick-fix' practices."

—Chrys Dougherty, Lynn Mellor, and Nancy Smith, National Center for Educational Achievement

2008 NCEA Methodology for Identifying Just for the Kids Higher Performing Schools: High School Grades

General Approach

NCEA's analysis included student test results from spring 2006, 2007, and 2008. For Grades 4 through 11, NCEA used an academic growth analysis that examines whether the students in each school perform above predicted based on their prior year's scores. Since students are not tested prior to third grade, NCEA looked at achievement levels of students from third grade who have been continuously enrolled in the school or district for three years or more, rather than prior test scores. In all cases, student and school demographics were taken into account in determining whether a school's students performed above predicted. For elementary schools, rather than aggregating the results of two different models, NCEA identified separate lists of higher performing schools in third grade and upper elementary grades.

The commonly used statistical technique of Hierarchical Linear Modeling (HLM) was used in both analyses. This technique makes a statistical adjustment to take into account the greater uncertainty of school results measured for smaller groups of students. The analysis was conducted separately for each subject, so that schools are ranked separately for reading/English Language Arts, mathematics, writing, science, and social studies.

Schools with many students close to the ceiling on the state test can't be measured well on growth. For this reason, and to include schools whose students are the highest performing in absolute terms, NCEA added to the list the top 5% of schools in the state based on the percentage of students scoring at or above NCEA's college readiness benchmark in reading, mathematics, or science. The state's advanced standard was used in any subject where no college readiness benchmark has been established by NCEA or ACT.

The steps in the analysis are as follows:

Step 1: Dataset creation

Merge each student's current-year test results with those of the same student from the previous year. Current year scores are merged with prior-year math and reading scores. For example, Grade 11 mathematics scores are matched with Grade 10 math and reading results.

Current-year fall enrollment records were also used to identify whether the student was enrolled at the same school during the current school year. Students without prior-year scores in the relevant subjects, or who were not enrolled in the same school in the fall of the current year, were not included in the analysis.

Step 2: Performance measure

For each subject, regress each student's converted scale score (z-score) in each grade and year on the following variables: the student's prior year z-scores in mathematics and reading; the student's low-income and English Language Learner status; whether the student is African American or Hispanic; the percentage of continuously enrolled tested students in the student's grade; the percentage of tested students who are low-income; the schoolwide percentages of English Language Learners (for elementary schools only), African American, and Hispanic students; and whether the school was a magnet school.

Hierarchical Linear Modeling (HLM) software will be used to calculate, for each school in each subject, a separate set of grade-and-year school measures and a combined school effect across grades and years.

Step 3: Identification of eligible schools

Identify those schools that were eligible to be ranked for a given subject based on their having an adequate number of students in the analysis in each grade and year in that subject. A school is eligible if it has no missing grades in the most recent school year and no more than the maximum allowable number of missing grades shown in the table below for the previous two school years.

Number of grade-year combinations in the school in previous two years 2-4 5-6
Maximum allowable number of missing grade-year combinations in the previous two years 0 1

A grade is missing for a given school year if either a) test score records are available for fewer than ten continuously enrolled students; or b) more than 20% of the records in that grade are deleted based on masking rules applied by the state.

Step 4: School rankings

Divide the eligible schools into four low-income groups (0–25, 25–50, 50–75, and 75–100% low-income students). Rank the schools within each of these low-income groups by their combined school effect across grades and years—the performance measure calculated in Step 2.

Step 5: School selection

Select schools that meet all of the following criteria:

  1. The school was in the top 10% of schools in its low-income group based on the combined school effect across grades and years.
  2. The school had better-than-predicted performance in all grades in the current year.
  3. If the school didn't meet Criteria 1 and 2 but was in the top 5% of the state based on the percentage of students meeting NCEA's college readiness benchmark, then the school was treated as if it had met Criteria 1 and 2. The website will inform users about which set of criteria the school made the list.
  4. In the case of reading/English language arts and mathematics, the school met federal Adequate Yearly Progress (AYP) requirements in the current year.
  5. In the case of writing, science, and social studies, at least 90% of the students enrolled in each tested grade in the current year took the state test.
  6. The school was not on the state's "watch list" based on the state accountability system in the current year.