This concept of using data to figure out how a person will act based on patterns of earlier behavior is just starting to come into vogue in American education. As a result of the federal No Child Left Behind law and other standardized-testing efforts, schools, districts and states collect massive amounts of information about students. They are only just now figuring out what to do with it.
"With all this data districts and states have," says Brian Williams of SAS, a North Carolina business software company that is helping schools construct analytical tools, "they begin to have the ability to identify through patterns what kind of learner an individual student is and begin to make some very smart decisions about what kind of lessons we should be using with them."
In the accountability age, parsing data has become a huge educational concern. No Child Left Behind mandates annual testing in reading, math and other subjects in grades 3 to 8. But the types of tests required under the law are too basic to do much good in terms of actually improving classroom instruction. They show whether a student has passed or failed, but offer little or no guidance about how to build on her knowledge or help a struggling student improve. That's why states and districts from California to Pennsylvania are turning to new and improved methods of both testing and utilizing data.
Educators frequently refer to the end-of-term NCLB tests as "autopsies" that can't help students who have passed into the next grade by the time test results come back. Many districts now are requiring additional tests throughout the year in order to get results they can use immediately to alter curriculum and tailor instruction for students who demonstrate a specific need. "We give our tests three times a year, sometimes four. We want to know during the school year how kids are doing," says Ray Wilson, director of assessment for the Poway Unified School District in San Diego County.
All this is leading to a quiet but significant shift in educational focus. When states started getting into standardized testing in a big way during the 1970s and '80s, they stopped looking at students as individuals and instead measured progress in terms of groups, such as classrooms or schools. Now, with low test scores from just one or two kids capable of putting an entire school's standing at risk, districts are turning their gaze back toward individual performance.
In this sense, educators are starting to live up to the promise implicit in the federal law. Even No Child Left Behind's many critics will concede that the law's emphasis on disaggregating data -- meaning each student's results have to be accounted for, not masked as in the past by averages of overall classroom or school performance -- has proven to be a silver lining. Not all schools are jumping on board the data-management bandwagon -- and many states game the system by lowering their standards so more students score as proficient -- but everyone concerned with education is now paying more attention to the performance of individual students.
NCLB continues to generate enough opposition from both conservatives and progressives that there appears little chance Congress will reauthorize it this year, as scheduled. Instead, the issue probably will be punted into the hands of the next president and Congress in 2009.
The law won't fade away, though. Without emendation, it will continue to require all schools to test students regularly and face penalties if too few make the grade. In order for all this testing to be helpful, though, states and districts are learning they can't simply collect yet more of the kind of data required by the feds. Instead, they have to make better sense of better numbers in order to improve instruction. "We're following the letter of the law," says Kory Holdaway, a teacher and Utah state representative, "yet still using our own state accountability system, which we see as being richer and more robust than the requirements under No Child Left Behind."
CLOSING THE GAP
For decades after World War II, educational policy was concerned largely with questions of equal access for racial minorities, females and the disabled. No Child Left Behind changed the rules of the game, demanding that all students perform at an adequate level. When they don't, the law considers this the school's fault, as opposed to the old "baggage" argument that many students come to school unable to learn because of personal circumstances or socioeconomic background. "We've had to shift on the run from providing access to making sure everybody is proficient, and that is a huge philosophical change," says Doug Otto, superintendent of schools in Plano, Texas. "It really caused us to look at each and every student -- that's a powerful change that's been brought about by the accountability system."Plano has been working with SAS to develop models to predict how kids are likely to do on their annual NCLB-mandated tests. The district runs its own tests early in the year and uses the results to inform the following months of instruction, seeking to arm teachers with specific analyses of each child's strengths and weaknesses. "There's a big push in our district for differentiation, how we can modify instruction for each student," says Beth Hubbard, who teaches at Aldridge Elementary. "Each might be in third grade, but they may each be on a completely different level."
The district helps with resources, whether it's training teachers to work in teams to address common problems or sending in literacy specialists. The results have been impressive. The district has closed nearly all the gaps that existed in 2003 between Anglos and racial minorities and low-income students in reading, writing and social studies test scores, while making good headway in math and science. Poway has done even better with its program, showing test-score improvement in every grade of every school for five years running.
The main change that Poway made was to persuade teachers, students and parents not to concern themselves too much with the scores on the state-mandated No Child Left Behind benchmark test. Instead, emphasis has been placed on setting learning goals for each child. Tests and classroom exercises have proven to be a better gauge of what level and type of instruction individual students need than the fact that a kid may be 11 years old and therefore should be at a sixth-grade level. The whole idea is that if kids are given individualized help on the fundamentals, higher test scores will take care of themselves.
As Poway has achieved success, its methodology has spread. When it started doing its own tests to supplement the state and federal requirements, only a handful of other California districts were pursuing a similar course. Now, there are hundreds.
Districts vary greatly in terms of how aggressively they use data, whether they're just keeping on top of the numbers they need for federal reporting requirements or whether they're cross-referencing test scores with demographics, transportation and attendance numbers and teacher qualifications in order to gain a fuller picture of why two kids in the same classroom are performing at significantly different levels. As always, there are problems associated with funding for staff to provide individualized instruction or even to receive the training needed to interpret test data comprehensively. "Districts and states are all over the map in terms of the systems they have in place and how they're trying to leverage data," says SAS's Williams. "But there's not a district we go into that doesn't want to do this."
GOING LONG
The data revolution in education is just getting underway. Learning how to analyze data to head off problems rather than just reacting to problems that testing has revealed is in its infancy. Most states still can't answer basic questions about their own students and schools. Only a handful can predict which students will succeed in college based on performance in high school, or how well students will do carrying a rigorous high school courseload based on achievement levels in junior high. "It's amazing how little we know," says George Wood, a high school principal in Ohio and director of the Forum for Education and Democracy.But states are starting to concentrate on these types of issues. In North Carolina, for example, the state wants more teachers and nurses, so it's asking what types of courses people who pursued those careers took at the primary or postsecondary levels -- and then trying to track them after college to see whether they've stayed in those fields or in the state, and if not, why. Arizona is interested in doing the same thing with mathematicians and scientists.
Virtually all states are gearing up to collect longitudinal data -- information about individual students that makes it possible to compare their histories throughout their time in public schools, from pre-K through college. Each student is assigned a number to make it possible to track his or her performance across districts and levels of education, while helping to protect individual privacy.
Florida has been a pioneer of this approach, with databases storing detailed information about students and staff throughout the state's educational system going back more than a decade. Its data is now being used in myriad ways. A state program that retains third-grade students who are not proficient in reading, for example, was based on findings from the data that showed third-grade was a trigger point for what happened in later grades. More positively, the state allows students access to data so that middle-schoolers can map out their coursework to follow paths to specific college majors successfully trod by students in prior years. "We're sitting on this gold mine of data," says Jay Pfeiffer, Florida's deputy commissioner of education.
Only a handful of other states have this type of longitudinal information, but nearly all aspire to have systems set up for its collection over the next few years. That will make it easier for states to push away from the snapshot results NCLB requires toward what is known as a "growth model." Under NCLB, schools are rated on whether they are making "adequate yearly progress" toward the goal of all students testing as proficient, with the results shown by one year's class of third graders being compared with the previous year's. Under the growth model, the scores of last year's third graders are compared with this year's fourth graders -- in other words, taking a look at how the same kids are doing after another year of instruction. The U.S. Department of Education is allowing 10 states to move forward with growth-model experiments.
TWO DIFFERENT DIRECTIONS
This interest in collecting more data and using it in more creative ways represents a significant change from the initial reaction many had to the federal accountability rules. At the same time, however, many states still resent the NCLB requirements and, in fact, have lowered proficiency measurements in order to help their schools show adequate yearly progress. Unless the law is altered significantly when it is finally reauthorized, more states will consider doing the same thing. And as states move closer to the law's impossible goal of 100 percent proficiency by 2014, they will soon face a situation in which most, if not all, of their districts will be rated as failing.That means No Child Left Behind will continue to move states and districts in two different directions. On the one hand, there will be some that try to fudge the numbers as best they can in order to make themselves look better. They will, in effect, be lying to students and parents about whether kids are actually proficient in basic subjects.
On the other hand, an increasing number of states and districts will be using data more confidently and aggressively in order to aid instruction. For them, the data requirements of No Child Left Behind have whetted an appetite for more and better data that does not merely show standardized results but can inform improvements in the classroom. "We're right on the verge of having the tools where learning truly is an individualized venture," says Pfeiffer. "I'd like to say we've exploited the heck out of the data that we have, but we really haven't."
