By Fred Dews | Originally Published at Brookings Institute. December 3, 2013

Rankings for the OECD’s 2012 Programme for International Student Assessment were released today, showing once again that Asian countries topped the charts while American 15-year-olds scored in the middle on the international tests of math, reading and science.

Tom Loveless, a senior fellow with the Brown Center on Education Policy at Brookings, has criticized the scores from China because, as he explained in a Brown Center Chalkboard piece in October, “China does not take the PISA test.”

A dozen or so provinces in China take the PISA, along with two special administrative regions (Hong Kong and Macao). But journalists and pundits will focus on the results from one province, Shanghai, and those test scores will be depicted, in much of the public discussion that follows, as the results for China. That is wrong.

Loveless explained why Shanghai, which topped all three areas of assessment, is not like the rest of China, due to a greater concentration of China’s elites, with more high school graduates going to college, and heavy parental investment in children’s education outside of school, plus the effects of the hukou system that forces many children of migrants back to their rural villages to attend school.

Shanghai has an economically and culturally elite population with systems in place to make sure that students who may perform poorly are not allowed into public schools. Second, the media should not present Shanghai’s scores as if they are indicative of China’s national performance in education. They aren’t, and no one will know how well China can perform on an international test until it participates, as a nation, under the same rules as all other nations. Third, the OECD should be far more transparent than it has been about the agreements it has with the Chinese government concerning who is tested and which scores are released. If China is treated differently than other PISA participants, the reasons for such special treatment need to be disclosed. And all data from the 2009 assessment of Chinese provinces should be released to the public domain so that scholars may conduct secondary analyses.

Loveless brought particular attention to the discriminatory effects of hukou:

About 66 percent of all children in China live in rural areas. The Chinese government’s policy initiatives in rural education are directed at school attendance, not higher test scores. A major reform effort targets migrant children. China’s remarkable economic boom over the past two decades has resulted in one of the largest mass migrations of people from rural to urban areas in human history. There are about 260 million migrants from the countryside now living in Chinese cities. Children in migrant families are severely disadvantaged educationally. Chinese cities, including Shanghai, require a hukou—proof of municipal residence, given at the place of family origin—to receive municipal services, including the right to attend public schools. …

Non-residents must either send children to private schools (often of low quality) or back to their rural villages. Every year, millions of children leave their families and return to the countryside, often to be cared for by grandparents or other relatives, because they cannot attend city schools. Recent efforts to reform the hukou system have been criticized as ineffective.

“The Shanghai scores frankly to me are difficult to interpret,” he told the AP today. “They are almost meaningless.”

Get more research on education policy and assessments from the Brown Center.

Fred Dews, Managing Editor of the Brookings Website