Education headlines this week have been dominated by the success of Shanghai high school students in this year’s PISA survey. If only journalists had asked those top scoring students about sampling issues, they may have learned that comparing Shanghai students to the otherwise national surveys included in the study is, well, something top scoring maths students would not likely do.
The PISA report (Program for International Student Assessment) is based on the largest single study of global schooling. Undertaken by the OECD, the survey includes 6,400 students from 155 schools in Shanghai, and the half a million 15-year-olds whom took part globally. PISA surveys began in 2000. Shanghai joined in 2009, and in the most recent tests, in 2012, came first in all three categories tested – reading, maths and science.
The reality is however that China is the sole country surveyed which does not permit the use of national data. Yet Shanghai is an entirely skewed sample of China’s academic performance. As one of China’s richest provinces (Shanghai is an independent municipal province), it is home to some of China’s best high schools and top universities. It also home to some of the highest earners in China, who are more than willing to spend significant sums on their single child’s education. Roughly 4 in 5 of Shanghai high school students go on to college, against a national rate of closer to 1 in 4.
Similarly, Chinese students face an exam-taking pyramid throughout their lives. An up or almost out model exists that pushes exam performance to whole new levels, least of all as result of population pressures. At the same time, that scale also means that it is wrong to believe that all Chinese students do is memorise. The numbers of hard-working and talented students are so high that in the end the only way to stand out between geniuses is to have memorised everything also.
The truth of the extreme of Shanghai’s PISA success however should be able to be explained by any of those talented Shanghai students. Given their position of socio-demographic privilege in China, let alone as young residents of one of the world’s fastest rising economic hubs, they are educated enough to know their results would at least somewhat more reliably be compared to the elite high school students of the nicer suburbs of Sydney rather than Australia as a whole, and their peers in high schools in Oxford, Cambridge MA, and Silicon Valley. If Shanghai students can outcompete those more equivalent peers, then yes, old education powers need to do a re-think of their education trajectories.
In the meantime, the OECD is rather ironically informing us of Chinese mathematical educational success without applying consistent statistical principles across its own sample. Maybe they could seek the advice of some students – in Oxford, Shanghai, or even Canberra.