What is PISA and what do the results say about New Zealand schools?

Results from the “PISA” test are often quoted in the media and by various people with an interest in education.  However, it seems that there is little easily accessible information out there for principals that is suitable for sharing with staff, Boards, parents and communities (unless, of course, you want to wade your way through a 451-page report...)  The purpose of this newsletter is to provide a summary of some of the aspects of the PISA testing.  We hope you will find this useful.

The OECD

The OECD are the organisation that run the PISA testing.  OECD stands for “Organisation for Economic Co-operation and Development”.  Formed in 1961, its mission is to promote practices that will improve the economic and social well-being of people around the world.

PISA

“PISA” stands for “Programme for International Student Assessment” – an international study that began in 2000.  It aims to evaluate education systems worldwide by testing the skills and knowledge of 15-year-old students in participating countries/economies.  Specifically, PISA assesses how far students near the end of compulsory education have acquired some of the knowledge and skills that are essential for full participation in society.  Reading, mathematical and scientific literacy are covered not merely in terms of mastery of the school curriculum, but in terms of important knowledge and skills needed in adult life.  Visit Youtube at http://www.youtube.com/watch?v=q1I9tuScLUA to see a video that explains PISA in more detail with accompanying graphics.

Since the year 2000 over 70 countries/economies have participated in PISA.  There are reports available online of PISA results for 2000, 2003, 2006 and 2009.  Visit http://www.oecd.org/pages/0,3417,en_32252351_32235731_1_1_1_1_1,00.html

 

Each time the PISA testing is held, the test data is analysed and reported, and also a series of ‘in-depth reports” are written.  For example, the reports available from the 2009 PISA testing include in-depth reports titled: “Public and private schools: How management and funding relate to their socio-economic profile” (examining the socio-economic profiles of the public and privately managed schools that participated in the PISA 2009 survey; and “Let's Read them a story! The parent factor in education” (examining how and whether parents' involvement in their child's education is related to his or her proficiency in and enjoyment of reading).  Other reports are also published by the OECD from time to time using PISA findings, for example: “Against the odds: Disadvantaged children who achieve in schools” available at http://browse.oecdbookshop.org/oecd/pdfs/free/9810061e.pdf

What Does the Data Tell Us About New Zealand?

If the results of the 25 or so top-performing countries are compared, there are only 5 countries/economies who have performed above the OECD average in all tests since PISA began in 2000.  These countries/economies are (in order based on an estimated overall performance): Finland, Korea, Canada, New Zealand and Australia. 

If other countries/economies are added that have participated in more recent years and who have performed above the OECD average in all tests in which they have participated, the list is (again, in order): Shanghai (ranked first in all categories in its first participation in 2009), Finland, Korea, Hong Kong, Singapore, Canada, New Zealand, Australia, the Netherlands and Estonia.

Other countries that perform consistently well are: Chinese Taipei, Liechtenstein, Switzerland and Belgium.

The United States, by way of contrast, has scored above the OECD average only once in total across the 2000 – 2009 tests, while the United Kingdom has done so 5 times in total across the 2000, 2006 and 2009 tests.

The performance over time of New Zealand is varied.  Overall, our performance in Science in relation to other countries has remained about the same, while our performance in both Mathematics and Reading has declined in relation to other countries.  This is accounted for in part by the fact the higher performance countries have entered the PISA process (e.g. Estonia) and Shanghai, Hong Kong and Chinese Taipei are all ranked as separate countries/economies rather than aggregated into “China”. (Download the full Newsletter from the APPA website “Information Centre” page to view a graph of these rankings http://www.appa.org.nz/knowledge.htm)

The mean scores over time of New Zealand are also varied.  The mean scores in Science have improved over time, while those for Reading and Mathematics dropped in 2003 from 2000, but have remained largely consistent since 2003.  (Download the full Newsletter from the APPA website “Information Centre” page to view a graph of these scores http://www.appa.org.nz/knowledge.htm)

Download the full Newsletter from the APPA website “Information Centre” page to view a table showing the top 20 performing countries/economies in the PISA tests from 2000 – 2009.  This information enables you to track the performance of vaiours countries.  New Zealand is highlighted throughout  http://www.appa.org.nz/knowledge.htm)

What are the overall results of the 2009 PISA testing?

By age 15, student performance is strong across all levels relative to other countries.  In 2009, New Zealand was ranked 5th out of 34 OECD countries for mean PISA scores across reading, mathematics and science.

However, this achievement ‘hides’, to some extent, 3 problems:

  • there are wide disparities in achievement between ethnic groups.  Pisa scores of Maori and Pacifika students are much lower than the average for Pakeha/European students. Whilst the gap has narrowed slightly for Maori, it has not changed for Pacifika.  In both cases the gap is not closing nearly fast enough for these groups to catch up to Pakeha/European;
  • there are wide disparities in achievement for students from different socio-economic backgrounds.  New Zealand has the greatest difference in reading performance between students from different backgrounds out of all OECD countries.  The relationship between achievement and socio-economic background is evident in all countries, New Zealand is the least successful at mitigating the effect that a student’s background has;
  • too many students in New Zealand leave school before their OECD peers.  More of our 14 to 18 year old students have disengaged from the education system than in many other similar countries.

There are a number of other interesting facts as well.  For example:

  • the gap between our high and low performing students is one of the widest in the OECD;
  • in 2008 New Zealand ranked second in the OECD for the percentage of public expenditure that went to education; and
  • despite this, the dollars spent per student is in the bottom third of the OECD, and is half what the highest ranked countries spent (given that our GDP per capita is below the OECD average)

What are the specific findings for New Zealand in each area in 2009?

(Source: http://www.educationcounts.govt.nz/publications/numeracy/PISA-2009/pisa-2009-our-21st-century-learners-at-age-15/our-21st-century-learners-at-age-15 )

Reading literacy

  • New Zealand 15-year-old students’ overall reading performance was substantially higher than the average for the 34 OECD countries;
  • Only two OECD countries and two non-OECD partner economies performed better than New Zealand. Four countries were similar and the other 56 countries performed at a significantly lower level;
  • Close to one in six of New Zealand students were top-performing readers;
  • The proportion of New Zealand students with a low level of reading proficiency was similar to that in two high-performing countries, but the other two high-performing and the four top-performing countries showed smaller proportions;
  • Girls outperformed boys in every participating country. Among the top- and high-performing countries, New Zealand had one of the largest differences between girls and boys;
  • There were Asian, Māori, Pākehā/European and Pasifika students who performed at the highest level of reading literacy. While Pākehā/European and Asian students were more likely to be at the higher end, Māori and Pasifika students were over-represented at the lower end; and
  • The reading performance of New Zealand students, on average, did not change between 2000 and 2009.

Mathematical literacy

  • New Zealand students’ overall mathematical literacy performance was significantly higher than the average for the OECD countries;
  • Five OECD countries and six non-OECD partner countries or economies performed better than New Zealand, four OECD countries were similar, and the other 49 countries had a significantly lower performance;
  • New Zealand girls and boys achieved a similar mean mathematical literacy performance; and
  • New Zealand’s 15-year-olds mean mathematical literacy performance did not change between 2003 and 2009.

Scientific literacy

  • New Zealand students’ overall scientific literacy performance was substantially higher than the average for the OECD countries;
  • Only one OECD country and three non-OECD partner countries or economies achieved a higher mean scientific literacy score than New Zealand. Six OECD countries were similar, and the other 54 countries performed significantly lower;
  • New Zealand girls and boys achieved a similar mean scientific literacy performance; and
  • New Zealand’s 15-year-olds mean performance in scientific literacy did not change between 2006 and 2009.

 

Back