Participating in the PIRLS pilot

Issue: Volume 99, Number 2

Posted: 14 February 2020
Reference #: 1HA5SK

What’s it like for schools to take part in international research studies? And why are certain schools chosen? Education Gazette hears from teachers, students and Ministry of Education analysts about their involvement in a recent pilot study for the next PIRLS (Progress in International Reading Literacy Study).

PIRLS pilot

Two Wellington primary schools, Maraeroa and Johnsonville, have recently taken part in a pilot study as part of the Ministry of Education’s preparation for the roll out of PIRLS 2021, a large-scale international study of reading literacy. Teachers and students share their experience of the pilot, a key step in the five-yearly study. 

Deputy principal at Maraeroa School Leka Farquhar says her students were excited to get involved in the PIRLS pilot. “They felt very special to have been chosen; they said, ‘Oh it must be because they know we’re good at reading!’ 

“Our children don’t have many wider world experiences, so for them much of the content contained information that they found absolutely fascinating; references to Antarctica, for example.”

Student feedback

The students shared the following: 

“I enjoyed getting a new pen and how you were very helpful, even though I wasn’t sure how things will turn out. I had a lot of fun speaking with you.
I liked it because we got to take breaks and not just keep on going.”

“I learnt new words like ‘negative’ and ‘sensitive’. The stories were hard and challenging, and I had a go because I learn from my mistakes. I also enjoyed doing the smile faces at the end of the book. I would like to do the tests again when you come back.”

“I tried my best. I looked at the pictures, but I skipped most of the words.”

“Something I really liked in the test was that it challenged my brain! It was funny because one of the stories was about pollution and my reflection on my writing last week was about pollution and how it is making our planet smell yuck, and how it is hurting our sea creatures.”

“I had to look at the pictures to help me understand what the stories were about because the words were too hard, and I have never seen lots of new words before.”

Field work invaluable

For Megan Chamberlain, Jessica Forkert and Rachel Borthwick, who administered the pilot on behalf of the Ministry’s Educational Measurement and Assessment team, getting out into the field is invaluable to their work.

“We get to hear the children’s feedback directly which is so valuable as it helps us know what actually makes sense and is enjoyable from their perspective, and the kinds of questions they ask also let us know when they find something less clear,” says Megan, who is the National Research Coordinator for PIRLS in New Zealand.

Schools with Year 5 students in classrooms (i.e. not correspondence students) are eligible for the pilot and the EMA team approached schools with whom they’d previously had contact.

The pilot involved students reading texts and answering questions, then providing feedback on whether they’d enjoyed the stories. Their answers go into the pool of international examples of student responses, which are then used for the planning and development of scoring guidelines. It’s a good opportunity to add some New Zealand student responses to the collection.

What is PIRLS?

The Progress in International Reading Literacy Study (PIRLS) documents worldwide trends in the reading comprehension of middle primary school students. 

Students complete a reading assessment and questionnaire that addresses student attitudes towards reading and reading habits, and questionnaires are also completed by teachers and principals, and the students’ parents or caregivers. Information from this helps build a picture of students’ school experiences in developing reading literacy. 

New Zealand has been involved with PIRLS since the first study in 2001 which included 35 countries. Development work for PIRLS 2021 is underway with two New Zealand schools already having completed pilots. he field trial involving 30 schools will be conducted in mid-March and early April, with the main study, involving 275 schools, to be administered during term 4 this year.

In the last PIRLS 2015 and 2016, New Zealand children scored an average of 523.

There was wide range of country averages with the Russian Federation at 581, and South Africa at 320.

Data reveals that a “reasonable” number of New Zealand children reached the high benchmark (41 per cent), meaning they demonstrated an ability to engage with increasingly complex texts and questions. However, slightly fewer children overall reached three of the benchmarks (low, high and advanced)
in PIRLS 2016 than in 2011.

Other key findings

  • New Zealand children’s performance was, in general, relatively stronger when reading literary texts than informational texts.
  • Reading comprehension was relatively stronger when using reasoning strategies (interpreting, integrating and evaluating) than using text-based skills (retrieving and straightforward inferencing).
  • Children who always or almost always spoke the same language at home as they used in the test (English or te reo Māori), generally had higher reading achievement than those who generally spoke a different language at home from that used in the test. Since PIRLS 2011, the size of this difference has decreased.
  • New Zealand children were much less confident about reading than their international peers. This is important because children who lacked confidence scored about 120 score points lower, on average, than those who were very confident.

BY Education Gazette editors
Education Gazette | Tukutuku Kōrero,

Posted: 9:25 am, 14 February 2020

Get new listings like these in your email
Set up email alerts