New digital script gives clearer picture of student learning

His new programme, called ‘It worked! Data analysis made easy’, gives teachers an understanding of the p-value – a number that shows the likelihood of results being a random variation – so they can draw legitimate conclusions about their teaching practices.

“It means you can use a teaching initiative, or put in a new technique, like introducing computers or whatever you’ve done and your marks change, your credits change. What are the chances that this change is random?” says Darcy.

“I’ve trained them up so if it’s below 0.1 they choose a sentence which says the credits are notably better than previous years, if it’s 0.05 it’s significantly better than previous years. What it gets down to is you can introduce something, you can look at the credits and say where it’s made a significant difference.”

In order to improve student learning, teachers need to change their teaching practices. Evaluating new initiatives properly is the key to successful outcomes.

“The average time a teacher waits after asking a question before they either accept an answer, clarify or answer it themselves is 0.8 of a second. If you can get people to wait 10 seconds, you can measure that improvement. That one thing is enough to cause learning to be significantly enhanced,” Darcy says.

“If your question is so superficial that someone can answer it in 0.8 of a second, it means they already knew the answer, they didn’t have to think about it. It was a factual recall.”

Although it is foundational, factual recall is at the bottom of the thinking hierarchy. A little more wait time allows for deeper thinking and comprehension. After further thinking, teachers can tap into students’ analysis and evaluation.

“If you ask a real question, something you have to think about, give someone at least 10 seconds to think about it, you get more people thinking about it, you get more people answering, more people trying ... if you look at your distribution, you can measure that difference with just that one thing.”

How the programme came about

“It used to take me about a day to analyse one of my subjects, but I did that for my science department because I enjoyed it and thought it was valuable,” says Darcy

After watching online tutorials about using scripts, he created his own in the hopes of finding a more efficient solution. His new computer programme reduces the amount of information to be extracted manually and helps Darcy to spend less time finding the information needed.

“The analysis which used to take me a day was taking me five minutes,” he says.

“Over the years I’ve had various colleagues ask me if I can help them and it was always ‘Yeah I can, but I need time to do it in’. I suddenly realised that I actually could do the analysis for all of my colleagues.”

This year, his focus is on increasing teacher capability within the Gisborne Community of Learning | Kāhui Ako where he is now a cross-community teacher. This means extending the programme to include primary, as well as secondary schools and helping teachers to understand their own data.

“One of the things about community learning is it’s ground-up planning,” Darcy says.

“We’re undertaking some sample analysis and we’re going to run a dialogue for all the primary principals so they actually see and hear not only me explaining how it works, but also teachers explaining how they use it.”

One school within the Community of Learning  was interested in analysing test results to see how their English professional development had been working.

“They were informed about gains in several measures, but actually one of the measures went down because they stopped teaching phonics, so they decided to start teaching phonics again to deal with that.”

The script can be applied across different year levels and to different national measures, as well as individual school goals. One school looked at their literacy test, while another school measured curriculum levels.

“With one school I’ve looked at they have a school goal that each student will make at least one year’s progress in terms of curriculum levels each year and they wanted to know whether that was actually the case. That’s a standard school question.”

Darcy is a firm advocate of evidence-based practice and says without this type of data analysis, teachers have no way of knowing whether learning has improved.

“Teachers are required now to carry out an inquiry to try something and then evaluate whether it worked and this is how education improves,” he says.

“If you do that progressively, the student learning is going to get better and better progressively, because if the change you make doesn’t improve student learning you can find that out.”

2018 Bright Spots Award

The Gisborne Boys’ High School’s analysis programme received a Bright Spots Award in 2018 and was also supported by the New Zealand Qualifications Authority (NZQA) and the Ministry of Education.

The Bright Spots Award provided funding for teacher relief, allowing greater uptake of the programme and helping teachers to collaborate and build analysis and evaluation into their standard practice.

NZQA designed and organised a new output providing all of a school’s NCEA results and created a script turning this whole-school data into subject-based data.

Applications for the 2019 Bright Spots awards open at the beginning of Term 2. For more information visit the Education Hub(external link).