What works? Teacher Action Research works.

 ‘I think the thing that attracted me to TARP was the fact that actually what we are doing is looking at something that is a lot more research-based… it’s much more useful to be thinking about what does make a difference in the classroom; how does one implement it; how does one disseminate it and how does one go through the process of being more professionally engaged with the science and craft of teaching.’ TARP teacher, 2019

What was the Teacher Action Research Project?

Between October 2018 and June 2019 I delivered a Teacher Action Research Project – TARP – to a group of 8 secondary school teachers from across England, meeting on three occasions at UCL Institute of Education (IOE), where I work. The project was funded by UCL Widening Access and Participation, whose remit is to increase the uptake of places at leading universities by students from disadvantaged backgrounds.

The teachers chose themes and interventions that they judged likely to make a substantial difference to their own students: some focused on aspects of their classroom practice that they wished to adapt; others conducted their action research on an element of their wider school responsibility. The only stipulation was that they must concentrate on outcomes for their disadvantaged students.

Summary of the interventions and outcomes

Three were interested in the process of student learning; they were keen to increase their students’ awareness of how they learn, and to see the extent to which this would contribute to their academic progress. These projects may be labelled metacognitive strategies. The self-regulation of learners was a feature across several other projects. One hypothesised that, to address the demands of new linear exams at A level, teachers needed to address the metacognitive skills of their students and could best achieve this through explicit modelling of written responses; two English teachers conducted projects which focused on boosting subject-specific vocabulary in Key Stage 4. These three enquiries can be loosely grouped as language and literacy. One teacher, whose whole-school role embraces Careers Education, was interested in the link between students’ self-efficacy and realistic career choices; the project of the final practitioner-researcher saw her department take a range of new evidence-informed approaches to teaching Geography at Key Stage 3. These last two come under the very broad heading curriculum development but, like the rest, are complex interventions concerned largely with students’ self-regulation.

There were many common findings across the eight action research projects. Many saw student confidence as a prerequisite to progress; others saw that increased confidence was an outcome of their projects. The same could be said for the issue of teacher-student relationships: some saw them improve, while others identified them as a continuing threat to successful outcomes. Teaching strategies that placed an emphasis on student self-reflection, their metacognition and self-regulation all had promising results for disadvantaged students and others. Likewise, modelling of exemplar answers, emphasis on appropriate subject vocabulary and the embedding of strategies which force students to retrieve prior knowledge: greater focus upon these pedagogical approaches did bear fruit for many of the disadvantaged students whose progress was tracked. Not all projects succeeded on their own terms. Those where the number or complexity of interventions were greatest found it difficult to implement change or to track the impact of the changes they did implement. In some few cases, the participating teachers speculated that their innovation may have in some way hampered the progress of the students they were trying to help, but this was a minority finding. Others were able to reflect upon the extraneous factors which continue to impede the progress of their disadvantage students.

What TARP has to say about innovations that make a positive difference to disadvantaged students

Modelling

Modelling happened in several ways across the projects. In some cases this involved the use of a visualiser in class to highlight features of exemplary practice; in another it included the repeated use of model answers provided by exam boards; one teachers referred to the ‘talking aloud’ strategy, whereby the teacher models their thinking as they construct an answer ‘live’ in front of their class. One found that ‘it was clear that all students, disadvantaged or not, wanted to see what excellence looked like and had a genuine thirst for knowledge.’ The progress of her High Prior Attaining Pupil Premium students improved, but so did that of most of the rest of the two Year 11 English classes in her study. Another – who had analysed the planning and teaching of A level subjects that had performed well since the reintroduction of linear exams – wanted to see what lessons could be learned by those departments whose results had dipped. A student survey identified more modelling of exam questions as the area that would help them improve the most. Teachers in two subjects agreed to a six-week implementation of explicit modelling – specifically of exam board answers and typically within homework – and to track the impact upon the bursary students through internal assessment and mock exams. In maths half the bursary students in the set improved their grade while the rest remained as they were. The picture was similar in economics, where one bursary student improved their grade while the other remained static.

 

Building good relations and a culture of trust

One teacher contended that ‘establishing a culture of trust within a class is central to students making progress.’

Some participating teachers pointed to the trust that was an outcome of involvement in the project. One (who did a lunchtime group intervention focused on the students’ knowledge retrieval skills) saw that ‘as the interventions progressed, the girls started to become more and more accountable for each other and encouraging one another to meet the demands of their courses.’ She spotted a relationship between that burgeoning culture of trust in the group and their regard for themselves as learners, which (for all but one of the group) improved also. Another noted that, the more familiar his Year 13 English Literature students were with simple cognitive processes and the rationale behind the approaches to feedback he was taking, the more they trusted him and the tasks he was setting.

 

Student confidence

For this teacher, this trust in him and his teaching was a contributory factor in the students’ growing confidence going into their exams. One of his students said she had begun to ‘trust my own thinking’.

Among one teacher’s Year 10s, after the four months of her careers and options guidance programme

‘82% of the students felt that they were now more confident to make decisions related to their career paths highlighting that the project has been somewhat successful in breaking down the barriers to career self-efficacy and increasing student confidence.’

She concluded that it was the early and structured nature of the intervention which helped the students be more confident in their decision-making.

A geography teacher, implementing a new Key Stage 3 curriculum, explicitly measured the confidence of her students using ‘feedback circles’ in the lessons. She and her colleagues were alarmed at the high rate of students describing themselves as within the ‘panic zone’ when they were asked to think hard. But, persisting with their innovations – and in particular the demand that their students ‘speak like a geographer’ – they found gradually that ‘most of the students moved out of the panic zone and into the stretch zone’.

Teaching which boosts vocabulary

An English teacher noticed that students, who were unfazed at the prospect of producing two to three pages of work would still feel anxious about giving ‘the right answer’, and this stopped them writing. A ‘heightened focus on relevant terminology’ was one of the factors leading to improved progress for Pupil Premium students with higher prior attainment. The geography teacher’s students, as they moved out of their panic zone, began to expand their discipline vocabulary.

‘This started to finally pay off and after a few weeks teachers and students started to use these words as a matter of course (in the interviews this is something teachers referred to again and again and were quite amazed by.)’

Another English teacher delighted that some of her students had become ‘word hungry’ – asking for vocabulary and prepared to use the thesaurus – but she sounded a warning note too.

‘Students accessed and used a range of complex and higher level vocabulary that they would otherwise either not have the tools or confidence to access… Students are empowered when given tailored vocabulary – however students do not have the skills to be able to build upon this independently.’

They remained reluctant to source higher level vocabulary for themselves, or to transfer the new vocabulary they had learned to other subject areas.

 

Metacognition and Retrieval Practice

Several of the TARP teachers used their enquiries to test their supposition that students make better progress the more they are aware of the processes of learning. As noted already, an A level English teacher found that his students trusted him and his methods the more they recognised that they had some basis in cognitive science. He concluded that placing subject knowledge at the heart of the curriculum yielded significant benefits that seemed to go beyond simply recalling knowledge.

‘The ease with which the students retrieved information decreased cognitive load. It seemed that being able to retrieve knowledge with relative ease freed up cognitive space to concentrate on structuring essays and writing well.’

The teacher of the lunchtime intervention group found that they began to bring in examples of dual-coding they had produced in Science and discussed flash card examples for their humanities revision.

‘It was evident that they had not only taken a more independent approach to selecting and using appropriate recall and retrieval strategies but were also open and willing to share these with their peers.’

Caveats

These were small studies, a few conducted across a department or between two or three groups, most confined to a single class. Within that, the teachers tracked the impact of their projects on the much smaller number of specifically disadvantaged students. It is an advantage of action research that it allows participating teachers to spot the often very small, yet significant, changes that can occur when a modification is made to teaching. The methodology is essentially qualitative, which allows the teachers to report on the attitudes and behaviours which are both factors in and outcomes of learning. But the datasets are necessarily tiny, and the measures of attainment – short of public exam data – are assessed by the teachers themselves. (TARP concluded before the end of the Summer term, i.e. before the release of the 2019 public exams.)

The teachers on TARP were clearly invested in the successes of their interventions, and they were mindful of the pitfalls of bias. Several, in their own reporting, speculated upon the possible negative consequences of their study. One teacher’s English department colleagues undertook a lesson study into the expanded use of higher order language to enable their students to express perspective. She noted that they enjoyed the ‘safe’ bank of vocabulary they were given and asked

‘Are we inhibiting students by providing them with too much? Do we over-stimulate or confuse students with SEN? Some students feel they have to use every new word, which then makes their written responses convoluted.’

 

The geographer in the group also expressed a residual concern for her lower prior attaining students. When she and her geography colleagues rewrote their Key Stage 3 curriculum, they found that their higher and middle prior attainers responded well. But the lower prior attainers – especially those deemed to be at Levels 1 or 2 – did not. She remained confident in the approaches taken by her colleagues but was cognisant also of making too bold a claim.

 

Final comments

The Teacher Action Research Projects have made a positive difference to disadvantaged students in the eight schools. They have also affected the teachers themselves, their confidence as practitioners and their sense of their place in the profession. All intend to continue with the practices they innovated this year, and several have expanded their plans.

Reflecting on TARP, one commented

‘I think that to be effective educators, we need to be able to process, explore and consider the research before it is implemented, so that it can have a more positive impact on our students.’

The final word should go to another, for whom the project has made her think about

‘how small changes can have an impact without adding to workload and that sometimes it is doing more of what already works rather than incorporating something completely new.’

 

I full version of this report will soon be available on the UCL website at:

www.ucl.ac.uk/widening-participation/TARP

Headlines are poor proxies for research.

So here goes. The other day Laura McInerney  at ResearchEd Blackpool posted on Twitter this much-repeated list of ‘Poor Proxies for Learning’, asking How many of these are you guilty of thinking will equal learning?

I replied

I have disliked this list for some time. A teacher who achieves these in their class IS doing a fine job. And, if not individually, in combination these will be good proxies for learning. Leaders should not be criticised for wanting to see them.

You can see some of the replies and thoughts on https://platform.twitter.com/widgets.js“>proxies for learning here. Of course, I understand Rob Coe’s point that busy or engaged or motivated students, or ordered classrooms, or ones where the course is covered are not the same thing as learning, but I worry about the reactions of some teachers and school leaders seeing this list.

  • It sneers at teachers who strive to achieve exactly the things which are on this list.
  • It implies that these conditions are not worth striving for.
  • It may lead some teachers and school leaders to downgrade the achievement of these conditions.
  • It encourages the despairing conclusion that we cannot ever know that learning has taken place until public exams results are in – and that only those matter.

I know I have substituted the word ‘conditions’ for ‘proxies’. I know that this means I am guilty of a category error, where I mistake the difference between a prerequisite for a thing and the thing itself. But consider the position of the teacher teaching a class, or a school leader watching the lesson. When I was teaching (I remember it well – it was not long ago), I would have been delighted if I ever managed to cover the full content of a curriculum, when that curriculum kept changing and growing. If I ever motivated all of my students so that they were interested in what I was saying and engaged in what I wanted them to do, I would have chalked that up as a stunning success. There were very many times when my classroom was not ordered or calm, so I was definitely happy when it was. More to the point, I know that my students did learn better when they were engaged in busy, well-ordered lessons. It took me several hard years to achieve lessons like those on a consistent basis. And, when it became my role to help other teachers, it took much energy on my part (and much more on theirs) to achieve something like those conditions in their lessons.

School leaders should strive for those conditions in their schools. I mean, imagine if they did not. I know that it is not the intention of those who produce and reproduce the proxies list to suggest that they are (all) undesirable, but I do fear that that is how they are interpreted.

Those who publish research on well-visited platforms have a duty to consider the consequences of misinterpretation, and to at least try not to cause it. Some recent EEF reports have been headlined in a way that I feel is irresponsible. For example, what is a school leader to make of this?

New EEF trial results: ‘light-touch’ approaches to research unlikely to impact pupil outcomes

Will they see the ‘light touch’ and think, erm… that’s exactly what I do? For most schools, to be engaged in research and to seek to be informed by research often is a light-touch exercise. They trawl through their twitter feed late at night, look for some promising blogs to read, maybe look to make something more of the connections they have with a local university. Surely to goodness, these are activities to be encouraged. It is not encouraging to learn from an EEF sub-editor that it is unlikely to impact pupil outcomes. Worse still, it might lead some leaders to conclude that that journal club they have running is a waste of time. They should have read the report, of course, and added their own emphasis (as I have here):

While the evaluators from the University of Bristol found no evidence that the programme led to improvements in reading outcomes for ten and eleven year olds, the findings suggests that there may be a relationship between how engaged teachers are with research, and the attainment of their pupils. There was also some evidence that being in a Research Learning Community increased teachers’ engagement with research.

And perhaps they ought also to have read EEF Senior Researcher Jonathan Sharples’ blog . In it, he concludes that the light-touch research approaches described in the report were indeed worthwhile, and he references the 2016 EPPI review.

Importantly, in addition to reviewing different mechanisms to mobilise evidence, they also looked at the behavioural requirements that were necessary for those various approaches to have an impact. This included having:

  1. opportunities to engage with the interventions
  2. the motivations to do so, and
  3. the skills and capabilities to understand and use the outputs.

Dig down far enough into the EEF website and there is everything there the teacher and leader needs to be well-informed. Bravo. However, how many teachers and leaders crack their spade on the hard crust of the top line?

digging_hole001

Headlines are poor proxies for research.

 

 

#Lower case r, upper case ED, 17

ResearchEd17 could be forgiven for being a bit self-conscious: in recent weeks it has been spoken of less favourably, had its grassroots raked over, its biases heat-mapped. Sure enough, when I arrived (like a marathon runner along Cheering Lane), it was clear that the cheering crowds had stayed away.

total gridlock ‘Total Gridlock’

There was barely a complimentary canvas bag in sight. Was TB defeatED? Was his love affair with geeks in the staffroom and policy wonkers in the anglosphere endED? Had his hash been tagged for the last time?

Nope.

It turns out that Chobham Academy, in the shadow of the Queen Elizabeth stadium, is larger than the average comp: they hosted the 2012 volleyball in their foyer – which is where we found the hordes and their canvas bags. Tom was in pink, Helene was handing out raffle tickets to win lunches, and all was right with the world. Someone (we blamed the Harris peeps) forgot the free pens, but we are high starters with 21st Century skills happy to photograb speakers’ slides and live-tweet our research. Our only concern was failing phone-battery power. #WorkingOutHowToMakeADyingPhoneWork.

I have engaged with ResearchED on many previous occasions, but this was the first time I got to engage in ResearchED. If you have never done it, you should, it’s miles better than sitting on the floor all day, and you get to rub shoulders with famous people and Nick Gibb. And you get to start the day in the speakers’ lounge, this year styled as the training room. I limbered up there with my fellow presenter James Mannion and teachers from the City of London School.

SpeakersRoom

Imagine if all the boxers from the undercard were put in the same dressing room: it’s just like that. Your competitors (those you suspect will draw a bigger crowd) are in there, as are real live people who misleadingly look nothing like their Gravatar. I’m represented to the online world by a three year old drawing by my daughter, who at the time loved me enough to ignore my greying hair. In short, there is little small talk in the speakers’ room; just people doing some research before it’s their turn to present.

You would get fit working in this school. It’s designed like one of those spiralling coin boxes that entertain you as you give your old euros to charity. I joined Lisa Pettifer for a lap on the second floor. ‘Is M213 this way, Lisa?’ ‘Just keep on walking, Mark, and it soon will be.’ Jonny Peacock and Christine Ohuruogu go to school here. Sensible presenters like Christian Bokhove wear t-shirts with penguins on them. He told us that spinach does not contain lodes of iron, that the myth apparently occurred during the Great Decimal Point shift, that that too is a myth traceable to the Readers’ Digest, which may or may not be available in the Netherlands. And the moral of his tale was: don’t pretend that you know stuff really well unless you really do; try a little nuance when discussing cognitive psychology on twitter. I will try my best, Christian, but it’s in my nature…

‘Mark Quinn et al’ were giving their talk on practitioner enquiry during session 3. Six of us at the front, going for the prize of Those Most Likely to Outnumber Their Audience. Tom helped us out by scheduling against us Sherrington, Christodoulou, Weston, Jones, Creaby, Davenport, and Hood and Fletcher-Wood… et al. Well, I don’t know how many flocked to the gurus this year but we were very happy with our little turnout. Everyone had a seat, they could join in on the chat, and I could pick out old colleagues Barbara Terziyski and Vivienne Porritt.

ResearchEd17 There are more people just out of shot.

We were making the case for the gnomes of the research garden, teachers carrying out the sort of micro-research that tests out the grand theories without ever being reported. Nick wanted to know if his year 8 had a growth mindset, and if they did did it show up in achievement and effort data. (They did, and it did not.) Joe, the head of RE, has an ontological interest in creativity: he wanted to know what his students thought about it and where they would like to see more of it. Richard wanted his year 11s to be more reflective about their work, had a hunch that peer feedback would help him get there, and found that it did. The great thing about ResearchED is that it showcases some of the disciplined enquiries that real teachers are conducting, but even if ResearchED did not exist these teachers would still be gnomically enquiring away.

Amanda Spielman finished my day. That’s great because she is passionate about workload, so much so that Ofsted will ask headteachers how they are reducing it. She is also passionate about research and will turn the inspectorate’s attention that way increasingly. I asked her if she would research the impact Ofsted have on workload, and act on the results. I can imagine headteachers replying to Sean Harford’s questionnaire by saying they tell their staff to ignore Ofsted. Ofsted could write a best practice review of all of those schools that ignore them. That would be great, because lots more schools would read Ofsted’s how-to guide to ignoring Ofsted. Spielman might pull her hair out at that unintended consequence. One to watch.