Headlines are poor proxies for research.

So here goes. The other day Laura McInerney  at ResearchEd Blackpool posted on Twitter this much-repeated list of ‘Poor Proxies for Learning’, asking How many of these are you guilty of thinking will equal learning?

I replied

I have disliked this list for some time. A teacher who achieves these in their class IS doing a fine job. And, if not individually, in combination these will be good proxies for learning. Leaders should not be criticised for wanting to see them.

You can see some of the replies and thoughts on https://platform.twitter.com/widgets.js“>proxies for learning here. Of course, I understand Rob Coe’s point that busy or engaged or motivated students, or ordered classrooms, or ones where the course is covered are not the same thing as learning, but I worry about the reactions of some teachers and school leaders seeing this list.

  • It sneers at teachers who strive to achieve exactly the things which are on this list.
  • It implies that these conditions are not worth striving for.
  • It may lead some teachers and school leaders to downgrade the achievement of these conditions.
  • It encourages the despairing conclusion that we cannot ever know that learning has taken place until public exams results are in – and that only those matter.

I know I have substituted the word ‘conditions’ for ‘proxies’. I know that this means I am guilty of a category error, where I mistake the difference between a prerequisite for a thing and the thing itself. But consider the position of the teacher teaching a class, or a school leader watching the lesson. When I was teaching (I remember it well – it was not long ago), I would have been delighted if I ever managed to cover the full content of a curriculum, when that curriculum kept changing and growing. If I ever motivated all of my students so that they were interested in what I was saying and engaged in what I wanted them to do, I would have chalked that up as a stunning success. There were very many times when my classroom was not ordered or calm, so I was definitely happy when it was. More to the point, I know that my students did learn better when they were engaged in busy, well-ordered lessons. It took me several hard years to achieve lessons like those on a consistent basis. And, when it became my role to help other teachers, it took much energy on my part (and much more on theirs) to achieve something like those conditions in their lessons.

School leaders should strive for those conditions in their schools. I mean, imagine if they did not. I know that it is not the intention of those who produce and reproduce the proxies list to suggest that they are (all) undesirable, but I do fear that that is how they are interpreted.

Those who publish research on well-visited platforms have a duty to consider the consequences of misinterpretation, and to at least try not to cause it. Some recent EEF reports have been headlined in a way that I feel is irresponsible. For example, what is a school leader to make of this?

New EEF trial results: ‘light-touch’ approaches to research unlikely to impact pupil outcomes

Will they see the ‘light touch’ and think, erm… that’s exactly what I do? For most schools, to be engaged in research and to seek to be informed by research often is a light-touch exercise. They trawl through their twitter feed late at night, look for some promising blogs to read, maybe look to make something more of the connections they have with a local university. Surely to goodness, these are activities to be encouraged. It is not encouraging to learn from an EEF sub-editor that it is unlikely to impact pupil outcomes. Worse still, it might lead some leaders to conclude that that journal club they have running is a waste of time. They should have read the report, of course, and added their own emphasis (as I have here):

While the evaluators from the University of Bristol found no evidence that the programme led to improvements in reading outcomes for ten and eleven year olds, the findings suggests that there may be a relationship between how engaged teachers are with research, and the attainment of their pupils. There was also some evidence that being in a Research Learning Community increased teachers’ engagement with research.

And perhaps they ought also to have read EEF Senior Researcher Jonathan Sharples’ blog . In it, he concludes that the light-touch research approaches described in the report were indeed worthwhile, and he references the 2016 EPPI review.

Importantly, in addition to reviewing different mechanisms to mobilise evidence, they also looked at the behavioural requirements that were necessary for those various approaches to have an impact. This included having:

  1. opportunities to engage with the interventions
  2. the motivations to do so, and
  3. the skills and capabilities to understand and use the outputs.

Dig down far enough into the EEF website and there is everything there the teacher and leader needs to be well-informed. Bravo. However, how many teachers and leaders crack their spade on the hard crust of the top line?

digging_hole001

Headlines are poor proxies for research.

 

 

Advertisements

#Lower case r, upper case ED, 17

ResearchEd17 could be forgiven for being a bit self-conscious: in recent weeks it has been spoken of less favourably, had its grassroots raked over, its biases heat-mapped. Sure enough, when I arrived (like a marathon runner along Cheering Lane), it was clear that the cheering crowds had stayed away.

total gridlock ‘Total Gridlock’

There was barely a complimentary canvas bag in sight. Was TB defeatED? Was his love affair with geeks in the staffroom and policy wonkers in the anglosphere endED? Had his hash been tagged for the last time?

Nope.

It turns out that Chobham Academy, in the shadow of the Queen Elizabeth stadium, is larger than the average comp: they hosted the 2012 volleyball in their foyer – which is where we found the hordes and their canvas bags. Tom was in pink, Helene was handing out raffle tickets to win lunches, and all was right with the world. Someone (we blamed the Harris peeps) forgot the free pens, but we are high starters with 21st Century skills happy to photograb speakers’ slides and live-tweet our research. Our only concern was failing phone-battery power. #WorkingOutHowToMakeADyingPhoneWork.

I have engaged with ResearchED on many previous occasions, but this was the first time I got to engage in ResearchED. If you have never done it, you should, it’s miles better than sitting on the floor all day, and you get to rub shoulders with famous people and Nick Gibb. And you get to start the day in the speakers’ lounge, this year styled as the training room. I limbered up there with my fellow presenter James Mannion and teachers from the City of London School.

SpeakersRoom

Imagine if all the boxers from the undercard were put in the same dressing room: it’s just like that. Your competitors (those you suspect will draw a bigger crowd) are in there, as are real live people who misleadingly look nothing like their Gravatar. I’m represented to the online world by a three year old drawing by my daughter, who at the time loved me enough to ignore my greying hair. In short, there is little small talk in the speakers’ room; just people doing some research before it’s their turn to present.

You would get fit working in this school. It’s designed like one of those spiralling coin boxes that entertain you as you give your old euros to charity. I joined Lisa Pettifer for a lap on the second floor. ‘Is M213 this way, Lisa?’ ‘Just keep on walking, Mark, and it soon will be.’ Jonny Peacock and Christine Ohuruogu go to school here. Sensible presenters like Christian Bokhove wear t-shirts with penguins on them. He told us that spinach does not contain lodes of iron, that the myth apparently occurred during the Great Decimal Point shift, that that too is a myth traceable to the Readers’ Digest, which may or may not be available in the Netherlands. And the moral of his tale was: don’t pretend that you know stuff really well unless you really do; try a little nuance when discussing cognitive psychology on twitter. I will try my best, Christian, but it’s in my nature…

‘Mark Quinn et al’ were giving their talk on practitioner enquiry during session 3. Six of us at the front, going for the prize of Those Most Likely to Outnumber Their Audience. Tom helped us out by scheduling against us Sherrington, Christodoulou, Weston, Jones, Creaby, Davenport, and Hood and Fletcher-Wood… et al. Well, I don’t know how many flocked to the gurus this year but we were very happy with our little turnout. Everyone had a seat, they could join in on the chat, and I could pick out old colleagues Barbara Terziyski and Vivienne Porritt.

ResearchEd17 There are more people just out of shot.

We were making the case for the gnomes of the research garden, teachers carrying out the sort of micro-research that tests out the grand theories without ever being reported. Nick wanted to know if his year 8 had a growth mindset, and if they did did it show up in achievement and effort data. (They did, and it did not.) Joe, the head of RE, has an ontological interest in creativity: he wanted to know what his students thought about it and where they would like to see more of it. Richard wanted his year 11s to be more reflective about their work, had a hunch that peer feedback would help him get there, and found that it did. The great thing about ResearchED is that it showcases some of the disciplined enquiries that real teachers are conducting, but even if ResearchED did not exist these teachers would still be gnomically enquiring away.

Amanda Spielman finished my day. That’s great because she is passionate about workload, so much so that Ofsted will ask headteachers how they are reducing it. She is also passionate about research and will turn the inspectorate’s attention that way increasingly. I asked her if she would research the impact Ofsted have on workload, and act on the results. I can imagine headteachers replying to Sean Harford’s questionnaire by saying they tell their staff to ignore Ofsted. Ofsted could write a best practice review of all of those schools that ignore them. That would be great, because lots more schools would read Ofsted’s how-to guide to ignoring Ofsted. Spielman might pull her hair out at that unintended consequence. One to watch.