Observing without grades: homework done

For good or for ill, my school has still been grading lessons when carrying out formal observations. Of course, we have been aware of what Ofsted have been saying on the matter, and also of what a number of campaign-bloggers have been arguing. Through much of the past 12 months, I and like-minded colleagues such as @Bterziyski have been conducting a quiet insurgency, devising toolkits to peer-observe without grades, messing with the minds of some of our teachers as we blankly refuse to use numbers or Ofsted dog-whistle words.

But so far we have failed to turn the whole ship around. My headteacher – for very sound reasons – wants to be able to point to a list and say, Here are my best teachers, or Here are the ones we need to work on. She absolutely understands that observation can and should be a development activity, but she needs it also be a tracking device. Before half-term I made my latest bid to SLT to consider the merits of LONG (Lesson Observation – Non Graded). After an amiable chat, I was set a homework. Below is my progress so far.

Appraisal at Our School

Appraisal must meet the needs of our school. It should also respect the reasonable expectations of teachers (as represented by their unions), whilst at the same time being mindful of government policy on Teaching Standards and Performance Related Pay.

Consequently, the Appraisal system at our school attempts to:

  • Develop the quality of teaching
  • Through this, improve the outcomes of students
  • Inform the PD plan
  • Provide evidence for the Headteacher to make decisions on staffing and PRP.

How we do this currently

Each teacher is assigned an appraiser. In many cases this is their HoF, or other immediate line manager; in all cases it is a colleague who works closely with them. At the beginning of the Autumn term, the teacher and appraiser together review the objectives set the previous year, using evidence from the teacher’s portfolio (CCPDP) – exam review, formal lesson observations and other evidence the teacher has chosen to present. The appraiser declares that the teacher has met the Teachers’ Standards (or not). The Head decides on PRP. Together the teacher and appraiser set 3 further objectives for the coming year, reviewing them on an interim basis in the Spring. They agree on the date and focus of an ‘appraisal lesson’, which is graded according to the school’s summative criteria. In the course of other internal reviews across the year, the teacher may be formally observed (and graded) a further two times. The Head maintains a spreadsheet on everyone’s graded lessons. The AHT (staff development) reviews all objectives, to discern common themes that can be addressed in the school PD plan.

Lesson Observation

“Ofsted does not award a grade for the quality of teaching for any individual lessons visited and it does not grade individual lessons. It does not expect schools to use the Ofsted evaluation schedule to grade teaching or individual lessons.” ( Ofsted clarification for schools, October 2014)

At our school, we have continued to grade individual lessons, using a template we devised in November 2011 to match the Ofsted inspection criteria.

Some numbers:

At our school, a mainscale teacher, teaching 25 lessons a week, will teach nearly 1000 lessons across the year; put another way, they will teach lessons lasting 50,000 minutes. Agreed with unions, we grade 1 appraisal lesson of 50 minutes, plus up to 2 more lasting 20-50 minutes. We say, on the evidence of this amount of observed time, we can grade the quality of individual teachers. Why might we be wrong?

  • Aside from teaching the classes, teachers spend an uncounted amount of time planning and marking, running after-school revision, intervention and other clubs. None of that is observed, and arguably none of it contributes to the teacher’s grade.
  • We know that ‘high-stakes’, ‘one-off’ lesson grading systems lead to cultures which are about proving, rather than improving; they encourage performance, not learning (Chris Watkins, IOE).
  • Teachers play it safe, covering ground they know their students are comfortable with.
  • Grading is unreliable. It is notoriously difficult to train observers so that they will agree on a grade.
  • Because graded lessons are poor predictors of student outcomes.
  • Because 63% of all lesson judgements are wrong (Strong et al, Journal of Teacher Education, 2011.)
  • If a lesson is graded ‘inadequate’ there is a 90% chance a second observer will grade it differently (Measures of Effective Teaching (MET) project, Bill & Melinda Gates Foundation, metproject.org)

Thanks here to @shaun_allison and others at the wonderful Class Teaching

However…

“Ofsted will usually expect to see evidence of the monitoring of teaching and learning and its link to teachers’ performance management and the Teachers’ Standards, but this should be the information that the school uses routinely and not additional evidence generated for inspection.” (Ofsted inspections – clarification for schools)

So we do need to have robust means of capturing the quality of teaching at our school.

More numbers:

Assume each teacher is formally observed for a total of 100 minutes, and receives feedback lasting the same amount. As this involves 2 (in fact sometimes 3) people, we can say that this consumes 400 minutes. If we say this happens to 80 members of staff each year, this amounts to a total of 533.3 hours per year. We could do a lot with that time. We might say that this was time well spent if:

  • It improved outcomes for students
  • It improved the practice of the observed teacher
  • It improved the practice of the observing teacher
  • It provided reliable evidence of the quality of teaching and learning across the school
  • It identified common development needs that could then be addressed by the in-house PD programme
  • It reliably identified weak teachers and strong ones, so we could perhaps know what we needed to do next and how
  • It contributed to staff wellbeing, because it was an element of staff development rather than merely appraisal
  • It was about Quality Improvement, not just Quality Assurance (Joanne Miles) The role of mindset in the move from graded to ungraded lesson observation cycles

 

 

How might we capture great teaching that leads to great learning?

“A formative teacher evaluation system – based on a continuous assessment and feedback rather than a high-stakes test – must incorporate a range of measures from different sources, using a variety of methods.” (Developing Teachers: Improving professional development for teachers, January 2015. The Sutton Trust.) Link here. It is crucial to avoid over-reliance on a single source of evidence (such as a graded lesson), as there can be “a lot of noise around a weak signal.”

The Sutton Trust report (compiled with the Bill and Melinda Gates Foundation from 80 leading teachers from across 6 Anglophone countries) identifies 6 approaches to teacher assessment. None of these should be relied upon in isolation; validity arises from their triangulation.

Moderate validity:

  • classroom observations by peers, principals or external evaluators
  • ‘value-added’ models (assessing gains in student achievement)
  • student ratings

More limited evidence:

  • principal (or headteacher) judgement
  • teacher self-reports
  • analysis of classroom artefacts and teacher portfolios

Our appraisal procedures may already bear elements of these approaches. We have:

  • Observations (graded for appraisal and internal review; peer observation for development purposes.)
  • Exam review (for individual teachers, using residuals.)
  • Headteacher judgement (based on provided evidence, for PRP)
  • Teacher self-reports and portfolios (in the appraisal booklet and CCPDP.)

I propose two changes.

  1. A new lesson observation protocol Great Teaching and Learning Toolkit
  2. Adapt our teacher portfolio, now to be the C…. Professional Development Portfolio (CPDP) CPDP Grid

Each change is designed so that we better identify the strengths and development needs of our teachers so that: they can benefit from a more strategic in-house PD plan; the Head can make staffing and PRP decisions based on more valid evidence; the overall quality of teaching improves and leads to better outcomes for students.

Lesson Observation Great Teaching and Learning Toolkit

“Successful teacher observations are primarily used as a formative process – framed as a development tool creating reflective and self-directed teacher learners as opposed to a high stakes evaluation or appraisal.” (Developing Teachers: Improving professional development for teachers, January 2015. The Sutton Trust. My italics.) The report claims that observation can be effective when it is collaborative and collegial; however, the research also concludes that challenge is needed in the process, so may involve school leaders or external experts. Overall, it emphasises the requirement to have strong observation protocols, including observers who are trained.

The new protocol has two parts. The first, Dialogue and Development, will be completed by the teacher and their observer. It records what happened, what went well and what could have been better. Crucially, it requires that next steps be identified, with some plan for their implementation. Therefore, this will not be a ‘cosy chat’; it is joint practice development (JPD): it will require two professional colleagues to think deeply about what took place in the lesson and how to develop their practice. They can be assisted in this by the second part of the protocol, the C…. Great Teaching and Learning Toolkit. When a colleague asks us what great teaching looks or feels like, it is fair that we should have some idea. Everything here has been derived from the advances in pedagogy we have made here in the last few years, from research conducted by colleagues and from the peer observation toolkits trialled by Development Coordinators. It absolutely is NOT a checklist against which to measure a single lesson, as it understands that much learning in not ‘visible’ and rather takes place over time.

This is a protocol which can be used formally (for appraisal and review purposes) and informally (peer observation between colleagues). There is no space for a grade; I believe that words such as ‘outstanding’ or ‘inadequate’ are also best avoided, as they tend to crowd out the more meaningful dialogue around development.

Teacher Portfolio CPDP Grid

We have a long tradition at our school of maintaining a portfolio and presenting it to the Head when making a threshold application. Though insufficiently robust as a single source, taken with lesson observations and exam review it may be a valid measure of the quality of teaching.

Why we should change the current portfolio

  • It is largely ignored by teachers not applying for UPR.
  • It inadequately reflects the degree to which a teacher is meeting the Standards at the level appropriate for that point in their career.
  • It is a compilation of achievements, or ‘jobs done’, but is not a reflection on future development needs;
  • It therefore fails to inform appraisal objectives for individual teachers; and
  • Taken together, they cannot be read for the purposes of PD planning for the school.

I therefore propose a C…. Professional Development Portfolio (CPDP), arranged around the 8 Teachers’ Standards, the wording of which is adapted to refer to the C…. Great Teaching and Learning Toolkit. Two further tabs have been added (on engagement in PD and on collaboration) in order to encourage JPD as a form of teacher development.

The CPDP Grid will act as the contents page of the portfolio. It will be completed as part of appraisal (in reality, mainly before the appraisal meeting) and provide the ‘baseline’ for the appraisal objectives to be set, and the ‘evidence of outcomes’ for the previous year’s objectives. [The Appraisal Booklet will be adapted to reflect this. Alternatively, the grid may simply appear as the first pages in the booklet.] In this way, the appraiser and teacher will have a clearer view of strengths and areas for development; the Head will be able to say, with more security, which teachers need more targeted support and which should gain PRP; and the AHT will be able to identify common themes and address these through the PD programme.

Next stages

Assuming there is agreement on the principles of this, we need to consult widely with students and staff to truly ascertain what we feel constitutes ‘great’ teaching and learning at our school. We also need to do serious work on observers and teachers, so that neither expects a grade (or codeword) to emerge from an observation. Then we need to take a deep breath, and watch as the whole thing unfolds.

Advertisements

6 thoughts on “Observing without grades: homework done

    • Thanks, Abi. Yes the bravest thing an SLT can do is to trust its staff – bizarre, isn’t it? But they naturally don’t like to loosen the reins when they need levers to improve teaching. My point is simple: work with staff to improve their practice, rather than spend time gathering dubious data.

  1. Pingback: Better than Outstanding | markquinn1968

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s