Piloting the Research Supervision Recognition Programme

The programme pilot involved:

• A set of statements of good supervisory practice.
• Written applications by supervisors demonstrating the alignment of their practice to the statements.
• Review by a panel of reviewers drawn from the UKCGE Executive.

Statements of good practice were devised on the basis of the literature on​‘effective supervision’ and following the model of the generic UK Professional Standards Framework for teaching and supporting learning, and clustered into three dimensions, namely:

• Supervisory activities,
• Knowledge and understanding
• Principles

In all, there were 22 separate criteria. In order to demonstrate the alignment of their practice to these criteria, participants were asked to write a reflective account of up to 5000 words, addressing these criteria and giving at least one example of their own practice under each main heading.

13 institutions from England, Scotland & Wales agreed to participate in the pilot project. An HEI in Northern Ireland was approached to participate but did not respond.

The participating HEIs were:

• Bournemouth University
• Coventry University
• Liverpool John Moores University
• Sheffield Hallam University
• Swansea University
• University College London
• University of Brighton
• University of East Anglia
• University of Edinburgh
• University of Glasgow
• University of Manchester
• University of Wales Trinity Saint David
• University of York

The pilot formally began on 1st March 2019 when the Good Supervisory Practice Framework, a Guide for Applicants, and a sample application were sent to the participating institutions for distribution to supervisors.

This was followed up on March 8th 2019 with an hour-long webinar for participants in which they were taken through the criteria in detail with suggested examples.

Participants were given until the 3rd May 2019 to write and submit their applications.

43 of the original 52 participants in the pilot institutions completed their applications, a response rate of 83%.

Applications in the pilot were reviewed by volunteers drawn from the UKCGE Executive Committee plus Professor Stan Taylor, with each application being assessed by two reviewers. Reviewers were given a guide to reviewing the applications and a standard recording template.

The overall standard of the submissions was extremely high and in nearly all cases the reviewers agreed that they fully met the criteria for recognition. In the handful where reviewers agreed that the criteria had not been met, this was for two main reasons.

The first was a failure to engage with the literature on research supervision; a few accounts of practice contained no reference at all to the literature and hence were unable to demonstrate that their practice was grounded in the scholarship of supervision.

The second was a focus upon describing institutional policies and procedures rather than upon giving personal examples of practice which meant that a judgment could not be formed on the practice of the individual in relation to the criteria.

Verbal feedback on the review process from two reviewers was positive. Most felt that the process had gone relatively smoothly, although it was perhaps unnecessarily complex in view of the number of criteria and unpacking evidence relating to those for knowledge and understanding and professional values within the areas of activity.

34 of the participants in the pilot completed an evaluation questionnaire, although not all responded to every question. The key findings were:

• Criteria — Participants were asked whether the criteria represented good practice in research supervision; 91% agreed that they did while 9% thought that they did partially (N=33). Of those who added explanatory comments (N=20), the vast majority were supportive but several felt that it would be helpful to consolidate the statements into a smaller number.
• Clarity of Guidance Notes — 48% of the participants thought that the notes provided were very easy to follow, 45% that they were fairly easy, and 6% that they were difficult (N=33).
• Participant comments (N=27) were overall highly supportive but some asked for greater clarity in the amount of evidence required and how far there was a need to demonstrate engagement with the scholarly literature.
• Ease of Application — 6% of participants felt that it had been very easy to produce the application, 72% that it had been fairly easy, and 22% that it had been fairly difficult (N=32).

Of those who offered comments about how it could be improved (N=30), the main issues were the number of criteria and the complexity of interweaving knowledge and understanding and the professional values into the areas of activity.

All of the 32 participants who responded said that completion of the structured self-reflection had been beneficial to their supervisory practice.

89% of participants thought that completing the application would improve their research supervision in future, while 12% were not sure (N=33).

All of those answering the question added their comments, nearly all stressed the value of reflection in itself and a large number also mentioned the usefulness of being exposed to and following through the scholarly literature on supervision.

In addition to the pilot, the UKCGE conducted a sector-wide Community Consultation on the proposed scheme between the 6th – 31st May 2019.

Members of the postgraduate community were invited to download the criteria and answer questions relating to the criteria, the potential benefits of supervisory recognition, and about any other comments they had on the proposed scheme.

In all, the consultation attracted a wide range of, in all, 116 responses although not all answered all of the questions.

Criteria

Respondents were asked how far the criteria reflected good supervisory practice:

• 43% said​‘very well’,
• 49%​‘well’,
• 5%​‘not well’ and
• 4%​‘not well at all’

(N=108).

Value of Recognition

As with the pilot participants, respondents to the community survey were asked how beneficial it would be to have their practice recognised in a range of aspects:

• 75% of respondents saw recognition as​‘very’ or​‘beneficial’ to their professional development,
• 73% in the advancement of their careers,
• 64% in attracting candidates,
• 62% in supporting funding applications.

Other Comments

Finally, respondents were asked an open question about if they had any other comments about the proposed recognition programme. The comments of the 75 who gave useable responses fell into three clusters:

• 25 (33%) expressed strong support for the proposed recognition scheme
• 21 (28%) offered conditional support
• 29 (39%) were opposed to it.

Among those expressing strong support, the proposed scheme was widely welcomed as offering a long-delayed opportunity for supervisors to be recognised for their professionalism.

Among those expressing conditional support, the conditions included simplifying the scheme and making applications less onerous and adding specific criteria, for example in relation to mental health and advising on non-academic career development.

Among those who were opposed, two main reasons were given. The first was a concern about possible duplication with existing recognition schemes, and both those associated with the HEA and the Staff and Educational Development Association (SEDA) were mentioned in this context. In all, 7 respondents (11% of the total) were opposed for this reason. The remaining 22 (28% of the total) rejected the whole proposal, in most cases on the ground that it was designed to generate metrics for use by institutions to measure and manage supervisory performance.

In response to both pilot and the community consultation the UKCGE, under the guidance of Professor Stan Taylor, updated the Research Supervision Recognition Programme.

Simplified Structure

The 22-attributes across the three dimensions of good supervisory practice were condensed to just a single set of 10 criteria. This simplified both the process of completing the structured self-reflection and the reviewing of applications.

Enhanced Criteria

The ten criteria were also enhanced to include references to supporting the personal wellbeing of candidates, and advising candidates on non-academic careers.

By the Sector, for the Sector

The UKCGE has made a commitment to make the Good Practice Framework and related resources freely available, and will continue to review and evaluate the Programme based on feedback from the postgraduate research community.

Following advice from Dr Elizabeth Adams and Dr Kay Guccione about extending the Research Supervision Recognition Programme to support early career research supervisors, and those involved in supervision ​‘informally’, the UKCGE undertook a small-scale pilot in 2021. 

The pilot demonstrated a need for an ​“Associate” award, proved that the Good Supervisory Practice Framework could be used at a different career stage, and that our reviewing procedures could meet the requirements of the new award.