We undertake research related to paediatric education and training, from an educational perspective. This work is undertaken by members of the team often in collaboration with RCPCH officers, exam leads, paediatric consultants and paediatric trainees.
The Psychometric sub-team and Quality and Standards Manager oversee an active programme of research, generating abstracts for posters or presentations at national and international conference and writing papers for publication in scientific journals. While research activity is continuous, it must be noted that operational activities are the primary focus of the team.
The majority of our research links to:
- theory examinations
- MRCPCH Clinical and DCH Clinical examinations
- Assessments (such as Supervised Learning Events and RCPCHStart) along with the recruitment, revalidation and the paediatric curriculum.
A selection of published (poster, presentation or paper) research can be found in this table and the accompanying files can be downloaded from the download box at the bottom of the page.
| Title | Summary | Publication or Conference | Author(s) | Year | Findings |
|---|---|---|---|---|---|
| 1. Criterion Referencing Judges: Who are the best predictors? | Angoff judgements were reviewed to determine whether there was any difference in the predictive ability of Senior Examiners and Post-Membership Trainees (SpRs) with regard to Candidate's exam performance |
ASME abstracts, 2008 (Presentation) | Kumar P, Dinwiddie R, Foard (nee Davis) L, Muir-Davies A, Muir G, Newell SJ | 2008 | The data strongly supports our policy of including trainees as judges in the Criterion Referencing of these high stakes postgraduate examinations |
| 2. Candidate Perspective of Post-Examination Feedback | New exam feedback was provided for the Part Two (now AKP) written examination; feedback was broken down by competency and subject area. Candidates views were sought on this new type of feedback | ASME 2008 (Poster) | Muir G, Foard (nee Davis) L, Davies-Muir A, Newell SJ | 2008 | Candidates found the dual format feedback useful and more directive and future feedback was/is provided in this way. |
| 3. A postgraduate Diploma in Child Health; do examiners and candidates agree on what it assesses? | The research examined views of candidates and examiners on whether the Postgraduate Diploma in Child Health (DCH) was a good assessment of clinical skills | ASME abstracts 2008 (Presentation) | Reece A, Foard (nee Davis) L, Todd P, Bellman M, Newell SJ. | 2008 | The DCH is a popular exam (and is expanding to include overseas centres) that is felt to be a good assessment of doctors paediatric clinical skills |
| 4. Candidate Appeals in a high stakes postgraduate examination | Appeals within the MRCPCH Clinical Examination were evaluated over three years (2005-2007). The number of appeals were noted to have increased over time and were spread across the clinical stations. The majority of appeals were raised by candidates who were just short of the passmark | AMEE abstracts 2008 (Presentation) | Newell SJ, Lee L, Crane D, Foard (nee Davis) L. | 2008 | Confirmation that an open, transparent and fair system is in place for candidate appeals. The increase in appeals emphased the importance of detailed examiner documentation |
| 5. The effects of Examiner Performance Feedback | The data examines the effect of Examiner Performance Feedback (Hawk-Dove Index) on examiner marking. Whilst 95% of examiners found the feedback useful, it appeared to not have a significant effect on examiner marking | AMEE abstract 2009 (Poster) | Davies-Muir A, Foard (nee Davis) L, Newell S, Muir G | 2009 | The effect of Hawk-Dove feedback continues to be monitored |
| 6. Examiner Performance Feedback: Is it a useful tool for examiners? | With regard to the Hawk-Dove examiner performance feedback (PERFORCE), 95% of examiners agreed the tool was a good way of assessing their performance, and found the data was well set out and helpful to them | ASME abstract 2009 (Poster) | Foard (nee Davis) L, Davies-Muir A, Muir G | 2009 | The effect of Hawk-Dove feedback continues to be monitored |
| 7. “Cruel to be kind?” – is it beneficial for failing MRCPCH clinical examination candidates to be forced to delay their re-sit? | Candidates achieving low scores in the MRCPCH Clinical are deferred from re-sitting for a specified period and advised to seek further training and educational support. The exam is scored out of 120, with a pass mark of 100. Low scores can lead to deferment. To explore whether deferment has any positive effect, candidates' past exam scores from first attempt to re-sit were analysed |
Arch Dis Child 2012, 97, Issue Supplement 1 | Lee L, Newell SJ, Foard L et al | 2012 | Candidates sitting in overseas centres benefitted from the deferment process |
| 8. Feedback on doctors’ performance from parents and carers of children: a national pilot study | A study to evaluate the reliability and validity of a children's and carers’ feedback tool, to explore the feasibility of delivering this nationally and to determine acceptability to doctors of this assessment |
Arch Dis Child 2012, 97, 206-21 | McGraw M, Fellows S et al | 2012 | |
| 9. Improving postgraduate clinical assessment tools: The introduction of video recordings to assess decision making | The reliability of the paediatric postgraduate OSCE was calculated. An objective structured video examination was designed to assess candidates’ clinical decision making ability when managing acutely unwell children. Assessed the ability of the video station to discriminate between candidates taking the clinical examination for Membership of the RCPCH, and to assess the degree of agreement between scores obtained in the video station and other stations in the clinical exam. |
Medical Teacher, 2012, 34 (5), 404-410 | E.A. Webb, L. Foard (nee Davis), G. Muir, T. Lissauer, V. Nanduri, and S.J. Newell | 2012 | The video station provides an important method of assessment and its use should be explored in other postgraduate examinations |
| 10. The Impact and Causes of Rater Severity/Leniency in Appraisals without Post-Evaluation Communication Between Raters and Ratees | In performance appraisals, some assessors are substantially more lenient than others. However, this study investigates the impact or causes of rater severity versus leniency in situations in which there is little or no contact between raters and ratees after the performance evaluation. The studies conclude that the severity versus leniency effect probably has a considerable impact on performance ratings in ‘no-contact’ appraisal settings, but that neither rater personality nor rater cognitive ability appear to play a significant role in this. |
Journal reference only: International Journal of Selection and Assessment, September 2013, 21 (3), 286-293 |
Dewberry C, Davies-Muir A, Newell S. | 2013 | Examiner personality or cognitive ability do not have an impact on performance ratings given in no-contact ratings, whereas their severity versus leniency trait does |
| 11. Recruiting and Retaining Angoff Judges | The RCPCH's fifty two Angoff judges were surveyed on what motivates them | Paediatric Education Specialist Interest Group (PEdSIG), Cambridge 2016, Poster presentation | Carroll W, Foard L | 2016 | Reasons for paediatricians becoming judges and what motivates them are multi-faceted, however the opportunity for professional social interaction was central to both |
| 12. Viewing time by Question Type | The RCPCH theory exams are now administered by Computer-based testing. Data about candidate exam behaviour is available for analysis; we analysed the viewing time data for all exam questions with particular focus on question type | Paediatric Education Specialist Interest Group (PEdSIG), Cambridge 2016, Poster presentation | Carroll W, Foard L | 2016 | Candidates were spending a disproportinate amount of time on MTF questions; further adding to the rationale for their removal from our theory exams |
Downloads