Executive Summary

0.1 Introduction

1. Descriptive data about the processes of British postgraduate medical education is sparse. Outcomes evidence is limited to pass rates for the Royal Colleges examinations and the individual judgements of assessment panels for GP registrars and completion of CCST.

2. Postgraduate trainees are employed as working health professionals, so their current competence is always an issue, as well as the competence they will be expected to demonstrate when their specialist training has been completed. Defining competence in terms of the expectations of the holder of a particular post helps to avoid confusion.

3. The central problem of postgraduate education is how best to combine work within a doctor's current competence, itself a source of learning, with the provision and use of learning opportunities to extend that competence.

4. There is a need for an agreed framework for the development of competence during each training programme, which puts learning outcomes on the priority list alongside service duties.

5. Areas in which competence is seriously underconceptualised include communications, teamwork and management in healthcare settings.

6. The recent report of the US Federated Council for Internal Medicine Task Force, Graduate Education in Internal Medicine: a Resource Guide to Curriculum Development (Sox et al., 1997) is a useful source of ideas.

7. Some perceive judgement as an attribute of personal expertise that goes beyond that competence which any fully trained doctor could be reliably expected to demonstrate. It can also be seen as a dimension of lifelong learning linked mainly to the improvement of decision-making through learning from experience over a long period, rather than the learning of new practices or keeping up to date with research.

0.2 Research into Medical Expertise and Decision-Making

1. Key features of expertise include the importance of case-based experience, the rapid retrieval of information from memory attributable to its superior organisation, the development of standard patterns of reasoning and problem-solving, quick recognition of which approach to use and when, awareness of bias and fallibility; and the ability to track down, evaluate and use evidence from research and case-specific data. Understanding the nature of expertise is important for self-monitoring one's use of heuristics and possible bias, sharing knowledge with others and supporting other people's learning. It is also critical for understanding the respective roles of clinical experience and evidence-based guidelines.

2. Research into decision-making under conditions of stress and uncertainty suggests that training in crisis management is needed, and that teamwork and other organisational factors are important.

3. There is a need for regular self-evaluation to maintain critical control of one's practice.

4. The use of evidence-based medicine requires on-the-job as well as off-the-job teaching.

0.3 Learning in Clinical Settings

1. Many features of the educational policy seem to be appropriate, but they are not being implemented in many hospitals. There is insufficient supervision and feedback. Educational goals are subordinated to service demands. While many house officers receive good clinical teaching, a minority do not and assurance of educational quality is weak. Learning goals are only specified at a very general level, so there is little clarity about priorities, especially at the PRHO stage.

2. This issue has to be tackled at local level where there is limited management of the educational process and clinical tutors have little time and no authority over clinical teaching. Deans do what they can; but quality assurance of postgraduate education lags well behind that for clinical practice; and the UK research base at this level is minuscule.

3. The learning of procedures in medical posts has been criticised for being too haphazard: there is often little continuity of experience and guidance is often provided by doctors who are themselves not very experienced.

4. The appropriateness for GPs of so much general hospital training has been questioned. Though research on this issue would be difficult, we think more research evidence could and should be gathered.

5. The key issues emerging from North American research are:
   • the time allocated for trainer-trainee interaction and making the best use of that time;
   • methods of providing feedback;
   • qualities of teachers rated as `good' by trainees and their colleagues.

6. One gets the impression that American residents receive considerably more clinical teaching than their British counterparts, but there is no British data to enable a proper comparison. The variation in the amount of training received by British trainees is reported as considerable, raising issues of quality assurance and trainee entitlement.

0.4 Learning in Non-Clinical Settings

1. Research into postgraduate teaching and learning in non-clinical settings mostly comprises evaluations of a wide variety of teaching innovations, rich in ideas but not necessarily generalisable.

2. Strong evidence that the effectiveness of off-the-job teaching is highly dependent on its links with related on-the-job teaching makes it unwise to evaluate off-the-job teaching on its own.

0.5 Continuing Medical Education and Lifelong Learning

1. Surveys of GPs, and also in a few cases consultants, have shown the importance for learning and changes in practice of a wide range of learning activities and sources of information. Moreover, they differ according to whether the changes involve treatment (including prescription), diagnosis and investigation, doctor-patient relationships, referral policy, health promotion or practice organisation. Models of physician learning distinguish between learning triggered by the problems raised by current individual patients and ``learning projects'' to acquire or improve proficiency in a targeted area of practice. The initiation of learning is dependent on significant background knowledge of what is out there to be learned to which CME conversations with other physicians, and reading contribute in ways which would not be revealed, for example, by evaluations of CME events. The importance of informal consultations with others and a reluctance to ``cold call'' experts suggest that facilitating social interaction among doctors and strengthening their networks should be a policy goal.

2. Evaluations of CME courses have demonstrated the importance of including activities such as the observation and discussion of visual material and/or supervised practical work. Though it has confirmed that short courses of 1 day or less are rarely effective, no controlled studies have been reported which used length of course as a variable. This deficiency needs to be remedied because much time could be wasted trying to improve courses which are too short; and unrealistic expectations of the learning time required for certain goals are easily developed by busy learners and under-resourced providers -- a form of collusion from which nobody benefits. Another important conclusion is that educational interventions on their own often fail to achieve changes in practice.

3. Research on innovation strategies points to the danger of focusing only on the development of competence. Competence has to be translated into performance and at this stage many dispositional and organisational factors come into play. Research on the implementation of guidelines, for example, indicates not only that the quality and utility of the guidelines themselves is important but also that both educational interventions (leading to understanding of their purpose and rationale) and administrative interventions (ranging from organisational changes to simple reminders) need to accompany the guidelines.

4. The discussion of recent developments in CPD reaches two conclusions. First, needs analysis is important for quality assurance purposes at three levels - the individual, the working group and the healthcare organisation (the last two are multi-professional). However, it should not be assumed that needs identified by audit, for example, will necessarily require an educational response. Second, following the advice of Fox and Bennett (1998), CME providers should adopt a coordinated approach to all three levels by facilitating self-directed learning, providing high quality individual and group education, and assisting healthcare organisations to develop and practise organisational learning.

0.6 The Role of Information Technology

1. Decision support systems have had a mixed reception over the years, but seem to be growing in acceptance as just another tool in the doctor's armoury. A training issue here is the need to ensure that doctors understand how decision support systems frame the problem so that they can judge the quality of the advice that they offer.

2. In terms of the development of such systems, we may expect the development of linked databases rather than fragmented sources and better explanations from systems about the reasons for their decision advice.

3. The training implication of this is that such systems may be excellent, but their use needs to be carefully integrated into the overall training programme -- not least so that the human trainers provide what the computer-based system cannot provide in terms of monitoring and feedback.

0.7 Assessment and Feedback

1. Summative assessment for the award of the Certificate of Completion of Specialist Training will be at least partly performance-based; and will need to be evaluated as it comes on line. Since Membership Examinations of the Royal Colleges are competence-based, the performance-based element at the end of Basic training is given relatively little attention. Thus the most critical certification issue arising from our review is the extent to which assessment regimes cover the full range of competence discussed in Chapter 3 and its translation into performance.

2. With formative assessment questions have been raised about frequency, coverage, and reliability; and, if it is to properly serve its purpose, the manner in which formative assessment is integrated into training programmes to support the learning process will also need to be researched. Levels of supervision are often affected by factors other than the competence level of the trainee, and feedback may not be based on any systematic (though not necessarily formal) assessment.

3. Learning to assess and reassess one's own competence and its limits is a long and complex process, which becomes increasingly sophisticated as a doctor progresses through postgraduate education. Its reliability significantly depends on access to good supervision and feedback. Feedback which contributes to a trainee doctor's self-assessment may come from patient outcomes, informal discussions with other doctors or health professionals, periodic appraisals or meetings for signing the trainee's logbook. Informal feedback on the wards tends to be spontaneous and incidental, i.e. not the result of a reflective judgement: it is mostly received from more senior trainees rather than consultants, and more likely to be negative than positive. Formal feedback appears to vary considerably in quantity, quality and breadth of coverage. Even the best designed log-books focus on competence rather than performance.

0.8 Monitoring Postgraduate Education

1. As reported in Chapter 4, there is sufficient cause for concern to suggest that research into the practice and effectiveness of supervision and feedback during postgraduate medical education is urgently needed. This should include the implicit delegation of certain supervisory and educational responsibilities to senior trainees. Should it be formalised, as in the US role of Chief Resident? Should it be accompanied by training? Which responsibilities could or should be delegated, and which should not?

2. Few Trusts have reliable mechanisms for incorporating clinical teaching into their organisation of professional time, though some may have formally agreed to do so. Nor are there any internal audit mechanisms at local level for monitoring and periodically evaluating a Trust's programme of professional education. Clinical Tutors have neither the time nor the authority to undertake such duties.

0.9 Mentoring after Qualification

1. Given the difficulties discussed above of finding sufficient time for supervision, giving feedback and clinical teaching for postgraduate medical trainees, the introduction of mentoring as an additional role and obligation might not justify a high priority. However, mentoring might be particularly well suited to the support of doctors during the first few years after completion of postgraduate training. Both consultants and GPs could benefit from such support as they grow into their new roles and responsibilities, learn to work with new colleagues and to contribute appropriately to their Trust or General Practice, and take greater responsibility for organising their own lifelong learning.