The “Flaw of Averages” was I understand first observed when a trainee statistician drowned while fording a river which he calculated had an average depth of 2 feet.
You can tell he was American. A British trainee would have made the average depth 0.945m.
We are seeing 7,000 online consultations per week through askmyGP, from about 250,000 patients covered. At 2.8% of total list, that would be about a third of total demand on average, right?
You know I’m going to say “wrong” and the flaw explains very simply why.
It turns out the practices divide into three cohorts, which I’ll call the swimmers, runners and rollers.
The swimmers see under 5% of demand online. There may be a lot of splashing, but speed over the ground is quite low, benefits are hard to measure and an adverse current may even sweep them away. An askmyGP icon decorates their website, leaflets and posters are all over reception but for any patient using it, service is slow and frankly, the telephone seems a safer bet.
Runners are in the range 20 – 40% online, it’s a main mode of access with good service and generally high patient satisfaction. Benefits for the practice are significant and they can handle all patient demand on the day. But they are still running two systems, with mixed messages to patients. While they know online is more efficient, they may still limit access, so patients revert to telephone.
Rollers are putting 100% of demand through askmyGP, between 50 and 80% online from patients, the rest by telephone into reception. In this total flow mode, GP digital triage means they manage all their workflow with an efficiency simply impossible by any other means.
What does this mean for us?
We want to increase usage because, while we incur some volume related costs, it’s much more valuable to us when customers and patients get the most benefit from askmyGP, and that is after all our vision.
But while it might be tempting to persuade, cajole or incentivise practices to push up their average % usage by a few points, it would be a waste of everyone’s time.
Quite simply, we need to get them all rolling along in total flow mode. They’ll experience the benefits all for themselves with no pushing and shoving from us.
Here’s the thing: all our customers have exactly the same software, and the same advice. They are just making different choices, and seeing radically different outcomes as a result.
The great news is that all our new launches are rolling from day one. It works best that way.
Any triathletes will have spotted the analogy but there’s an added twist: transition between the modes seems to be remarkably difficult. People settle into a mode of operation and to shift seems just, well, a bit of an effort.
It’s not impossible, with Witley & Milford shifting three weeks ago and immediately doubling their speed of flow. After one week, someone briefly went back to the old system and was very quickly corrected!
So while new practices are all getting the max, how do we move the others?
PS What about the spectators? Very simply and with no commitment, you can be the GP managing the incoming total demand. It’s real, randomised and anonymised patient data, and when you have your login it will take about 15 minutes to rattle through 50.
Register for our free Digital Triage Experience.
Good news has a hard time getting heard.
This week we’ve seen new, comprehensive data from NHS Digital on the wait to see a GP, splashed across all the papers:
It’s all “true”, though I’m afraid the spin is not. The argument that “40% are seen the same day” rings hollow with anyone who has hung on the telephone for half an hour, to be told that all the same day slots have gone. Many of those 40% have been trying for several days just to get through.
Blame the patients, blame the government, blame whoever else we can think of. Or take a different look.
These tiny stories from the last few days are just a handful of the hundreds we see each week from patients grateful to their GP:
Your service and reliability are amazing. Thank you! (f 85)
Amazing fast system thank you (m 41)
Amazingly swift and very easy process than trying to jugggle around work – thank you so much! (f 24)
Amazing…More personal…Super speedy (parent, boy, 3)
Love how easy it is to speak to your own doctor . Amazing (parent, girl, 4)
Love this new system…so easy and quick , and have the problem solved without having to sit around at the surgery. (f 49)
Love ‘askmygp’. Making it so much easier to get info and solve problems whilst holding down a full time job! (f 40)
Wow. I am impressed! (m 69)
Wow, just wow. Have been in terrible painall night…absolute godsend… Thank you so much for your skills and innovations (f 65)
WOW, great system, quick easy, and no need to travel. Many thanks (m 63)
We’re now over 4,600 feedbacks from 80,000 episodes since August, and the trend is better and better. Please do have a look at the live rolling 7 day summary chart. We ask patients whether it’s better or worse and the ratio as I write has moved up to 9 which is so exciting.
These patients are getting an outstanding service from their own regular NHS GPs. The GPs have no extra funding (they pay us), and no complicated extended hours 8 to 8 hubs (that didn’t work)
Patients didn’t have to switch to an out of area GP. They could name their own GP. They were seen same day if needed.
And the GPs are happier too – happier professionally to be giving such a service and bringing the joy back into their working lives.
If you haven’t yet watched the Burnbrae video, please do and click for the demo at the end. This is one of her 5,000 patients last week, helped within 2 hours:
“fantastic service and Dr Arnott is an amazing doctor, Shotts is lucky to have all these new changes.”
How do we get this into the headlines?
Shotts, North Lanarkshire. Small town and surrounding areas, ex-mining community. Traditional, somewhat deprived area.
Dr Sue Arnott, full time single hander GP. Team consisting of 8 session ANP, practice nurse.
“We were looking to do things completely differently”
As well as hearing the story, you may like to see the data. Best in full screen, then slideshow.
Innovation is risky, and change is risky, so it should come as no surprise that we have failures. Perhaps we have more to learn from failure than success, which means we must reflect.
Since launching askmyGP v3 in August we’ve had three practices turn it off. I won’t name them, suffice to say they differ widely in size, demographic and location, but you all want to know why they gave up.
The common theme in what they told us was that they felt unable to cope with patient demand.
Yet patient demand was very close to predicted, within 10%, as it has been with the great majority of successful practices.
We do point out that unmet need is uncovered when limits are removed, which may appear to be a rise in demand in the early weeks. A small number of patients will abuse the new system, as they did the old one, but we’ve found from our Datalog audit that in the GPs’ view this is around 3% both before and after launch.
We don’t say it will be easy. We do say that with perseverance both patient service and GP working lives improve. But those that start from a strong patient service ethos seem to do best for their own working lives too.
The commitment to fast and appropriate response from the whole team puts them in control and minimises rework. That doesn’t mean saying yes to what every patient “wants” – that way lies madness. It does mean sufficient breaks around GP and staff needs. Indeed the day can be much more flexible, with many opting to work from home part of the time.
While failure means a return to the previous state with all its frustrations and stresses, success is a journey not an endpoint. It’s all about flow, measured continuously:
- Patient demand, by type, mode and timing over weeks, days and hours. This matters for designing the service.
- Elapsed time to complete requests. Usually the chief concern of patients.
- Continuity, where appropriate. Often key to patient satisfaction, GP job satisfaction and quality of care.
- Efficiency, a chief concern of providers as it drives workload, quality of working life and profit.
- Patient satisfaction, for which we publish summary charts in real time:
The benefits from wholly embracing system change are orders of magnitude greater than from any hybrid system, and a large part of our work is giving practices the confidence to do so. Followng that is an enchanting journey of learning, experimentation and refinement.
We too are continually learning, often finding we can help the flow with new features, better measures, or working with practices to solve unique problems. Sometimes we change our advice,
While change cannot be absolutely risk free, for many businesses, staying the same may be the most risky strategy. We only have to walk down our high streets to see the consequences. For most GPs, protected as they are by permanent contracts, staying the same and offering a service no worse than others locally may not seem too bad, though it hardly inspires.
Our greatest challenge, and by far the greatest cause of failure, is failure to overcome fear, to reach consensus and therefore failure even to start. Let’s re-iterate our purpose in undertaking this work:
We make it easier for patients to get help from their own GP.
We make it easier for GPs to provide that help to their own patients.
Sometimes the way may seem hard, but reward comes through perseverance. We have a mantra to take us through those times:
“First for our patients, then for ourselves.”
PS After the rain, sunshine. I’m about to share some of the most moving patient feedback we’ve had, just from the past week.
I’ve no doubt Jeremy Hunt meant well by what he called his birthday present to the NHS, a new NHS app. But as W Edwards Deming said, “Best efforts are not enough, you have to know what to do.”
“I want this innovation to mark the death-knell of the 8am scramble for GP appointments that infuriates so many patients.” says Hunt.
He’s right that innovation is needed, right that there’s an 8am scramble and right that patients are infuriated. One phoned me this morning, absolutely fizzing about her practice, but not one of ours and there was nothing I could do. She told me she could book online, but there were never any GP appointments soon enough so she physically went this morning and still no joy.
The gap is in understanding the problem: it’s the system. It’s not lack of online access, standard for some years. Bad news, it’s the system, meaning the operating system of the practice. Good news, it’s the system, meaning it can be changed. By whom? The GPs who run the practice.
Even better news, it isn’t a matter of resources. The BMA is right that Hunt’s NHS app won’t create any more appointments, but their knee-jerk reach for the begging bowl so lacks imagination.
I won’t bore you with how we are helping practices to achieve 30 – 40% efficiency gains, and help patients within minutes, because you’ll tell me it’s too good to be true.
But I’ll share with you a brand new chart which astonished me this week, and it goes to the heart of Hunt’s problem definition. A month ago we started asking every patient when they send in from askmyGP how they would like the GP to respond, whether email, phone or face to face. This is from 12 practices who have done Transform, online varies from 15% to 80% of demand, average 35%.
Even though around 30% of patients need a face to face, only 15% are asking for one. GPs are having to persuade some patients to come in.
It seems obvious after all: patients don’t want an appointment, they want help with their medical problem from someone they trust.
But if you make it a thing to book appointments online, then that’s what they will do, and take 10 minutes of GP time, even though neither party wanted it.
The BBC listened, thank you, and we have been saying this to NHS England for a long time, but they aren’t listening. Can you help?
PS #GarethSouthgateWould not mention that 6 out of 9 England goals have been scored by a Harry, so neither would I.
PPS All the above practices started with Pathfinder – Could you be ready to change? It’s normally quiet in summer but we are surprisingly busy and it is actually the best, quietest time so do get in touch today.
Links are provided below to published material evaluating the eCONsult/webGP software supplied by eConsult Health Ltd (formerly Hurley Innovations Ltd). To our knowledge no other independent studies have been released – please respond below if any are missing.
BJGP, from CAPC, Jon Banks et al.
“Conclusion The experiences of the practices in this study demonstrate that the technology, in its current form, fell short of providing an effective platform for clinicians to consult with patients and did not justify their financial investment in the system.”
Number of eCONsults received per practice per day: 0.9
BMJ, Uni of Exeter, Mary Carter et al
“Results: WebGP uptake during the evaluation was small, showing no discernible impact on practice workload.”
Number of eCONsults received per practice per day: 0.9
BJGP, Michael Casey et al. The product name has been changed to “Tele-Doc” but the context leaves no doubt that this is a study of eCONsult.
“Uptake of Tele-Doc by patients was low. Much of the work of the consultation was redistributed to patients and administrators, sometimes causing misunderstandings. The ‘messiness’ of consultations was hard to eliminate. In-house training focused on the technical application rather than associated transformations to practice work that were not anticipated. GPs welcomed varied modes of consulting, but the aspiration of improved efficiency was not realised in practice.”
MDPI, J Cowie et al, University of Stirling, 11 practices in Scotland over six months to August 2017.
“However, there is less certainty that it has fulfilled expectations of promoting self-help. In addition, low uptake meant that evaluation of current effectiveness was difficult for practices to quantify.”
3.1.1 “The distribution types of eConsult submitted were 32% specific conditions, 27% administrative help, 41% general advice (24% for a new problem, 17% for an existing problem).”
3. 4 “Consensus was that a straightforward phone call simply requesting an appointment could be handled quickly and in less time than processing an eConsult.”
3. 5 “However, at the current levels of submissions, there was a general consensus that eConsult did not offer cost savings”
Number of eCONsults received per practice per day: 1.3
NIHR Journals, Atherton et al, 8 practices including 2 using eConsult.
“E-consultations were also very rarely used, accounting for 0.22% and 0.23% of consultations in those practices that offer them.” p89
“The current very low level of uptake of alternatives and the lack of clear evidence of benefit may influence their uptake on a wider scale, something which is favoured by policy-makers.” Conclusions p98.
One of the policy-makers was Dr Arvind Madan, former NHS England National Director of Primary Care (resigned 5/8/18), also a partner in Hurley Group, owner of eConsult. Other Hurley Group senior partners and major shareholders in the company are CEO Dr Murray Ellender and former RCGP chair Dr Clare Gerada.
This NHS England engagement document contains on page 6 a case study, mentioning 145,000 eConsults in 2017, from 4.4 million patients in 465 practices. This allows us to calculate the proportion of total demand sent through the platform. The average practice sees 6.5% of patients seek help from a GP each week (figures collected over 7 years by ourselves, and used in all capacity planning models). In a year, 4.4 million x 6.5% x 52 weeks = 14.87 million consults.
145,000/14.87 million = 0.97%
From their own data, fewer than 1% of consults in their customer practices are through eConsult.
Compare the above with claims made on their website econsult.net. At the time of writing the above links could not be found on the site.
Increasingly practices ask us how others use askmyGP, so we have collected here examples with agreement to be in the public domain.
Please be aware that they are all busy GP practices so have not committed to answering in person an unlimited number of queries. They are all different in some respects from your practice, yet they all share common features of a registered list of patients whom the GPs are committed to serve.
They are all on a journey of change, which started with Pathfinder – could you be ready?
Concord Medical Practice – 14,500 suburban family practice, north of Bristol. SAPC poster.
Central Surgery Oadby – 8,700 suburban Leicester. Presentation given to the CCG. Webinar with Dr Chris Thompson Online Consults – Our (very short) Journey of Change
Balaam St Surgery – 5,600 East London practice, blog post of interview.
Witley & Milford Surgeries – 11,200 rural Surrey, two sites. Dr Dave Triska @dave_dlt tweets as launch unfolds. This 45 minute recording could change your life: Witley and Milford launch, as it happens – Dave Triska interview.
Every practice will work out their own mode of operation, and with our help can seek to optimise effectiveness and efficiency. Each of the above sees between 30% and 80% of demand arriving online and their numbers are part of over 70,000 patient episodes managed through askmyGP in the first year of version 2.
With the publication in this week’s BMJ of the Tele-First study into the telephone first model of general practice, you would expect me to read carefully and respond. So here are the headlines:
- 65% of patients report being phoned by a GP in less than one hour.
- 56% of patients find it more convenient vs 22% less convenient
- Large improvement in length of time to be seen, 20% move in GPPES survey.
At a time when we are told repeatedly that patients are having to wait ever longer to see a GP, often measured in weeks, these are quite astonishing figures, all quoted direct from the report. But, dear reader, these are not the headlines you have seen in Pulse or the BMJ Editorial are they? Studies, and the interpretation of studies, are political. We have an interest, and so does everyone else.
Therefore the first thing I want you to do is read the full text so you can make your own mind up independent of headline writers. It is much more detailed than the print version, framed by an angry looking GP model and a scared looking patient model, giving more space to a commentary piece than the actual study.
There is much to absorb but for brevity I’ll comment on the summary section.
What is already known on this topic
- GPs are struggling with the current demands on general practice and looking for effective ways to manage patient demand
- Claims have been made, reproduced in NHS England literature, that a telephone first approach, in which all patients wanting to see a GP are asked to speak to a GP on the phone first, results in major cost savings for primary care and reductions in secondary care costs
We do not make those claims, but you can still read them here on the home page of PPC Doctor First, a 20% drop in A&E and £30,000 saving per GP per annum. I’m grateful to the authors for proving these false. *
What this study adds
- In general practice, many problems can be dealt with by a GP on the phone
- The new telephone first approach resulted in more phone calls, fewer face to face consultations, and, on average, more time spent consulting
- There was wide variation between individual practices, including large increases and large decreases in workload after adoption of the telephone first approach
- There was no evidence that the telephone first approach would reduce costs of secondary care
In a way it is disappointing to see no secondary care effect, but not unexpected and unless the evidence changes, that is what we accept.
But what has really got GPs aerated is this finding of “more time spent consulting”. This was derived from data sent by us to the study, which we have not used to make a calculation on workload for several reasons: much of it is missing (and as the authors state, had to be imputed), it shows wide variation, and it cannot account for total workload. Let’s consider:
Workload = demand/efficiency + non-clinical work + waste
We do not have a reliable way to measure the total, and given that the study used only one of our three datasets, I don’t see how they can make this assessment. Just one example: many practices have told us of the drop in home visits, each one saving the time for many surgery consultations. This is not measured. It may be a good thing to have more recorded time consulting, if less time is wasted. Not only does this finding seem to me unsafe, it also brings us back to the question of purpose, for the study and indeed for the NHS.
If the purpose is to minimise GP workload, we can do so very simply: design the working day so you see 4 or 5 patients in the morning, take a good lunch and a nap, then spend a little time in personal reflection and development before heading home., purpose achieved.
I’ve worked with a lot of very hard working GPs and they would not be satisfied with that purpose. No, the purpose of general practice and therefore the purpose of change must be to improve patient care.
There’s a missing term in the workload formula, and that is “unmet need”. Behind those words lies untold suffering and frustration of patients, heard perhaps by a receptionist (one wrote last week, “I dread having to tell the patients there’s nothing left”) while others do not even get through on the phone. This is the dirty secret of general practice, and over many years we’ve measured it in practices we’ve helped, variable around an average of 14%.
One in seven patiients is told to go away. Although we offered this data to the study team, they didn’t want it and took no account of it.
Their figures cannot distinguish between the workload of one GP helping 30 patients in a day, who had all waited two weeks, and another helping 40 patients in a day, on the same day they called. It could be life changing for those 10 patients, indeed all 40 of them for not having to suffer two weeks of disease, pain, or anxiety.
Both GPs may have equal skill and compassion, but the difference comes from efficiency.
By framing the question on workload rather than efficiency, the study misses a huge opportunity. It offers no help on how to become more efficient, and while it found wide variation in performance, the data were munged into averages rather than investigating in detail why the best ones worked better.
I’ll tell you a secret: we’re in this for the patients. To help the patients we have to help the GPs be more efficient. There is never a final answer to the method, there is only “the best we know for now, while we look for the still better way”.
We’ve helped around a million patients so far, with another 50,000 to be added in the next month. and as telephone triage (done well) is more efficient than pre-booked face to face, digital triage is already proving to be the next step. Sometimes we fail, but we press on.
Every day over 100,000 patients are told by practice receptionists “Nothing left, call another day”. Not on any basis of clinical need, just because the GPs have no slots.
It’s my personal mission to eliminate that phrase. What’s yours?
* The 20% A&E effect came from my 2011 study, based on pioneer practices with up to 10 years running the model, and promising at the time. The figure was copied by Dr First but never attributed. We could not show that the effect was reproducible, and therefore stopped making any specific claim about A&E 3 years ago. £30,000 saving? We make no such claims, although if GPs tell us about savings we are happy to report them. Why did NHS England swallow this?
Hypothesis: efficient operation of primary care depends on clinical triage of all demand, to optimise the use of scarce consulting resource – GP time.
The faster and simpler the system, the more patients will co-operate.
Who does what, when and how?
Patient “I need help…” Make it easy to provide enough detail for triage. Online, anytime.
Reception ”I’ll assign you to a clinician, unless I can help you myself” (within minutes, verify patient, choose clinician)
GP “I’ll work out how to help, usually phone, may see you, send a message, or refer” (take seconds, within minutes, from online entry)
Consult & complete – precisely appropriate for the patient and episode.
Presented at EFPC European Forum for Primary Care, Annual Conference Porto 24-26 September 2017
Download the poster here:
A quick note with exciting news, we’ve just had our poster published at the Society of Academic Primary Care SAPC Annual Scientific Meeting in Warwick.
askmyGP has now passed over 50,000 patient episodes, 4,000 of them on the all new platform launched just two months ago.
The case study with Concord Medical Centre, Bristol, is here:
What took demand to 30% online? In a nutshell, it’s:
– Personal (“Hello, I’m Dr Bradley…)
– Universal (all patients, all problems)
– Responsive (we’ll get back within the hour)
– Simple (“Easy to use” main theme of feedback)
We took the decision in version 2 to take OUT the clever technology we’d put in v1.
It’s much simpler, with the aim of putting patients in faster, easier, touch with their GP, and vice versa. It builds trust by allowing patients to express exactly what they mean.
The result? Positive feedback has shot up, both from patients and GPs.
Dr Simon Bradley comments:
“The thought that goes into putting something into writing often helps the patient to have reflected on their problem prior to initiating a request.
Then for the clinician to have reflected on the request and reviewed relevant elements of the record means we can be more aligned with the patient’s agenda.
Online communication is asynchronous which allows both patient and practice to use their time more effectively.”
Aha. Time. The only absolutely finite resource.
Time for recreation too – enjoy the weekend.
Download pdf: What makes patients use online consultations?