Creating the Ultimate Doctor Receipt Format for Product Quality

Doctor receipt format for Product quality

it's a a real pleasure to be here um i've kind of caught up with so many um old friends um uh from my time when i was uh either out here in as an intern at the rural jubilee hospital or when i was uh uh any oncologist in halifax so it's it's really like coming home and i i really appreciate the welcome that everybody has um given me so i thought it might be interesting to uh talk to you guys about the uh national ghanaian oncology registry which is something that i've been uh part of setting up it's obviously not just me it's the whole group of people that have done this but um it's been a work in progress but we're finally um um getting some rewards for all of that so i thought i might uh talk to you about um uh this won't move forward oh there it is um about the clinical quality register why we thought it was important uh what exactly it does uh how we got it going and um i guess most importantly including the mistakes that we made uh and i also say to my registers you know you must always learn from your mistakes but it's actually better to learn from other people's mistakes so i'm hoping that if i talk to you guys about it and you think that this might be something worthwhile for you to pursue that you'll uh take a few lessons from from us uh in that regard and also a little bit about what the future holds for us um so i think uh like it's really important in this day and age um that we that we look at our surgical outcomes uh quality of care is important um those who are involved in in giving the care and those who are involved in receipt of the care really want to know that they're getting the best care possible and um especially with increasing demand and decreasing resources we have to know that we are providing the best care that we can um and so ultimately quality care is a value judgment which reflects the values of the health care system but also the values of the society at large and and they may be different from uh country to country or place to place um if we're going to use targets we have to be uh the targets need to be smart that means they have to be specific they have to be measurable they have to be achievable they have to be result oriented and they have to be timely and you can use these targets to establish baseline information um to benchmark against peers just set standards and priorities find out what you're doing well and what you're not doing well and maybe what you need to change and of course inform quality improvement but this doesn't come you know easily it requires a lot of time effort and resources i'll take you through what what we did and how we did it so we do know that variations in care exist um this is an old publication from way back when uh 20 years ago where we uh a patterns of care a study was done in victoria which is one of our states not victoria the city and that they found that a third of women with ovarian cancer had sub-optimal care both in the form of the surgery and the medical oncologists weren't off the hook either chemotherapy was given inappropriately as well and although things have improved over the last 20 years they we know that anecdotally there is still plenty of room for improvement um it's not just australia that has difficulties when there was a publication in the landsat in 2011 we saw that there were significant variations in the one and five year survival in ovarian cancer patients across several jurisdictions in several countries you'll be pleased to see that regardless canada seemed to come out on the top but i would say that there should be some caution in interpreting data across countries because of uh differences in healthcare systems in ascertainment in uh incidents in uh you know genetic differences as well so you know that sort of says to us that what we need is a is a uniform system um of comparing um outcomes uh and that's really what a quality uh registry clinical quality registry does what do we know about current ovarian cancer care in australia well there's something called the aocs which is the australian ovarian cancer study which ran from about 2004 i think to 2008 which collected data on 800 1800 um ovarian cancer patients across the country um and it's really a rich source of data because it has not only the um it has clinical data that's associated with pathologic data and um uh and other data so we've used that to to get a lot of information about and have a a countrywide picture of what's going on so from the aoc cs data we found that there were variations in chemotherapy you're less likely to get chemo if you're older than 70 and more likely to get single agent chemo in total less than 70 percent of patients got combination carbotaxial and only 50 completed treatment without dose reduction or delay um when we looked at the aoc aocs data regarding surgery um we saw that only um 68 percent had lymph node sampling and early endometrium epithelial ovarian cancer and we also found that there was inequity of access and that remoteness socioeconomic disadvantage and comorbidities were all associated with poor outcomes so um you know there's some evidence that we're not doing that well in terms of equity surgery or chemotherapy um and you know i'm very grateful to amy and to jessica and to all the people in the group that um nicely published this work just in time for my talk which shows that it's no different in canada and this is reflecting in endometrial cancer care which obviously is the impetus of having an objective system like the promise classification but there's variation in practice which one can only assume will result in variations in outcomes as well so um how do we approve outcomes in ovarian cancer um well we have control over more things than we realize and it's yeah it'll be a shock to surgeons to find out that it's not all about um surgery uh it's much more than that it's it's it's important that all of us that are involved in the care of women with endometrial cancer or with gyne cancer in general um are involved in in quality care so it's not just surgical staging and debulking we obviously have to have correct pathologic diagnosis we have to have select appropriate patients for brca testing we have to optimize chemotherapy identify high-risk and low-risk patients we obviously need to have access to clinical trials but also supportive care during and after treatment prevention secondary surgery optimal management and recurrent sex setting timing of treatment and avoiding over treatment and under treatment so it's more than just the surgery and i think that's what we we came to realize when we were putting together our um clinical quality indicators is that it wasn't all going to be about surgery so what is a clinical quality registry well it's um it collects uniform data to measure performance of either a practitioner or a health service or you know if it's international a a nation um a country about against a range of agreed clinical quality indicators and the important thing is that it has to be an agreed um clinical quality indicators and so everybody has to develop what they think is important for them for their patients for their hospital for their province for their uh country um it has to be risk adjusted so you know one hospital might get all the the lean young patients and another hospital may get all the old uh obese patients and obviously uh comparisons between those two hospitals and need to be risk adjusted and benchmarked and what it does is it empowers health service providers to improve patient care and outcomes so if you find that your hospital you are have got a below average indicator then it would be a powerful uh piece of information to give to your um uh health service providers about how things should be improved and a way uh potentially of going to those uh hospitals who appear to be doing it well and say what are you guys doing different that that we could um learn from and then it gets back into the whole quality improvement cycle which is not just a one-off thing uh it's you know you collect data you risk adjust it you provide feedback you um assess the outliers you make improvements to the system and that results in better care and improved patient outcome and then the whole cycle begins again um i um i think we i think i've got here outlier assessment and we actually don't use the word outlier because that has a somewhat derogatory term we we use the word unwarranted variation so there might be warranted variation because one hospital is dealing with very um sick patients but if there's unwarranted variation which is why it's so important to provide risk-adjusted data so if you're going to do quality assurance it has to be sensitive and what that means is that you have to have every patient or almost every patient so this cannot be an opt-in system it has to be an opt out system um so that you get 90 to 95 percent of all patients because if you're only collecting data on 50 or 70 of patients you're not going to have the the sensitivity that you need i talked about detection of our unwarranted variation in care and it can be an early warning system for problems in care if we providing this data on a yearly or a um two yearly basis it is much quicker than i don't know what what your um uh outcomes that are published by the national um uh cancer registries are but for us they're five years out of date so uh it's a slow um that's not quick so we want this early warning system and there's no doubt that you know if you're told that you are underperforming um you know doctors are kind of competitive people and they want to to be the best and do the best it certainly would be an impetus to improve patient care and we're hoping that this will also be the inspiration for and the platform for research projects that questions will come out of all of this and and people will want to embark on research projects because they will be a rich uh database of information um its way of bringing clinicians together and stimulating collaboration and ultimately we want it to um improve the care of women and help implement systemic change so i just wanted to give you some examples from the prostate cancer registry that has been um up and running in victoria a that is the state of victoria since 2012 and they um the information that we get is given in this funnel plot with the the average um the first one standard deviation and two standard deviations and so this site here was identified as falling outside the two standard deviations of care um in terms of a positive surgical margin rate and so when they looked at it there were some changes that that needed to be made and uh pathology was reviewed and surgical training program was reviewed and there subsequently were changes and they also um found that there was subsequently it there was an adherence to a better adherence to guidelines for low risk cases and there's also a victoria lung cancer registry which um which identified differences in the timeliness of care in public versus private hospitals which i know is not an issue in canada but it's a big issue in um in australia so um clinical quality indicators um have to be they can be outcome uh indicators they can be process indicators or they can be structural indicators so i'll go through these and just and just give you an idea about what we're talking about so the outcome indicators are uh relate to the recovery um and restoration of functionality and survival of patients so they're a pretty hard end point um so they're easy to measure um and they're certainly valid as a dimension of safety and quality but they're a little bit difficult to use as direct measures of safety and quality because um of the patient factors that i've spoken about before i e do you have very healthy patients in one center and and very unhealthy patients in the other center and so um an example of this would be 30 and 90 day post-surgical mortality it's a clinical outcome or it could be a patient reported outcome you could say the percentage of patients who report survey or pain at you know two weeks post surgery or whatever so those are those are pretty hard endpoints um but don't always directly relate to the safety and quality of care that's that's given but can do process indicators measure the extent of the application of good health care and so um usually in reference to standards that are set guidelines and standards they're usually more sensitive to differences in quality than outcome measures and can be easier to interpret so if we say the standard is that all patients should have a confirmed histologically confirmed diagnosis prior to starting chemotherapy then that's a pretty hard endpoint that you can use and the third one is structural indicators and that's that that's things like um adequacy of facilities and equipment you know is there a gynae oncologist on staff um is the does the hospital or healthcare system provide enough infrastructure for you to do the work these are easy to collect but not always easy to clearly link with patient outcomes so an example of this would be the number of primary ovarian cancer surgeries performed by ghani oncologists so um in about 2015 and we talked about in in the australian society of gyne oncologists we talked about this off and on for many years about how we wanted to have a surgical clinical quality registry that we wanted to measure how well we were delivering surgical care and uh [Music] professor rob rome who's a gyneoncologist in um in melbourne really ran with this and got us together when we all said we didn't have time and we had other things on our mind he just said he kept go on and on and said we need to do this so we established a working group of clinical experts that gyani oncologist radiation and medical ecologist who who wanted to be involved in this we conducted a literature review um and we subsequently conv convened meetings of each working group we started with the ovarian cancer because that seemed like the easiest one to do to define a set of key quality indicators and we subsequently expanded that to other uh tumor sites and then i think the really important thing is that we really needed to um pilot the data um uh to make sure that these clinical quality indicators were um correct and um we've learned some lessons pretty quickly about that so these are the ones that we came up with in 2017 and you have to have a reasonable number but you can't go overboard and so we chose some things that we thought were important like appropriate staging um present presentation at mdt meeting uh histologic or cytologic confirmation prior to chemotherapy um first line therapy which is platinum based debulking surgery and the extent of residual disease whether it be primary debulking or interval debulking um we wanted to look at complications we wanted to look at adverse events post-surgery and then we wanted to look at um genetic testing so um those are the ones we started with um we had minimal funding um we had some money from asgo uh australian society of ghanian oncologists i think fifty thousand dollars the cass foundation is a philanthropic group that gave us some money and ovarian cancer australia which is a consumer-led group also gave us some money um we had to get ethics approval for this for each site and uh we started out with 10 public and private hospitals in new south wales victoria and tasmania but ended up um i think only having seven hospitals which who collected enough data that we could use it in the initial pilot stage we had to it this was a problem uh um ethics committees are used to having opt-in um ethics approval and an opt-out ethic ethics approval is was seemed to many of them a step too far and it required a lot of um persuasion to say that this whole thing was uh pointless unless we had an opt-out uh system um because we had limited resources we decided that we were going to utilize the existing databases that many of the units had and unfortunately we found out that the individual unit databases were not compatible and so what we ended up doing is downloading the information onto an excel spreadsheet and then sending it to the central registry um and having them collate the information the problem with that is that this is all added work for the unit data managers if they had one and if they didn't then it was added work for somebody else and although we said that those nine clinical quality indicators were really important they were not routinely collected in every site so that meant that in some cases there was a large amount of missing data anyway we pressed on because we thought we've made a start we've got some money we really need to keep going um but you know there were problems with uh as i've said incomplete data sets um incomplete patient ascertainment delay in sending data we quickly realized that some of their cqis were not appropriately defined and importantly that co-morbidity information was not routinely collected and i've gone on and on about why we need that for benchmarking um so um i received the first report in 2019 and it was de-identified except your own site was um was identified and of course you can imagine i was very pleased with this um with this result because obviously we're first on the top of the leaderboard which is where i always want to be everyone says i'm just tad competitive um so that was great we were doing well 100 of patients were discussed in mdt before during which a treatment plan was made [Music] this was not so great so what happened here is that ing to this registry only 4.5 of our early ovarian tubal and peritoneal cancer patients were appropriate stage which i didn't believe but um that's what the registry said and if you go back you'll remember what the first cqi was it was the longest of them all and there were all these points where interpretation could be made because what you've got remember is the people putting the information into the registry are not doctors they are coders and so there was too much in here room for interpretation so we're clinically at uh applicable mucinous tumors only if fertility preservation is not a priority so all of those things had to be interpreted and i suspect what happened was it was not interpreted correctly we did well on patients receiving platinum-based chemotherapy um we did well on optimal debulking um we did very well in intraoperative complications um we didn't do so well with uh post-operative uh um adverse events within 30 days of surgery and ing to the registry um over 70 percent of our patients had either death unplanned return to theater icu or ccu admission or hospital readmission and another 10 percent had all of those other complications which uh the problem is is that that was poorly um collected data and and not benchmarked and i really don't think that was was correct so like i said uh the problems with this is that with incomplete data sets no bmi in in almost half the patients no e cog in a third of the patients no stage in 13 of the patients so it could not be um it could not be benchmarked uh we were pretty sure that we weren't getting all the patients the clinical quality indicators as i said were often um not appropriately or incompletely defined and the comorbidity information uh was only collected in about less than 10 of patients so um fast forward to april 2022 and what's happened since then well a number of things have happened um we have refined the epithelial ovarian and tubal and peritoneal cancer module and refined the cqis on the basis of the feedback from the first episode we've now recruited uh 1500 patients um and we're really quite fully operational there um we have completed the endometrial cancer module cqis and have pot and are piloting them in two sites um and have recruited uh just over a thousand patients and we've um developed a rare ovarian cancer non-epithelial module and um have recruited 130 patients to that in the pilot phase um we've got 25 sites approved in victoria new south wales tasmania south australia west and western australia you will notice i don't know if you can follow my marker here or not but this amounts to about 60 geographically of um australia uh and uh but about 80 of the population lives in these areas so i'm not saying we've got 80 of the population but we've got access to 80 of the population the big outlying areas are northern territory which did not have a ghanian oncologist for a long time and just had one now and queensland who were kind of doing their own system a little bit but i think we're going to persuade them to come on to um to this uh this registry so um so it's a work in progress but we're getting there and these are the um hospitals that are involved so there's a fair number there's a fair amount of work to get everybody here on board through ethics and ready to go this is obviously not all the hospitals in uh australia but it's most of the hospitals and where a ghanian oncologist would be doing work and it includes both public and private hospitals so that's great so um what's happening with us so we've got two areas that um we're working on one is the ovarian cancer registry and um it looks after the core ovarian tubulin peritoneal cancer but as i mentioned before we've got a rare tumor module as well and we've also got a patient reported outcome module as well that we're working on and the second part is the epworth expansion project which i'll talk about in a in a in a few slides so let's just go to the ovarian cancer registry module so um this came about as a result of a 3.5 million um grant application that was successful in 2020 and what we said we wanted to do and this is from the federal government through the medical research fund um and what we said we wanted to do was we wanted to increase clinical data collection and recruit a greater proportion of the australian ovarian cancer population and what we were doing how we were doing that is we were giving the site some money to help with um patient uh data collection which had been a big problem um and we've said we would um produce timely qr reports uh that we can then uh send uh off to uh both individual clinicians and to um health care services we're going to measure quality of life with patients with ovarian cancer by collecting a patient reported outcomes and and reporting this back to the health services and um create a rare um tumor sub-module um to recruit women with rare tumors so we're at this uh point now um and so i'm gonna show you what what now we've had reported back to us so um this is data from 2017 to the end of 2020 um so that's when the patients were operated on and then it takes a little while for all the data to come through to us and it's a total of 573 patients if we're looking at the advanced otp cancers um that was 573 patients and so i'm going to give you the funnel plots for stage 3 and 4 disease with either no residual disease at primary surgery or no residual disease at interval surgery and so this is the proportion of patients with advanced otp who underwent primary cytoreductive surgery and so this is the uh average this line here which was if you recall was about 46 this is the one standard deviation this is two standard deviations so you can see here that most people are doing well they're um they're meeting the cqi requirement but there's obviously an outlier here or there was there's unwarranted variation here and so this is an early warning system that maybe something is going wrong uh here and that that needs to be investigated um and this is a proportion of patients with advanced otp cancer who undergo interval cytoreductive surgery and have no macroscopic residual again the percentage is about 44 and um again there's another outlier we have no idea if this is the same outlier as before or a different outlier because we only get told our sites um but and then there would be potentially a cause for concern there as well um the other example i'm going to show you is one um where we're looking at intraoperative events and so the problem with intraoperative events is that there was um no real grading system and so an awful lot of free text was provided um oh diane's just arrived diane i'm on a zoom call doing a lecture to cqi so anyway she's um she's just she's just arrived but anyway we'll keep going um so the problem with this you can see there's a few cases here that fall within the area that we want um but there's a few that fall out lie and and we don't want free text free text makes it very difficult for the coders to really um code things correctly and so we need a classification system of intraoperative events and and that's something that we really feel that we need to work on um post-operative events um was a little easier easier to collect because there's an a grading system and so we've encouraged people to use the clavian dindo grading system and it avoided freehand entry which made it easier for the coders to figure out what was going on and you can see that there's a much narrower window here and and supposedly we have no unwarranted variation there um so the other part of the uh ngor is the epworth expansion project and it's uh called the epworth expansion project because it's made possible because of a philanthropic um um donation from the epworth from audrey voss in the epworth medical foundation if we lived in vancouver it would be called the audrey vos expansion project but we don't live in vancouver um and so this really was um expanding from endometria from ovarian cancer to endometrial cancer cervical cancer and vulvar cancer um and we did the same thing that we did with the ovarian tumor site module working groups literature review conf convene meetings to um define their clinical quality indicators and then pilot them so we're at a a pilot phase now and um so these are the endometrial cancer clinical quality indicators a lot of the stuff is the same you know is it discussed in mdt has appropriate imaging been done was a hysterectomy done as as part of the primary treatment um those are who are at high risk um did they have surgical staging appropriately intraoperative vents um was the pathology right uh contain all the minimum required elements um what were the adverse events and then things that are kind of pertinent to endometrial cancer such as adjuvant treatment our radiation oncologists were really keen to have a a um cqi about timelines of radiotherapy because of the known uh problems with um extending out the radiotherapy um and then obviously genetic testing and i really wanted to put in clinical trials and so it had escaped the ovarian cancer one but it's in the new one and we put the clinical trials in as well um cervical cancer much of a muchness mdt appropriate imaging appropriate surgery we wanted to talk about negative surgical margins uh we wanted a proper surgery report and pathology report again adverse events um those who have uh platinum-based chemotherapy along with their radiotherapy as primary treatment duration of radiotherapy because we know that affects outcomes and we also wanted to collect to make sure that we were not [Music] inappropriately doing surgery on patients who then needed to have radiation afterwards uh and again vulvar cancer quality indicators same sort of thing multiple mdt um discussion uh obviously sentinel nodes is a big area here and so those sorts of things did the central nodes were they appropriate to have a central node procedure was it a successful procedure was the pathology um adequately processed and reported if it was a central node procedure margins adverse events and then obviously growing recurrence is an important cqi so um so what does the future hold for the national gynecology registry well one of the things that's going to happen in the future is personalized report cards so every surgeon is going to get a record of his patients that are on at the registry um and what his results are and i'm sure there'll be some sort of comparison benchmark to other um hospitals and other um clinicians as well um we obviously need to finesse the ovarian cancer module and we need to add in the patient reported outcomes and the rare tumors um once our pilot study is finished with our endometrial module we'll look back at those cqis and see whether or not um they need to be finessed and we obviously need to pilot the cervix involver cancer modules as well um we're in discussions with the scottish group and the dutch group who already have their own clinical quality registries the scottish have kind of had theirs developed since about um i think 2013 and we actually um relied quite heavily on um theirs as a template and they've got some um um really interesting data that they collect and have shared with us as well and and which is available um publicly as well um and we've just been in uh discussions with the dutch group as well the dutch group are really keen to harmonize international some uh international um cqis so that we can compare uh cross country and look it would be really great um if we could collaborate with the canadians because they're my preferred collaborators and um and work together with you guys to see if if you can come up with something like this as well if you think it's appropriate so i'll just go back to the personalized report cards because i just remember that i did actually ex uh put an example up so this is data so this is this is not me um or nor is it anybody because we haven't actually developed this um but what you would get is you would get an overview of data completeness for your patients how many patients you put in uh how many opted out primary side of tumor you know something about the histology stage of diagnosis grade genetic testing outcome something about your surgery and what you did extent of residual disease and what kind of surgery you did um something about your operative adverse events um and um something about chemotherapy as well so um so all interesting um information um so i think that might be all i have to say what are we doing at so we've got um we've got a few minutes left on it but i don't really think there's oh obviously i need to make my acknowledgement so this is uh rob rome and i i describe him as a dog with a bone he just did not give up he was the one that went and out and got the initial funding that allowed us to um to to start off with our pilot project um and he uh collaborated with uh john salzburg who is our academic lead um who was instrumental in setting up the prostate cancer registry and the lung cancer registry in victoria um and together rob and john put together the grant that allowed us to get 3.5 million from the federal government to um to progress the second part of of the clinical quality registry um and listed there are all the working group chairs who have worked hard to get all the um clinical quality indicators for um various other areas um honestly this is like everything the doctors have the great ideas but it's other people that do the work and that's obviously the data managers at all our sites who've worked tirelessly to put that in and then our patients as well for whom a very few percentage have decided to opt out of all of this so we're obviously very grateful to them to allow this to continue and i think that is now it