Collaborate on Simple Receipt Format for Quality Assurance with Ease Using airSlate SignNow

Simple receipt format for Quality Assurance

on quality assurance best practices and tips we really appreciate your time we know your time is valuable and we hope to provide you with a lot of great uh tactics uh strategies and and best practices that you can take back uh to your agency uh today and start uh either developing a quality assurance program or uh building out your existing quality assurance program I'm pleased to be joined by my colleagues uh Cheryl orberg and Nathan Lee from Denise Amber Le Foundation uh I'll introduce them here shortly in our vision uh we'll go through uh the importance of quality assurance and um Sher will take you through an overview of the new acon An approved quality assurance standards we also talk about tools that can help uh you develop your quality assurance program and improve that U some evaluation templates that are available for free as well as uh some quality assurance software tools that are uh avail and a case study as well just a brief introduction my name is Patrick Bo I I'll be your host today uh I'm the director of Workforce optimization at VPI I've been with the company for 12 years and uh have worked with uh hundreds of various uh Public Safety customers throughout North America and the world uh implementing quality assurance programs and uh uh recording systems you know um phone radio text messaging reporing systems and so forth and um I also co-authored a book recently uh called the high performing Pie app it's a quality insurance and ng911 recording guide book and if you'd like a uh a copy of it if you haven't read it already uh feel free to uh shoot me an email here and uh I I get one sent out to you uh next month so uh if you're not familiar with VPI we've been in the uh recording quality assurance Spas for over 20 years now uh Our Roots are in public safety recording we have over uh 600 customers uh just in North America Alone um and then throughout the world uh even more than that and we've been actively involved in the Next Generation 911 um the ice committees the ice six the ice 8 for logging and recording and a longstanding appco Nina partner as well and all of our uh our R&D and support um is based here out of the us and we're very proud to be an American company now I'd like to introduce uh my colleagues who who are joining us today uh Nathan Lee he's the president of the N Amber Lee Foundation which he founded in uh 2008 he's really the driving force behind so many improvements uh in public safety uh since starting this initiative uh relating to 911 procedures training and quality assurance he's a an acclaimed trainer uh if you've had the pleasure of joining his training sessions you know how fantastic he is uh he's also a consultant and uh speaks it quite of the quite a few of the national International and Regional conferences we're also pleased to be joined by Cheryl orberg she serves as the board of directors of the nise Amber Lee foundation and she functions as the quality assurance director of the Foundation she's also a senior consultant and partner of Pep Concepts and solutions she's a registered Public Safety leader and emergency number professional her emergency communications career began um 1970 and includes police fire and EMS dispatching supervision and 911 coordination in addition to her last position as executive director which he's held for more than 20 years as the executive director of a multi- Juris IAL multi-discipline uh p app in the Chicago land area she was responsible for providing General management overall policy directions and fiscal control of the $5.5 million budget with a staff of 50 employees she's also the past president of APCO um Illinois APCO and served in various elected chapter positions 1985 through 2014 uh as you can see she's got a quite a colorful background and uh was also very involved in the U quality assurance standard initiative that she'll be reviewing so at this point I'd like to turn it over uh to Nathan who will uh take you through the story and and the founding of Denise Amberly Foundation thank you very much Patrick um really appreciate the opportunity the VPI is allowed the foundation to partner with and and I really appreciate everyone uh for taking the time to be on this webinar today it's you know become such a Hot Topic and and should be a Hot Topic in the 911 industry uh and that's quality insurance and how important it is and and uh why we why hopefully every every 911 Center in the country within five years will have quality insurance of some of some form or fashion uh and it's we're going to kind of go over a lot of that today what I want to do is uh just kind of give some some of the folks on here a background uh of of the Story of My Wife and why the Denise Amber Foundation even exists I'm sure a lot of you that are on here are familiar with it but I just wanted to do a quick overview again and and and the reason I wanted to do that is because I think you know quality assurance was in place uh in this 911 Center back in 2008 uh I really believe that I wouldn't be here today um Denise would be and uh so so what I want to do Denise was uh kidnapped and raped and murdered back on January 17 2008 by a man named Michael King who was a complete stranger uh who uh police believe that it that it took place around 14:30 that he came to our house around 2:30 in the afternoon uh and took Denise from our house leaving our two little boys in the home alone in the house alone um the city of Northport is situated about halfway between Tampa and Fort Meers on Florida's West Coast and um Northport has their own Police Department uh but they're only a secondary Pie app they just dispatch law enforcement uh Sarasota County uh is the main p app in that area who answers all the wireless 911 calls um and ironically enough and that made this story I think more more tragic and also more popular on a uh on a national level was just the fact that uh Denise's father Rick G was a was a sheriff's Sergeant with the charlot county sheriff's office um at the time and still is currently to this day uh which is one of the agencies that was involved in this and actually the agency that made the biggest mistake that could have saved my wife's life so there there was a total of five 911 calls made on that day one for myself uh which was when I came home and found my two little kids no and Adam home alone uh and Denise wasn't there uh the second call was actually made by Denise herself uh at 1814 that evening from the Cap's phone in the back of his Green Camaro uh came in as a phase one 911 call uh they were only able to get a tower location so there was no lat long NO phase two and so they were not able to get the exact location or an estimated location of where Denise was but another 16 minutes later probably the most critical 911 call out of all of them uh was made by a woman named Jane Kowalski uh who was at a stoplight right at the Charlotte County in Sarasota County Line and saw what he thought was a child screaming in the back seat and banging on the window uh they proceeded after the stoplight uh she took off from the light the Michael King got behind her and she calls 911 which got routed to Charlotte County Sheriff's Office because they just crossed into Charlotte County um now a call or 15 years took that call realized uh it was Denise based on the description that the caller gave of the person driving the vehicle uh and the vehicle's description uh and at that moment um she stands up and yells that info uh to the two radio dispatchers across the room now the caker because she had a she was a 15-year veteran at the time um and there was no uh quality insurance proced there was no there was a very um lackluster level of accountability and Supervision in that Center at the time she had a history of writing things down on paper because she started on cards early in her career and she was not held up to a standard of having to type things in the Cat in real time um and that started becoming an issue in that Comm Center uh with her other co-workers uh the dispatchers were getting fed up over time that she wasn't doing that and that she was shouting information or writing things down and passing it to the two to the to the dispatchers in the com Center and I guess that because of the chaos because this went from nothing in a few hours to one of the largest man hunts in Southwest Florida history that actually involved the daughter one of their own uh because of the chaos in the 911 Center those two dispatchers um basically told her to put in a call in the cad and then we'll dispatch it and the call taker never entered a new call for service in the cad she entered uh an update uh into the existing bolo call that was already open for Denise's to appearance uh no one saw it the dispatchers never saw it and no one uh was ever dispatched regarding that 911 call and so Michael King uh then drove another four miles down the road pulled off on a barricaded street that was under construction uh and takes Denise out of the car shoots her and then buries her on a shallow grave um luckily Michael King was found uh was seen leaving that same road a few hours later uh and he spotted entering Southbound 75 where he's later arrested by a Florida Highway Patrol uh trooper and he was later convicted in 2009 and his current and is currently sentenced to the death penalty and is on death row uh so you know one of the other things that was that was very interesting in this case was you know it took it took almost two full days to get to find Denise uh she was finally located and obviously all the evidence that they found was very critical uh in getting him convicted of the death penalty so so after realizing all of this uh happened over the weeks and months that ensued after this happened um and really getting motivated and inspired uh by not only our local community but also but mainly from uh your industry uh the 911 industry um here getting letters uh emails um Facebook Myspace messages uh in support and um expressing their apologies for what happened and how the 911 system let let us down and let Denise down um I really wanted to do something to give back and to help the industry I I learned of a lot of the challenges that the industry faces from training standards to a lack of a lack of Standards with quality assurance and a lot of the technological advances or a lot of the technological issues that you all face every day as far as cell phone location and other things um so I was inspired and wanted to do something positive to turn this negative tragedy into a positive so I decided to form the Denise Amber Le foundation and we currently have national board of directors um from all over the country uh and as you heard Cheryl is one of those uh board of directors and we're just really really really happy and and thrilled to see the difference that Denise has made over the last few years and and in all honesty that wouldn't have been possible uh if the 911 industry wouldn't have allowed us to do that and so I'm extremely grateful to have the opportunity not only today but um to travel the country uh like we do teaching this story and trying to do anything we can to help but our mission statement is to support and promote public Public Safety through uniform training standardized protocols toine measurable outcomes or quality insurance and technological advances to the 911 system and as I just mentioned uh we really want to be that driving force uh and be that emotional driving force because we've seen the power of her story and how it can be used in a positive way so as I mentioned a little bit before um I I think our vision uh and our partnership Vision with VPI and I know Cheryl and myself and other members of our foundation uh we really en to see every piece at employing uh quality insurance processes within 5 years and I think that's an achievable goal um I think it's such a hot topic now and it's extremely important not only is Cheryl's going to get into not only for just trying to prevent these types of tragedies from happening and identifying as Gordon Graham would say those risky behaviors and identifying risky habits but it's also a great tool to use for the positive and to tell the people in a com Center how amazing they are and how well they're doing uh because that's not happening enough um and all the negative that they have to deal with uh it's really important that a Common Center focus focuses on those things that they actually can control and that is you know making sure they're recognizing their their greatest resource which is the which are the people the telecommunicators answering 911 calls um but some of the other services and some of the other programs that the foundation has been trying to do and and has has been doing over the last few years um we do training classes across the country we've developed a 4 and eight hour training course we're actually in the process of developing some other ones that we're teaching across the country um as patri mentioned we do uh we do attend all the national conferences and many state conferences and Regional conferences uh telling the story and talking about other Hot Topics as far as like hiring the right people quality assurance and teamwork different things like that uh but this new program that we're launching now that Cheryl is taken on uh is to try to help and consult with with 911 centers uh in helping them develop a quality insurance procedure or program or improving an existing program that they already have and based around that new quality insurance ancy standard that is that came out this past April so um like I said I'm I'm really really honored and and privileged to have the opportunity to be here today uh once again I want to thank PPI but I'm going to pass it over now uh to Cheryl uh who has now become a very very close friend of mine uh she is a um member like I mentioned of our board of directors I got to meet Cheryl many years ago uh probably five or six years ago when I attended the Illinois appco conference and was a speaker and we over the next few years um started working more and more together on the importance of quality insurance and I recognize her passion uh for trying to do anything she can to help this most important industry on the planet in my opinion so uh Cheryl it's my honor and privilege to to introduce you thank you thank you so much Nathan that was very kind I'd like to start out uh telling everyone a very interesting uh incident that occurred that I think speaks strongly to our topic today there is a Dispatch Center that I'm aware of uh that on this particular day had both uh call Taker and dispatcher being the same person while they had multiple uh TC's on duty they did both tasks of the call taking and dispatching and A call came in for a house fire and uh the gentleman explained his address and the call taker then immediately dispatched the responding fire units they got on the scene and they were able to save part of this gentleman's house and rescue him from the second floor bedroom the fire chief was so pleased with how well this call had gone and how well his staff members have handled the fire scene that he actually took the time to contact the peace app and say you guys did a great job thanks for being a partner with us well this just doesn't happen happened very often at least not in my experience especially if you're a separate piece but not attached to either a directly a police department or fire department we're sort of the invisible part of the public safety triangle no less important we just are not visible so the the chief called and this day there happened to be an acting supervisor who was really pleased that the chief took the time to actually Comm amend the uh the task that had been done so he congratulated the caller dispatcher and he wrote a little email and sent it to the off-duty supervisor off-duty Supervisor was equally pleased you know this is a feather in his cap for his shift to actually get a verbal commodation from the fire chief again not something that happens very often so he sent back to the call taker dispatcher and shift as a whole great job guys really made us proud and copied it to the deputy director deputy director was even more pleased he thought well this is really good you know when things work well and the public safety entities all come together said this is a great thing I'm I'm really really pleased so he sent a more formalized congratulations back to the call taker dispatcher and copied the director director sent a formal congratulatory letter to the call taker dispatcher actually came in and said you know you did a great job in front of the peers couldn't have gone any better we're really proud and at the next uh director's meeting he informed his bosses of this particular situation and the um board said you know next next time we have a meeting why don't you bring that person in because we'd like to show our appreciation for a job really well done has this ever happened in your P wouldn't you be proud wouldn't you be excited there's just one problem nobody listened to the tape my house is on fire I'm at 123 Main Street I am handicapped and I am trapped in the northeast corner of my house on the second floor please send help at no time did the call taker dispatcher ever tell the responding units that they had a trapped individual who was handicapped and precisely where in the house that person was only by the grace of God did the fire department find this gentleman and get him out of his house safely now the person certainly should not only not have received all these congratulatory uh messages emails accommodations they should have been disciplined at the very least this was a horrible mistake and had the media gotten a hold of this this could very very easily have been on 2020 or Dateline if this had gone wrong and the gentleman had not survived and they played back the tape and they went oh my gosh a very serious change next please all the training in the world is useless if the professional telecommunicator isn't being continuously monitored and reinforced for proper procedures as far as I'm concerned management or supervisor Senior Management could be standing on the operations floor right next to a TC who is taking a call and be totally unaware of how the incident is being handled because the information is going into the headset so only one side of the conversation can be heard so if you are not doing quality assurance you do not know what your people are doing you have no way of knowing what your people are doing next please why quality assurance first thing to assure that proper protocols and procedures are followed because that will deliver consistent quality of service service establish credibility and accountability with our Public Safety partners and highlight strong skills and abilities in your TC's as well as identify weaknesses minimize risk the defense against liability lawsuits is quality assurance because the quote unquote I don't know that was happening is no defense you will lose that case when there's quality assurance that you could have used but didn't assessing training Effectiveness there's a perfect example in in my p app we had a relatively new TC and she just flew through training and everybody was saying this one's going to be a star she's got everything covered when we looked at the qas we found a strange anomaly on uh domestics she was forgetting consistently to ask about weapons huge officer safety issue the rest of the call was textbook but she forgot the weapons so here is a targeted training issue for a singular employee because quality assurance pointed that out so we can make our employees better in the areas where they need to be uh more informed and better improve employees motivation and reduction in turnover so often TCS are told what they are doing wrong but seldom notified when they have done things well now I realize fully that this is not a profession that gets a lot of eates but quality assurance is one way where you can cons consistently point out those who are doing the things well timely and consistent feedback boosts confidence and performance accolades for a job well done is a significant morale booster now unfortunately turnover in TCS is somewhere between 50 and 70% which is a major negative impact on your staff CTO budget and organization as a whole next please several quality assurance standard are required by organizations such as appco the American Society of testing and materials Kalia uh necm the national Center for missing and exploited children of course Mina and NFPA all of them require some form of QA some are suggested some are mandated in the state of Illinois um EMD calls are mandated to be quality assured reviewed but not police and not fire but we're trying to fix that next please the Nina appco 911 quality assurance standard defines the recommended minimum components and best practices of quality assurance quality improvement program within the public safety communication centers recommended minimum requirements for the evaluation of call processing are intended to ensure a consistent effective and efficient level of service this was a very collaborative effort next please by all of the people that you see here it was an eyeopener for me to see the various organizations all come together for a common goal to put this out because quality assurance is so important we had APCO we had Nina we had the International Academy of emergency dispatch we had power phone and we had p app leaders from across the country all working on this so that it would be seamless and easy to use for uh the peaps around the country next please the overview this is a starting point for any size agency uh we have done all the heavy lifting the committee worked very hard to put together something that any agency could use whether they had money or no money in their budget for it um it is absolutely free it costs nothing to use it it is voluntary unfortunately not mandatory um it is very easy to implement it's a complete system for all call types we have scoring templates in Excel for law enforcement fire and EMS both in call taking and in dispatching and then of course it's vendor and product agnostic we wanted to make sure that uh it went out so that everybody could use it depending or not depending on what they have now next [Music] please the key sections there's the general agency responsibilities the qie evaluator duties and responsibilities the component of a QA System review process requirements performance review and evaluation procedures the documentation criteria the reporting criteria the recommendations for improvement feedback criteria and action plans and the objectives of the QA program are accountability determining the areas of excellence and those of concern mentoring motivating improvements regular feedback helping retention and document document document the less is more principle just does not work in public safety Communications next please selecting your evaluators these should be supervisors or managers or possibly if you have a separate qae as in the larger organizations which can afford to have somebody there uh full-time just doing cues the selection is very important the quality Public Safety Professional who reviews telecommunicator work performance and documents an evaluation of the level of compliance with agency directives and standards this person should know your agency's policies and procedures front to back they should be dedicated to your agency's advancement within the public safety Community they should be without any doubt and I cannot stress this strongly enough someone who has Integrity they must be honest fair and ethical no friendship discounts can't be well they didn't know this exactly right but it's a he's a real nice guy or you know that that's that's the guy I went out last night with or um or the flick I don't like that person so I'm going to judge them more critically that is unacceptable and of course they need to exhibit professional attitude in Behavior as well as performance your quality assurance evaluator is expected to be thorough consistent and objective review of recorded calls and dispatches and rate them on the QA forms the evaluation should be combined with reviewing CAD records and notes to confirm that all the necessary information was requested collected and entered accurately the qae is to provide timely feedback and he or she is responsible for generating and disseminating QA reports as defined by your particular agency next please okay as starting point uh we are recommending 2% of all calls now that may be overly burdensome to an agency that is large um so they may need to uh decide that it's going to be possibly something uh less arduous than the 2% I mean if you have 700,000 calls that would be 1,400 QA in a year and that may be two burdensome for someone who does not have separate qaes um so when you're setting this up you want to select a random call or dispatch for evaluation if it's a hangup or other type of call that won't produce useful QA results you need to select another one um however if the qae should never manipulate the call process by disregarding a call or dispatch that has gone wrong to secure one with a better score because that will undermine the entire process paap agencies under the normal business day should review about 2% but again if that's overly burdensome that they can decide on a more realistic uh level of case review the catastrophic loss uh if you have exceptional uh loss of human life or significant dollar amount of property then that's something you probably want to review as well not as random selection but as a a defined and dedicated call also the high Acuity emergency situations that require an elevated or atypical emergency response such as uh service may involve large scale incidents that result in significant loss of life or property damage or of such magnitude that they produce high stress reactions and telecommunicators and early emergency responders as well as the press and that may be a key if you think the Press is going to be looking into this it's something you want to uh review before they're knocking on your door or calling you on the phone next please okay setting up your uh forms again we try to make this as simple and objective as possible the answers to questions are yes or no refused or non-applicable so you determine the uh the value point for each question next slide please thank you for each question the more important the question the larger the point value should be each category is scored separately based upon the percentage of total points for that group a target of total score for each evaluation should ultimately be at least 90% though you may start with a lower threshold such as 75% and work up to this Target in steps of two to 5% increases every three months until you hit that that gives your staff time to acclimate to this new process in the beginning you may find the scores to be not what you would like them to be but you want to give them time to improve and if you have the feedback they know what they're doing right they know what they're doing wrong it is highly recommended to thoroughly explain the QA process to your staff members prior to implementing the program I cannot stress that enough don't want to just drop this in on them you want to show them what you're going to be uh evaluating how you're going to be evaluating it so they have a sense of what their responsibilities are the program will be better accepted when you start out selecting calls or dispatches that have positive results so that the telecommunicators do not become apprehensive of the QA process this is the one and only time that I would recommend that you select the calls not necessarily in random fashion the first uh experience with the QA system should be good one so you may have to look through a few calls to find oh that's a good one I don't suggest manufacturing a call just start out on a good foot where the TC is getting a pat on the back so sort of worm instead of cold prickly next please okay setting up your forms the evaluation result categories you have exceeds require Ms so that excellent performance is promoted and noted promptly and recognized meets requirements they're following your policy they did what you expected them to do or needs Improvement and you want to have a prompt review turnaround and possible coaching where necessary and needed the QA review identifies calls or incident processing issues that need Improvement an unacceptable QA score below the board should be signed by both the supervisor and the recipient it is very helpful to have the TC review the audio of the call taking or the dispatching and grade the incident themselves this methodology assists in making the conversion with the supervisor to be more objective and they more more productive dissemination the TC should always receive a copy of their completed QA evaluation as soon as possible time is of the essence in making QA process a valuable learning tool the quality assurance standard recommendation that the original evaluation should be placed in the employees Personnel jacket to be incorporated as part of their periodic annual or quarterly performance evaluations if you did two calls per week times 52 weeks for each TC that would give you 104 qas for their performance evaluation at the end of the year making it a far more objective process than a subjective one next please calibration I cannot stress how important calibration is whether you have one qae or whether in my case I had six that happen to be the supervisors uh they each had the responsibility for doing quality assurance on their shifts QA evaluators need to be periodically assessed to ensure that they are being fair and Equitable in their evaluation techniques it is very important that management periodically review their evaluation ations to ensure fairness and consistency Senior Management should select a couple of call taking and dispatching incidents for review grade the incidents and then ask the QA or QA to do the same if the scores do not match discuss the discrepancy with the outlier how did they come to that uh conclusion and remind the evaluators that there is absolutely no room for opinions and friendship discounts the questions are objectively answered with either yes or no never sort of kind of or maybe or the person's having a bad day I'm going to give them a pass I'm going to say they they they did what they didn't do this will totally undermine your entire process so it is so important that not only do does Senior Management periodically sit down with the qaes or qae and say okay here grade this but randomly just go in there and grab one of their uh uh reviews and do it yourself and say okay this is done correctly or need to talk to this qae and get them right on the track where they belong calibration holds your team accountable with regular calibration sessions the participants should be of course your QA your supervisors which may be one and the same your managers and possibly even your trainers uh calibration frequency I would certainly start in the beginning with weekly and then once you get everybody calibrated and they know that Senior Management is checking then you could handle it monthly and now I will turn it over to Patrick uh talking about uh improving the consistency the fairness the productivity and the impact of your QA proog program um so wanted to started off first by uh going through a a um a story I was talking to a communication supervisor and asked them to kind of describe the their typical QA workflow um how that looks like and some of you may nod your head as I go through this but uh first you know they search through the cad records and then an emergency pops up uh and as he kind of described he said then uh we begin searching in in the logging recorder trying to find the the calls uh then getting distracted again then return to the search in the recorder and then uh they would burn their their recordings uh to a CD and send those uh to Quality Assurance then the qae the quality evaluator would score the call uh in in one of the templates in a spreadsheet and then the supervisor would receive a print out and the call taker would receive instruction often you know two to four weeks uh after the actual call took place so some of you may be nodding your head um that this may be similar to your workflow but it is common to what we see today so just wanted to go um talk about nine of the kind of benefits of using a QA software tool um again it's it's important to note that uh these tools you know there's some fantastic uh spreadsheet templates available through the standard highly recommend starting with those um unless you have a good foundation uh in your QA program you know the tools will just uh they'll they'll help you improve the efficiency and the effectiv of your program but you really need a solid foundation um so it's just important to note that you don't need these tools but these just help with the efficiency so one thing um the first thing important to note is that uh the QA software does work side by side with any logging recorder so what we do is we record a copy of the console audio and then you get to choose up to three monitors you like recorded as well so it's a total pixel size it usually um is about three monitors so it could be the cad screen uh the mapping screen um and another screen and you'll actually see that when you play back when you're evaluating the call you actually get to see the screen video it plays back like a movie so you can see how uh call takers are navigating through the cad system uh and so forth so um gives you another perspective and it it will continue recording screens even after during the wrap time even after the actual call Audio uh has ended then this is the real Secret Sauce um this is what really helps improve the efficiency is what we do is as we're recording the console audio and Screen we actually tag incident information from the cad system directly to those recorded call so things like incident ID uh incident location the incident type is obviously a big one uh for quality assurance right because there's these high Acuity High uh priority calls that you might want to evaluate 100% of those types of calls or maybe in your jurisdiction you're required to do 100% of things like Cardiac Arrest or 100% of domestic violence calls now you can easily just search by those um those incident types or um you know a lot of people also just use this to find incidents quicker searching by incident ID we'll go in there and just type in the incident ID and pull up all the calls related to that incident uh so anything that shows up in the cad system uh we just look at a copy of the cad database so we're not Direct in integrating directly to the cad which just a mirrored copy of the database allows us to passively capture uh that information while we're recording the calls so virtually any CAD system uh it works with uh another thing is that we've built in uh and and these Mo almost all these capabilities were designed by our customers so they said hey we'd love the appco Nina standard QA forms in this software so we built them uh out of the box uh you can start with those you can um tweak them adjust them change the point scales the ratings and so forth um but at least you have a great starting point there's you know six fantastic templates that are provided in the standard there's a you know the EMS for uh uh call taking dispatching law enforcement and fire both on call taking and dispatching so you have a nice starting point there and then using that Chad data or just a random rule right because it could just be a random rule of 2% of all calls um you know that weren't hangups maybe they're over 30 seconds long um you can create business rules in the background to automatically schedule the calls for either your qaes if you have dedicated qaes or supervisors uh maybe you have a you know a team of Supervisors that that do QA on on on other teams uh first thing in the morning they Hammer these out essentially what they get is a to-do list they don't have to go look into the recording system to look for the calls it'll automatically serve them up it'll tell them the type of call if it's a rule that's selected it maybe it's a high priority uh call it will it will let them know that as well if you'd like and then it knows what type of call it is so it will pull up the appropriate QA form so it'll pull up uh either you know the the the MS the fire law enforcement or you may have one just for non-emergency customer service calls some agencies have that as well uh it'll pull up the form along with the audio in the screen and just not having to go into the recording system to find calls uh that'll easily double the evaluator productivity this point I'd like to share a case study so um this is uh Hamilton County uh tattanooga Tennessee um Tammy chamber runs the quality insurance program over there she does a fantastic job you know we used to struggle to perform 90 QA checks a month uh and now since they've implemented uh the QA software and and the business rules to automatically schedule call we're now at 1680 QA valuations per month and and keep increasing so um this is you know more than two times 18 times um of the number of evaluations they can perform each month which is significant uh and I'll show you kind of how they did that so um they're a larger agency with 130 telecommunicators answer over 2,000 calls a day on average and um they do uh respond for law enforcement fire and EMS so they're now at a point where they're reviewing at least 5 to 12 calls for dispatch it per month and this kind of the secret sauce here is uh we put a business rle in place that says you know we have you have 150 different cat incident types right and what we want to do is we want to look at that incident type and if it's EMS bucket it into the EMS bucket if it's a fire type we'll put it in the fire bucket and law enforcement put it in in that pocket and then what they what they do is they they like to um their goal is and what they're doing is 7% of EMS uh 7% of fire 3% of police and then a smaller amount of non-emergency calls and those automatically get scheduled uh to the appropriate qaes uh for evaluation so um as you can see you know you imagine having we kind of went through the work for having to go in the cad system find each of these types of calls we just take a lot of time um just finding the calls but now they can focus on more on the actual quality assurance evaluation and the coaching so a big uh portion as Cheryl mentioned of the of the standard focuses on um delivering feedback in a timely manner right so this is critical uh so that people aren't making the same mistake over and over again so if you're only you know doing a few quality assurance and but you're not coaching till the end of the month you know two things happen one is they don't always remember those calls and it's not as significant of a coaching impact but the other thing is they could be making the same mistake uh over and over right until they get that coaching so um by you know chery's suggestion of doing you know a few a few calls or a couple calls per week is a great one uh and it gives more consistent feedback one thing we've we've developed with our customers is these little tickers it allows you to put metrics um you know quality assurance metrics or other metrics it's number of calls process time to dispatch speed of answer right we're we're Gathering you know that those metrics from either the phone system the console um or the um the cad system and you can put them into these tickers but the key one here is the quality assurance score and as soon as an evaluation is completed in the ticker um the the call taker dispatcher is able to see their most recent QA score and they can click on it and they can then listen to um the their evaluated call and see the comments and everything and if they'd like to they can provide comments back so they can provide you know comments back to the evaluator maybe there's some sort of dispute or or something they can do that and all that is documented so when you print out the the form or put it into the records uh all those comments are there as well so it really helps with documentation and then of course the timely feedback so they can figure out what they did wrong and uh and and and and make those adjustments and Corrections so this uh this uh slide talks to Cheryl's last point on calibration it's it's critical that that um evaluators are calibrated so if you have you know either supervisors or qaes doing the evaluations or some agencies even allow for um call taker dispatchers to do some self- evaluation as well with with can't be valuable um this allows you to essentially select a call and it's not going to necessarily won't be graded for an individual call taker dispatcher but you use it for um this calibration session everyone gets that in their to-do list they now have to calibrate that within it'll tell them the due date of when it needs to be Cal scored by and um and then it allows you to pull up reports to see how uh people are calibrated the different values not only on an overall basis but uh by different segments within the QA form or most importantly uh the individual questions right because it's great you know you could have a straight line across here everyone's calibrated overall right at let's say 90% but if you look at the individual questions are there discrepancies between the evaluators and it's really important to look at the question level um so provide a really robust reporting around the calibration as well um built-in coaching tools so uh recordings and quality assurance evaluations tend to be fantastic uh fantastic uh tools to use in not only new higher training but ongoing coaching so we allow you to redact calls you know you can uh use the screen recordings the audio recordings as training clips and um and redact Parts you know where there might be some sensitive information you don't want to share and then this is a really Nifty one that was um came from feedback from from a customer is hey um you know we have these tickers we' love to be able to to send messages to the tickers and um and so what we allow you to do is add an action to each question in your QA form so you can add an action and an action might be just as simple as a message right that's kind of where it started a message that pops up in the ticker that says hey you forgot to do this on your last QA that was just created please remember to do this next time and they have to click okay and acknowledge it they can't say you know you never told me to do that because it it's it's automated right the system's automatically sending that message out whatever message you type in below the question will automatically get delivered it could also be a video a PDF an e-learning course for those of you who are maybe doing some e-learning it can trigger that and send that out as well so a great tool to standardize kind of the coaching um across shifts across supervisors this will automatically happen it's not to be used as a replacement for live coaching but it's a um a supplement to that and then um in our latest release we've also included a kind of real time dashboard so uh if you take your your uh ey to the the right here you see flag call schedule for QA review um and some people call this a dashboard I call it a workspace this is where you can start evaluating calls right from here you log in here's your to-do list but we can also since we are gathering metrics from the cad system and you know the phone system you can also display uh any type of metrics you trended over time it could be you know calls handled or you know QA scores or speed of answer time to dispatch you whatever is important to your agency some agencies even put these up on as wallboards um kind of on TV monitor for others to see as well so uh really nice customizable reporting you can create a dashboard and push it out to all supervisors and they get the same one uh relevant to their team uh and to to to what they focus on so not everyone has to create their own you just create one and push it out and then kind of the the last uh capability uh that we created and and we work with a PRI dispatch on this uh they have uh a a team that does um evaluations for other agencies and um and they were looking for a way to improve efficiency so we work with them just to put a playback button within their software which will automatically play the file um I was talking to someone there prior dispatch just at the appco Atlantic show here re this week and um they said you know on average you know it could take up to you know 14 minutes or so um to do the evaluation and a to that could be just finding the call in the system so uh by kind of removing that time you know and just having them play the call back right from Aqua that's a great little app that we provide um for any of those who may be using PRI dispatch for either EMD Andor a police or fire and um to kind of close it out wanted to just uh put this quote up from our customer Matt Still well he's a colleague of mine from the city of edman a comprehensive QA process can not only expose if someone's not doing their job right it can also expose whether or not the policies we have in place are in line uh with what we need to provide good quality service to all citizens right um and Matt really stresses you know uh doing quality assurance on all uh call types not just EMD because of um you know that's what citizens expect they expect good service on all of the call types that we're handling so thank you so much and thank you Nathan and Cheryl sorry I was on mute there for a little bit but U just really appreciate everyone's time and I see there's a lot of great questions uh that have come in and um we'll go ahead and uh and start answering some of your questions here so um so what is the standard what is the standard Rel in April where can I get it um so it's available if you just type in U appco Nina quality assurance standard uh in Google um you will find it on the appco website and you can download it right there in in PDF format if you'd like um oh sorry about that it's also available on the Nina website um with the uh spreadsheets in Excel form in Appo they are in the PDF form perfect that's a great tip though so if you guys want to download the the spreadsheet templates um which will you know kind of help you get started or you can start that use those as a starting point um you could visit the Nina site for for that as well so thanks sure um we will be sending you a copy of a recorded copy of the webinar and also um copy of the PDF slides uh that you can use as well so you'll receive those early next week that was one of the questions uh so this one's for sh do you think it's necessary to have Sops in place prior to starting a qaqi program that is a great question and the answer is absolutely yes you can't grade someone or you shouldn't grade someone if the TC doesn't know clearly what your expectations are in the um the new standard there are sample criteria for scoring um and anyone who is interested if they will email me um very clearly do they want law enforcement do they want EMS or do they want fire or do they want them all I will send them the whole packet in Excel along with um the scoring criteria and how to customize the actual um spreadsheets so that they fit with your agency not only what the questions are but you can also change the numeric score for each question things that you think are more important of course have a higher score less important a smaller score thank you Cheryl and I know we're at the top of the hour but there are some fantastic questions that have come in so we'll stay on for five more minutes for those of you you can stay with us U our this is for Cheryl as well our agency handles 400,000 calls annually we're currently writing and updating our Sops and and QA program uh the 2% of calls seems overly burdensome for us how do we determine appropriate number of percentage of calls to evaluate who's doing the evaluation is it a qae or multiple qaes or are they um offline supervisors would be my question yeah I'm not sure we'll get the answer to that right now but however go ahead try 1% okay thank you try 1% for the larger agencies where it's overly burdensome uh try cutting it in half and see if that's possible and if that's not possible you'll you'll have to make it something that works um and that's an individual agency uh issue the 2% is a recommendation yeah I I would agree with that so I think you're right on I mean if you can do 1% right whatever that ends up being um at a minimum at least one a week right so there is continuous feedback um but you definitely want to um see if you can't get up to 1% and one a week would be one a week per employee would be the absolute minimum that's the minimum agreed so and usually there's two parts to the call you've got the call taking and then you have the dispatching which may or may not be the same person so actually from my perspective it needs to be at least two per employee per week at an absolute minimum great thank you Cher uh on the QA forms do you calculate the overall score based on an average of the section of scores or as a percentage of total points earned or total point value total point value and it calculates it it's actually all of the calculations are in the spreadsheet for you um and again the explanation document tells you how to standardize that but you'll get a percentage for each of the sections and then a total percentage great thank you um have you experienced or handled any Union objective objections Joe no uh we were a union shop and we were Union shop for 20 some years um basically we do not have a um performance-based raise system unfortunately but it is what it is is uh so no did not have any uh backlash from the unions but again you need to explain this to the staff before you implement it and to go back to the first question yes you need to have your policies or your criteria clearly laid out so that your staff members know what the expectations are and again when we started back I don't know 10 12 years ago um we started at the 75th percentile and there were some people that were just right there at the 75th we said okay we're going to run this for 3 months and we're going to show you um and a turnaround was very important here here's what you're doing uh right here's what you need to work on here's what you're excelling at after 3 months it went from 75 to 80 and then 3 months later it went to 85 and then 3 months later it went to 90 we also uh in a in a pretty elaborate uh frame every month we put a list of every TC that had a consistent score for every QA at 96% or above because those are your stars and you would be surprised they want to see their name up there so again there's a there's a way of um giving somebody an Accolade fantastic thank you uh there's several questions on um uh which CAD systems does the VPI QA software integrate with and um uh so we have probably uh two dozen that we've integrated with um there's not one we have mailed to so far so um we're just looking again at a copy of the database which typically can provide be provided by either it or um or the cad vendor usually will help out with that as well so um we'd love to work with you on that and and see if um we might be able to help you tag CAD data to the recordings um and then here's a question so what are your recommendations Cheryl for working supervisors that are online throughout the entire shift you have a very tough job absolutely in my mind working supervisors are the middle of The Hourglass you got Senior Management above and you have the TC's below and everything filters through the working supervisor on the way up and on the way down so CS off to the working supervisor um you are Paramount to the wellness of your organization try and find um a little time somewhere where Senior Management will allow you to unplug so that you can do your QA and if you do you know one a day then at the end of the week hopefully you will have uh some of them done or that you could plug uh unplug and work at uh another console position that's quiet uh while you do your QA but no it is not an easy position to do and certainly if you had it automated um it would be much more uh easily handled than if you were trying to do it manually one of the things that I was most impressed with VPI and I'm not easily impressed uh was their ability to work with any recording system you had because initially I said when I heard about their their software I thought well that's great but people are not going to be able to afford a new recording system and I was most pleased to hear that this uh capability would work with any recording system that you might have in your P thanks sh um we'll have uh finish up here with one more question and then what we'll do is we'll actually write a blog article with the questions and answers uh we'll detail out the answers and send you guys a link to that article along with the PDF slide uh and the uh the webinar um so this one is it's a common one that comes up as well we're using uh IAD IED um for EMD medical and uh can we use this software for police and fire uh and absolutely we have a lot of customers that do do start out that way um you also can add the um that playback button uh in the uh in the aqua software if you use Aqua so you'll be able to speed up the workflow for the for the EMD and then you can um just so you are evaluating police and fire efficiently you can um use those formed within the VPI software um until uh perhaps you transition one day to all I IED um if you'd like to it's up to you so um that's a great question and uh there's some more questions that come in but at this time just wanted to thank uh Cheryl and Nathan for the Fantastic information and just the passion that they have it's so inspiring um and we're really excited to uh so you'll be seeing probably uh 911 QA every day everywhere you go uh we we have nice lapel pens and things that we're going to be handing out at conferences and we we invite you to help us spread the word and um and and join us in our passion for um you know having every piece app uh have a formal consistent quality assurance program uh within the next 5 years and and thanks for your time and again we will follow up next week with um more information if for those of you are interested in either learning more about the services by Denise Amberly Foundation or uh our software tools we're happy to help you out and look forward to partnering with you thanks so much and have a wonderful afternoon