Sign Montana Terms of Use Agreement Later

Document type sign terms of use agreement montana later

hello everyone this is carol sullivan from content club many thanks for coming and you've made it to the webinar about seven signs that your linguistic quality program could use an upgrade i'm giving you all some time to join and we'll start very shortly i hope you're all doing great today definitely look good just kidding i can't really see you um okay maybe we'll give it a bit of time and we will start shortly thank you for joining for those of you who joined later my name is kirill you're definitely at the right place if you made it so far just waiting for a couple more people to join us and we will begin all right by the way we will be making some good use of the q a feature and the chat feature today at this webinar so please make sure you know where to find it so i'll talk about it a bit more in just a second we're almost ready to go by the way this is being recorded so if you need to drop off early no worries so if you will be watching this later on no worries as well you will be able to get the entire content that you came here for all right the people are coming but it's time to get started hello everyone once again my name is carol solov and welcome to this live webinar on seven signs that your linguistic quality program could use an upgrade so let me briefly walk you through how this will actually go today so first just to make sure that you are at the right place who is this webinar for this webinar is mostly for localization teams at what we call the translation buyer side but if you're actually an lsp don't despair if you really want to run linguistic quality programs for your clients or maybe if you're already running them it might be useful for you as well what kind of stage will we focus on today um maybe you have already started some kind of a quality program you have something in place or maybe you're just thinking to do so but are not quite sure yet either of those i do hope to share useful insights with all of you today um in terms of the flow this is scheduled for roughly 40 to 45 minutes and then we'll go into actually tackling your questions live so we'll look first at why localization teams rely on linguistic quality programs at all if you don't have anything like that yet that might be interesting we'll spend the biggest part of our talk today to look at different symptoms that you might need to have a formal linguistic quality program or that you might need to work further on improving it or improve it faster and finally i will share several strategies on how to start addressing some of the symptoms we'll focus on today before we start though just a couple of words about myself my name is kiriel soloviev i am the co-founder and ceo of content quo and we'll talk about the company very briefly in just a second i've been in this industry for over 18 years right now so some of you might have already met me before if yes i'm very glad to see you if not i will just share a couple of things about what i have been doing all those 18 years uh i did quite a bit of travel especially lately you might have seen me popping up at different conferences like thousand lock world and whatnot mostly in europe but also a little bit in asia and in the states as well so traveling is definitely a big part of my life or it was before 2020 and the pandemic has hit uh i actually do other things in my spare time believe it or not i like digital photography especially black and white i do go out and bike rides when the weather allows that it doesn't happen that often because i'm here in the north eastern corner of europe in italian estonia and it's kind of difficult to bike throughout the year but i do that when i can over the weekends i also love to hang out with really smart and creative technology people on hackathons so helping them build exciting prototypes for new software solutions to some wicked problems um i've done my share of public speaking and conference presentations as well and i also like to play different kinds of musical instruments most of them quite badly to be honest so that's me anna what about the company what about content quo we are a technology company headquartered right here in the fabulous medieval tallinn yeah we started five years ago uh with my co-founder alexander on a simple mission uh we've both been in this industry for a long while and have been struggling with lack of transparency around translation quality and localization quality and linguistic quality and we didn't have anything better to do other than to actually launch a startup and start building software to fix it uh it's been a quite a ride uh since the time that we've launched in 2018 very fortunate to have worked with some of the smartest localization quality managers and we've picked up quite a lot of insights from them some of which i am actually going to share today so let's get started but first let's briefly talk about you i also want to know who is on this webinar today very briefly use the chat window if you can and let me know are you on the buyer side or on the lsb side uh and what's your role what stage is your quality program currently in is it early stage is it mid-stage or is it late stage and super brief what's your biggest pain point with linguistic quality in your organization or in your company if you could just use the chat window and type a few things out it would be amazing and i will give you a bit of time to do that if you can find the chat button you might need to click the more button in zoom first which is hidden under the three dots menu and then you might be able to go into the chat so yeah super briefly your role the stage like how far along was managing linguistic quality you are today uh early mid or late very mature program and if there is anything in particular that bothers you let me know i would love to tailor this to to your needs a bit better if i can one have time to do this in full but i will try my best to make it as relevant for you as i can uh but yeah thank you for answers it's great to see that uh just to clarify when i say program i don't mean software uh i i'm talking about the process and the approach and methodology and everything that goes around it so not not that program not the software type of program but rather the solution or the process type of program cool early stage uh not very easy to use and so on and so forth yeah any other thoughts please keep them coming into the chat window and i'll start moving along with some of the ideas i wanted to share today uh first why are we talking about linguistic quality uh when i was trying to come up with a headline for this slide i thought about quality essentially being invisible to most especially outside the localization industry unless it's really bad have any of you heard about the recent amazon launch in sweden if it's a yes just type that into the chat window i'll talk about it shortly but i'm curious how many of you have actually seen that piece of news i think it happened last week yes yes okay cool uh so yeah for those of you who don't know exactly what's been going on basically amazon has rolled out its uh online e-commerce store in sweden in the swedish language of course and they faced certain challenges with how the names of certain items have been localized by apparently their machine translation engine so this is usually the only situation when linguistic quality makes the use and you know the amazon case is fascinating uh knowing how much work and how much processes and how many team members in amazon actually work in localization uh but let's not focus on that too much but basically that's how everybody knows right when something goes wrong now when i say a linguistic quality program what do i actually mean by that again i'm not going to teach you or everything that i know but just super briefly this is when you inspect the quality of your translations usually at regular intervals and gather data about how good or bad they are might look something like this a bunch of metrics you can use this kind of programs to look at your machine translation engines you can look at how your in-house translators perform you can look at how your translation vendors or lsps perform right the crux of it is being data driven not opinion driven but actually trying to gather some useful kpis about the quality of your translation from the linguistic standpoint right now in terms of benefits why the teams i've seen and the teams i've worked at back in my old localization director days i used to run a program like that as well uh why do they do that most commonly number one just really figuring out what the hell are my translations like let's face it once you go beyond a certain scope of languages it's very difficult to really know how good or bad your localized content really is so the first reason why teams put this kind of program in place is to get transparency what am i actually shipping to the customers to the users and to the partners of my company the company i'm driving localization for number two very useful thing once you have regular flow of data about your linguistic quality you can actually use it to make your translations better to continuous improvement kind of thing and of course feedback is the central element to that yes you need to know how good or bad things are but the sheer act of measuring something as you might know often helps it improve so yeah being able to achieve continuous improvement in the quality of your localized content is another popular reason why teams put this kind of programs in place managing downwards to your vendors to your in-house linguists to your individual freelancers whatever your supply chain is doesn't really matter the principle is always the same last but not the least you don't only have your own supply chain to manage right not just your team not just your lsps but sometimes other people in your company might be interested to know how good is localization doing management teams and executive teams usually speak the language of data really well and if you can speak the same language to them and explain something interesting about your localized content chances are you can actually move faster and achieve the change you want to push for so managing upwards is another important reason of course this approach has its limitations when you use quality i'm not going to talk about this today but in general this is the direction this is where data driven really helps at least it did help me back in my career i wish i did more of that question to you any of those reasons seem compelling to you or maybe you had your own rationale for starting your existing quality program some time ago if so what was the main driver out of those three or maybe you had another one why did you think it's compelling why are you thinking to start a program like that use the chat window just type a few words hit the send button why are you thinking about a linguistic quality program or why did you decide to put one in place if you already have it running oops i was too early ah i'll need to go back for a sec sorry about that continuous improvement that's great anything else any other drivers uh you had for implementing like a data driven problem or why would you consider to do that what's the the rationale that seems to drive you monitoring and controlling yes honestly when i was a localization director back in the day i really hated not being in control i was a bit of a control freak myself and just the sheer knowledge of not being able to be sure that my team was doing a great job was driving me nuts i wanted to be in control yeah knowing your languages i do not speak have actually good quality or not exactly right no way of finding out so you need to do something to be able to sleep at night especially when the number of languages grow so this is a major pain point now let's start moving deeper into this direction seven signs of problems that i have seen that might allow you to figure out that you need to start thinking about a linguistic quality program you don't have today to putting it in place or maybe about improving your existing program and getting the most out of it i'll talk about seven of course there are many more but we need to start somewhere right so number one or symptom number one have you ever had this feeling especially if you're new to the job i know some of you have recently joined their current companies where you are bombarded from all sides of the company internally and externally by bad feedback about your localization i know because i've been there myself when i when i joined my last localization by a role japanese was a major problem if you feel the same do let me know in chat as well but i think japanese is always a major pain point in terms of quality so yeah just you know starting to have things dropping onto your desk where you know you need to act but you don't know exactly how to do that is definitely an early warning that you might need to be thinking about a formal linguistic quality management program that is based on data the reason for that is simple if you get a negative feedback it's already too late you should have done something before the feedback lands on your desk right if it's still there then yeah maybe it's time to start considering pro active measures number two at content quo by the way we see number two a lot uh so this is the opposite of number one having no clue about the quality of your translations uh and i think some of you have mentioned this in the chat uh the one kvat what i've noticed is that oftentimes you have your high priority languages what we call tier one localization where you might have some idea right maybe you speak the languages yourself maybe you have team members that can cover them that can give you a clue right even before you have a formal program in place so but anything that happens beyond tier one is tara in cognitive you have no clue and what you're doing you're basically sitting there and contemplating hmm am i getting bad feedback on this language this month or not uh and if you don't get it it doesn't mean you're lucky it basically means that people are just too lazy to tell you that things are bad but it doesn't mean that there's no need to do anything about it um any of you here facing number two i i find that this is extremely common actually just simply not having enough information about the work that we are doing in localization is it good or not if you face number two just type two in the chat or type yes yeah okay yes i know i've been there myself i used to have like the top three languages for my old company coverage i had linguists for japanese german and russian which were the top tier languages for that company back in the day but anything beyond that was very difficult to see through so uh yeah like i said for me this was super frustrating number three oh i love that one i've been guilty as charged with this this is when you start to bring up the topic of the quality of your translations to your management team but they just give you a blank stare just like this lady here what are you talking about i don't understand what do you mean the quality of translation can it be of different quality aren't we paying for premium translation every single time please explain that to me uh i've had a bit of that in the past but yeah the the thing is even among us inside the industry we cannot really agree of what quality of localization or linguistic quality really means so chances of explaining it to the outside world without resorting to some kind of data are tiny just like that so yeah it's a difficult place to be in especially when you already feel you need to change but you cannot justify this to the management team number four hmm this happens a bit further along the the progress of rolling out the the the linguistic quality program um this is i i'll tell you my story from back in the day how number four went uh for me um so i used to purchase localization services uh was a significant budget back in the day i used to be this localization director for a software company and um at some point we've engaged some of the best lsps in the industry i know some of you must work with them today they're still around and doing great uh and the part of the package t ey offered was actually checking 10 of all the translation volume that we gave to them and passing it through a formal linguistic quality assessment and then sharing the data was with me and with my team with the team members who were in charge of quality and vendors and the like this is a very common setup or in the high end of the translation services so maybe you have something like that or maybe you find you have to resort to this but my problem was that uh over time i was very happy at first that i'm getting some structured data on my language quality but over time i actually grew very weary of this process why because for every tiny little thing for every tiny little question that i wanted to answer like figuring out the direction do we spend more in this language do we spend less do we work harder or can we relax a bit for every tiny question like that i my team actually had to go back to my lsp and extort the data from them and they would only give it to us once per month why not because they are greedy but because the infrastructure and the processes they run we're just not built for answering questions quickly so yeah especially if you're in a smaller and agile team or a continuous localization process like this can be super frustrating so at content quote we talk to teams who do continuous localization and that's what they say oh but like it's a very slow moving i can't fit a linguistic quality program into my process usually that's because the the way that the vendor is trying to put this together for you is kind of wrong um so yeah that's how it manifests itself number five my favorite one um when were you on board new customers at content quality this is super frequent or maybe you were guilty of that too uh i was talking to to a localization manager the other week and they are at an early stage of their quality program so we talked about like how to shape it and basically how to run things before they commit to something specific and start to scale and uh one thing that that stuck is like they told me this like we are thinking to check the quality of our translations once per quarter once per quarter right that's four times per year now let's think about that for a second we are still talking about translation right human language text maybe creative maybe not creative but it's still not formalizing all the methods in the universe that currently exist for assessing the quality of your translations are kind of subjective because they rely on human judgments or proxies for human judgment so how far along on the objectivity axis do you think you can get if you only check it four times per year like do you think that's enough to actually gather solid statistic evidence about how the quality of your translations is changing like does four data points actually give you you know statistics like is it is it real stuff or are we kind of deluding ourselves i wonder what your position on that is to type it into the chat like do you believe that four data points per year is enough to get a solid grasp on how good or bad your translations are especially if you're translating like daily i'm curious what do you think four data points enough or do or maybe we need a bit more than that let's type into the chat window it depends well yeah i agree if you only do four translation projects per year absolutely four assessments kind of covers it very nicely and you're hitting like hundred percent of uh of your project my part generally speaking no yeah i uh generally speaking i would agree any other opinions yeah all right keep them coming uh i'll tell you what i believe right what we at content quote believe like anything that is below at least five data points per month is kind of bordering on pseudo statistics to be honest just because how this whole process is laid up and what kind of methods we use to find out and measure the quality of our translation your mileage may vary we work with some organizations that do dozens of assessments per day and they need to do that because their volumes are quite significant so uh at that point yeah pseudo statistics finally stop but let's move on i still have two other things uh as we progress along the the maturity of your linguistic quality program uh this is of course my favorite one uh once you actually ramp something up once you start to do some kind of measurements of your translation quality on hopefully some kind of regular basis like this is what sometimes happens if you see that think about it maybe there's a better way to do it so in order to run this process continuously and repeatedly what happens is like at some point those pesky axle spreadsheets pop up and you need to enter the data there you need to collect the feedback there you crunch the numbers there and you send them by mail from party to party and this goes on from your team down to your lsp down to fair lsp down to lsp of their lsp down to the individual freelancers and then back again into the supply chain and inside your team basically this goes on forever and at each point you're actually wasting highly paid time off your internal and external team members just basically to send files around sounds stupid right well i've seen incredibly advanced teams in all other aspects of automation their translation flow that still do that so if you got to this point where you actually have some kind of a problem running ask yourself maybe this is one of the symptoms that you need to pay attention to at content quote we see this all the time basically and the last one ah this is the most difficult um this is for the late stage linguistic quality programs among us this is when you already have done an amazing amount of work to start getting the ball rolling you're gathering data by the boat load but you still don't understand what's happening uh i i think there's there's some kind of widespread misconception that we basically need to just start gathering the data and then the insights magically flock into our amazingly bright heads right well no this doesn't work this way i often see teams that actually collect lots of translation quality results data points metrics but yeah they still have no clue why things are going the way they're going or even don't know which way their translations are going for different languages for different product lines with different suppliers for different production processes like mt versus human translation and so on and so forth so this is really the the biggest pain the way i see it because if you got so far you actually expected to bring your real insights and in many cases it still does not so yeah these are the seven symptoms maybe from the earlier stages of the problems that we've seen to maybe the latest stages that we've seen before we go into ideas on how to start working around them i want to ask you a question so out of those seven what's like the pain point that appeals to you the most what's the symptom that you feel you might be facing right now you just type in the number into the chat so i'll go through them uh super quick number seven was having data but no insights number six was spending and wasting a lot of time on your axle based on email-based process number five pseudostatistics not having enough data to make reasonable conclusions number four having to rely on your lsp or your suppliers for you know everything in regards to quality number three not being able to explain the quality problem to the management team uh number two just not having a clue about quality at all apart maybe from the most frequent most important languages and number one a ton of negative or just unclear feedback that your translations might be bad let me know how it looks like for you and i'll give you a bit of time to respond and we'll get back to my suggestions oh the chapman is very active six seven six two four one six yeah number six is a clear winner i know it's so weird when i've started in this industry uh working or as part of localization quality programs in 2005 i think we were using excel spreadsheets when i started this company in 2015 the companies were still using excel spreadsheets and guess what it's 20 and we still haven't been able to replace all of the excel spreadsheets yet but we are getting there uh thanks for sharing uh how you feel now i know this can all be incredibly frustrating i want to offer some ideas on how to solve things there's a bit of a trick like uh the symptoms repeat many times over like they are shared the solutions to those symptoms are maybe a bit more individual for each organization for each company for each localization team i'm sorry i cannot really give each of you a silver bullet today uh i will give instead some general recommendations of what to focus on for different stages of your program early stage mid-stage and late stage right if you want to drill down into more details we can always schedule a call you'll tell me about your specific situation and i'll be very happy to offer some suggestions on how to go about them right now specifically for you we do this at content core very frequently it's completely free so i'm very happy to share what we've picked up over the years and i'll give you a link to our website later on where you could schedule a call with us now suggestions on where to go from there and by the way if you feel a particular aspect of those things is something you want to dig into feel free to send a question into the q a window ideally you go to the q a window so that i can see it easily and don't lose it into the chat so if you want to talk about one specific aspen we will have time for q a and i'm almost done after the section we can actually look at your questions and listening to the answers is also useful as well i know that many challenges are shared and sometimes what works for one company might work for another now early stage linguistic quality programs basically you know you need to do something but you're not quite sure how to go about it so we call the stage the shaping stage it's like a it's like a pottery um a pottery making process have any of you guys have actually made any pottery with your hands uh type yes in the chat if if you're done i have never done poetry myself but one of my in-laws actually is great at that so i've seen uh the type of work she's doing fascinating stuff uh like it's basically taking something very soft and intangible and then with your hands like you give it some very definite form and then you uh make sure that it keeps that form so early stage quality programs are exactly like pottery right you need to move very carefully in order to give it the right shape because if you don't it will break down later on a bit of a challenge now what to think about number one is methodology how are we going to measure the quality of our translations which way are we going to use to what data are we going to collect about the linguistic quality of the work that's done in your localization program a bunch of options in there you have the analytical approaches like taos dqf and and so on and so forth you have the holistic approaches like adequacy fluency which can be used for machine translation you have hybrid approaches uh you have tons of ways of defining those precisely so lots of things to factor into the decision making process next one cadence how frequently are you going to look at your translation so how many data points do you want to get in order to make better decisions about your translation quality not an easy answer of course uh it depends is the best way to respond to the cadence question but this is really the cornerstone the crux of the decision you know what data together now you need to decide how frequently you actually will do that um next one topic three scope so what are you actually going to measure which content will go through the quality assessment process and which content will not how do you decide out of your entire localization program company why what's worth spending effort and time on inspecting and what might be completely okay how do you assess the risk of poor quality before you actually decide to invest into checking it and gathering data so this is the scoping question last but not the least resourcing so let's face it in order to figure out the quality of translation in detail you need a human expert there is no way around it even though machine learning technology is kind of getting there but it's still at best a supportive measure now what people will actually gather those quality metrics for you will it be in-house team members in-house linguists or in-country personnel perhaps maybe will it be dedicated freelance linguists that you contract on the long-term basis will you engage your lsp into that or will you engage the same lsp as you use today or maybe you work with a dedicated lsp just to provide you insights on the quality of your translation so a bunch of specialized companies some of which we at content call do work with regularly that can do that but there are of course many other ways to gather the metrics and of course all the big lsps are experts at running this kind of program so you have many ways of resourcing your linguistic quality program and at content core we often help teams make decisions at this early stage what's the best that they can do in order to resource when you sum it all together it all comes down into the budget you need to allocate for your program it's not free to measure the quality of your translations regularly far from it it's actually quite expensive you need to make sure you can secure the required investment for running all those wonderful things and forgetting the transparency that you know you want to have and you know that you deserve i'll talk about this one question any of the things you already thought about you that are at the early stage of program what's the biggest concern out of uh methodology cadence scope resourcing or maybe something else right type this into the chat window or maybe even in the q a if you have a specific question you want me to answer and i'll try to make my best suggestions so for the early stage i should let me go back to the previous slide of course these are not all the considerations you need to cover when designing your program but some of the more important ones methodology yeah i agree there's so many options uh earlier just before this call i was actually talking to one of the quality managers at google and we spent 45 minutes uh figuring out ideas about methodology for evaluating a certain type of a localization production process and how to make the best use of the available methodologies on the market and how to combine them for their needs so yeah methodology is tricky anything else bothering you at early stage programs besides methodology resources yeah i know this is tricky there are a bunch of options uh i used to run like a very hybrid quality program back in my day when i was managing localization i had a mix of in-house linguists uh dedicated freelancers and lsps working together on different parts of my quality program it was messy but yeah i can confidently say i've done it all and have some room for benchmarking so if you decide to schedule a free consultation with me i'll be happy to share what i know what i know works and what doesn't work again depending on your specific circumstances um keep them coming uh post stuff into q a if you want me to go into more detail i look at the next stage scaling this is when you've laid the foundation for how you will gather the data what data we will gather how frequently you will gather uh who will actually do that and so on and so forth so this is the the kind of the most critical stage now you know what you want to do you're ready to go now you need to start executing that's the mid stage or the scaling stage at content quote we work a lot with teams at this stage because they usually realize they're starting to hit walls they need better solutions to move forward so typical considerations to solve are as follows number one we know what we want to do how can we do this with the highest possible efficiency how can we reduce the overhead how can we remove the waste from our linguistic quality assessment and data analysis processes right so lowering overhead is like the mantra at the scaling stage the less overhead you have the more you can dedicate to the core of the process which is still in 2020 let's face it having translators linguistic experts look at your translation and give structured objective well-defined feedback either for other translation teams who produce this translation or maybe for the benefit of your machine translation team if the goal of your program is to improve the empty quality second consideration at the scaling stage faster turnaround time i know i want to inspect this piece of translation how little can i make the delay between my decision to look at something and getting an extra one data point in return that's the key question how quickly you can actually run through this measurement cycle and add more data points in your database for your analysis of course the shorter the better and there are some tricks and tactics that can be used here in order to mitigate the the delay and the supply chain structure and so on and so forth ultimately the two key goals for the scaling stage get as much data as we can while still staying on budget of course more higher quantity of data but also make sure that we're gathering data in the cleanest possible way to make sure that it's consistent with itself and when we're comparing you know different types of fruit in our program there's actually apples and apples instead of apples and oranges for instance or mango right so that's the quality of your data and by the way excel is not a good solution for ensuring great quality of data nor of course for faster turnaround and especially not for lower overheads so yeah it's kind of problematic uh the way to go is usually automation and integration right connecting everything to run as mostly as possible getting excel completely out of the picture so that you can actually get to the third stage of your linguistic quality program late stage question for you first before we go to late stage let me know if any of you are looking at scaling already which of the the four things i have described bugs you the most what's the biggest hurdle you're facing when trying to scale an existing program so going from just defining the shape what you will do planning strategizing and stuff to executing repeatedly uh if there's a specific item like from those ones or maybe other stuff let me know last one oh i actually have a typo here i'll need to fix that before i send the slides to you it's not mid stage of course it's late stage so once you've started executing once the data starts flowing as efficiently as possible and the data is as clean as possible this is what you want to be doing you actually want to start deriving benefits from your linguistic quality program which is pretty much hard work right but this is all the previous steps uh they are leading us to this in order to benefit from a linguistic quality program you need to kind of keep working on it getting data in is just the first step but what you need to be busy with is a regular analysis of the data it's fun to measure quality and argue about how good or bad it is but if you don't analyze the data points you've been collecting it's all for nothing you still don't understand anything at all even though you might be giving feedback root causes why did things go wrong in this translation usually it's about going wrong right very few teams analyze root causes for great translations this might be a practice we need to see more of to be honest so finding out the reason for why the quality is not where you want it to be and figuring out how to actually fix it this is the dreaded copper thing uh any of you know what kappa actually is type into the chat window if you can what does copper stand for c-a-p-a anybody knows yeah monica i definitely am not surprised that you know uh perfect alona great job yeah so kappa is corrective action preventive action very simply put when you find a problem you need to figure out one or maybe two things hey how do you fix the problem right now that's corrective action and b how do you prevent the problem from reoccurring in the future right so that's kappa for you and ultimately you do all of those processes repeatedly in order to slowly slowly bit by bit actually get the quality of your various translations to the point where you need them to be but not further than that by the way having too much quality is another type of problem so yeah these are the four typical considerations that we at contentco have seen working with late stage linguistic quality programs at localization departments and localization teams i'm wrapping up now we're pretty much done i will summarize briefly what we went through today we looked at why the hell teams are interested in linguistic quality all of a sudden and i thought about the recent amazon scenario which is a great example of what not to do like how to use quality to get transparency to manage downwards to manage upwards we looked at no less than seven distinct reasons or signs that you might need a linguistic quality program in your organization or you might need to improve it further because it's not where it is supposed to be and finally would look at some high level strategies for the three stages of your quality program early mid stage and late so shaping stage scaling stage and benefiting stage and how to go about you know preventing or fixing some of the typical problems we are now open for q a if you're not happy with how your current program is laid down you need some support in order to define the next steps maybe you think to start your own program not quite sure where to start please drop me a note you can respond to the emails where we'll send you the slides and the recording after the webinar and we'll also send a free ebook by the way probably early next week to give you a bit more insight just reply to to my email and we can definitely schedule a free call to talk about your specific situation in detail and i will be happy to provide advice and insights on what i might do in the stage and you can also do this with our website as well no obligations and no strings attached thank you very much and we're open to questions please type them into the chat window or into the q a window i think at this stage we can do both valeria says that was very interesting i'm glad to hear that any questions any thoughts you can ask me about your your program if you want about the things that bug you if you want or any generic ideas just riff on what i was saying before how would you measure the quality if there are no metrics involved ask sunder ah that's a funny uh tricky question hmm uh i would probably go philosophical on this one and and um ask a counter question but can we say we're measuring something if there are no metrics like can we pretend that we measure if you know if there is no way to measure i i'm i'm not quite sure about the answer but like i i want to give something useful uh like i said we're having uh this discussion with google just before this webinar and we talked about like a next type or next level of metrics that can sometimes be used in quality programs under quite specific circumstances to provide more insights uh we talked about linguistic quality it's great because we can actually measure this ourselves right being inside the localization process before we publish the content for the rest of the world for the readers for the buyers for the players to see but we can also look at what happens when we actually publish the content when we put online the documents when we launch a campaign when we publish our localized app and so on and so forth we can measure the outcome of localization in whatever business metrics make sense conversion metrics or uh i don't know uh product metrics or business metrics like sales it really depends on what content we're uh we're talking about but you can gather performance metrics at least in digital channels of course they are the ultimate pinnacle of measuring quality without measuring quality so to speak but there's a bit of a challenge with performance related metrics a they take a very long time to gather and in some cases you might not have enough of them and in some cases you might not have them at all imagine if you are not distributing your content digitally what would you do it's really difficult to get performance metrics customer feedback is the only proxy we can rely on right but b they come in well really late uh but b they are not self-explanatory they don't give us the reasons for why things are wrong they don't give you enough detail enough granularity to start answering what do i have to change in my localization program in order to prevent this issue and this is again where we're usually back to square one we need to gather metrics on linguistic quality try to see if they correlate with the outcomes and see what we can change in our localization production process in order to improve the outcome several months down the road tricky stuff be happy to talk more about it we can probably do like a whole webinar or a panel discussion about it later on with a few other smart people uh if you're interested if you want to do that let me know uh we can definitely dab deeper into that any other questions or just comments or thoughts or observations from your own experience with linguistic quality in your organization's current ones previous ones i just a reminder that you will get the recording of the session as well and we'll also send you the slide deck with all the seven signs and some of the strategic questions to answer later on in the meantime let me know if there's more things you want to talk about except amazon all right looks like there are no more questions and that's great if you have anything popping up in your mind please feel free to reach me at this email kiril content quote.com i will personally respond to all the questions you ask or just reach me through linkedin or through our website and we can definitely continue the conversations and see how we can provide you more specific and targeted advice for your situation thank you very much it's been a pleasure to host this webinar today for you please stay tuned for the recording tomorrow and the e-book early next week and we'll definitely continue talking about quality management and localization programs later on stay tuned for further webinar announcements stay safe stay healthy thank you very much for coming today i'm carol solove with content quo and we'll be in touch bye-bye