Empower Your Business with Outbound Sales Automation in IS Standard Documents

How to create outlook signature

[Music] hello and welcome to this power platform webinar focused on intelligent document processing with power automate and ai builder i'm gwynel bego principal group program manager for ai builder and i'm joined by joe fernandez joe i'll let you introduce yourself my name is joe fernandez i'm and i'm a program manager working on a builder i'm excited to be here today to talk to you about how you can use ai very easily within the power platform awesome thanks joe so yeah as joe mentioned today we're going to show you how you can use power automate and ai builder together to automate document processing in your company allowing you to reduce the risk of errors process document faster and increase overall employee productivity we'll be covering five different talking points in this session so first we'll give you an overview of what we call intelligent automation recapping the different sets of ai driven scenarios that you can automate with ai builder we'll also show you intelligent document processing landscape so that you can understand the types of documents that you can process with our technology next we'll show you the different features that are supported today and walk you through how to create your own custom document processing model we'll also show you how you can use your model in an end-to-end process with our document automation solution and finally we'll go over some common use cases for document processing and recap the benefits that you can generate by automating a process all right let's kick things off by going through an overview of intelligent automation now intelligent automation can generate incredible benefits in your company it fuels increased productivity cost reduction greater accuracy in business processes with the end goal of providing higher increases in revenue over time now we think of intelligent automation as a combination of automation technologies infused with artificial intelligence allowing you to automate complex processes end to end and microsoft our one intelligent automation platform is power automate it brings together cloud first automation with multiple core capabilities to create a comprehensive and connected hyper automation platform it includes such capabilities as robotic process automation or rpa which is ui based automation to automate activities on your desktop with desktop flows it also includes artificial intelligence or ai driven capabilities like intelligent document processing that we'll be focusing on today it includes digital process automation or dpa to automate cloud-based processes across native and third-party apps using the connectors ecosystem it includes business process management to automate guided multi-step processes managing everything from basic approval processes to advanced workflows using rp and dpa and finally it also includes process advisor which is a new process mining capability that helps you to understand and analyze bottlenecks in your business processes so that you can improve them using automation now today's focus is on ai and ai capabilities specifically so let's zoom zoom in to our ai capabilities uh that are available with ai builder today ai builder allows you to automate intelligent processes across four different types of scenarios the first one is documents which allows you to extract and process content from multiple different document types documents such as invoices receipts forms or identity documents the second category of scenarios that we support is language which allows you to translate transform and extract key information or sentiment from unstructured text third category is images to detect objects in images and classify them and the fourth category is decision to protect new values and structured data that are based off of historical outcomes now with ai builder all of these scenarios can be built without code using point-and-click wizards in just a matter of minutes and its back end ai builder leverages industry-leading ai technology uh that is backed by azure cognitive services and all those technologies and models are customizable to your company's specific needs ai builder is also directly integrated in the power platform so that you can easily add in ai driven actions into your power apps and into your power automate flows so now we've covered the high level overview of intelligent automation let's dive into intelligent document processing specifically and describe that functionality in a little bit more detail intelligent document processing is a software solution that captures and transforms data from documents in an end-to-end process multiple different input formats are supported like emails text pdf or scanned documents an idp uses multiple different ai technologies together such as computer vision ocr natural language processing machine learning and deep learning all these technologies are used in combination so that the ai model can understand the context of a document and so that it can analyze categorize and transform its content allowing you to export its data to external systems and to truly build an end-to-end process idp can also understand a wide range of document layouts anything from structured documents such as forms invoices receipts identity documents or unstructured documents things like memos contracts or customer feedback requests now it's no secret that documents are still widely used across the business landscape there are multiple functions and industries that rely on standard document types to manage everyday tasks some common examples include invoice processing and purchase orders in the finance department quality assurance or change requests in manufacturing proof of delivering supply chain or application forms in the government space many departments across these functions rely on workers to manually process information from these forms and generate an outcome that is stored in a centralized system imagine if all the time that could be saved if these documents could automatically process themselves and generate the outcomes that you would want without the need for human intervention and with idp technology this is all possible and together with joe we're going to show you how let's deep dive on the technology for a little bit now with traditional ocr technology it's possible to transcribe a document but then you need to set up complex rules to write and interpret and extract the data that you need which makes it hard to automate a document in an end-to-end process idp technology is a little bit more complex it's not just simple ocr it actually leverages a deep learning ai model that's trained to understand the structure of the document and to return only the information that you need from that document it can understand specific fields like an invoice date or an invoice number it can understand complex tables that are used to store information and it can even understand specific fields like a checkbox in this next section let's go through how you can build your own ai model to automate document processing with our easy to use point and click solution now remember just because it's easy to use it doesn't mean that it isn't powerful all of our ai builder models that are generated through an easy point and click wizard end up generating a powerful azure ai model in the back end with the same level of security and reliability that you'd expect from integrating directly through azure ai developer apis in terms of the different types of models that are available there are two different types of categories of models that you can leverage for document processing the first one is pre-built models the second one is custom models now pre-built models are already pre-trained by azure ai and they work out of the box they're able to read standard document formats like invoices receipts identity documents or business cards on the other hand there are custom models let's say you want to process a specific document type a specific form that is unique to your business function or that is unique to your supplier in this case you'll want to train a custom model to understand that specific document structure and extract only the specific fields that you care about so to recap there's essentially three types of models that you can use in ai builder to process documents there's text recognizer which is the generic ocr model that we talked about you can use that model to extract text from the document and then use any of our language models to analyze that content and transform it there's also our set of enriched pre-built models used to extract data from an invoice a receipt or an identity document and then finally we have our custom models our custom form processing models that you can use to train an ai model to understand a specific document format that is unique to your business function or to your supplier let's take a look at custom model features what you're able to do when you want to set up a custom model is train it to understand only specific fields that you want to extract it's all quite easy there is a point-and-click wizard what you have to do is you have to detect the specific fields in the document that you want to extract uh draw a specific table around those documents and then you're able to extract the content and determine only the specific content that you want from the document and joe is going to walk you through that process in a demo in a little bit in terms of features we also support different document layouts let's say you have a process with multiple suppliers that each have a different specific layout we have a concept of collections that we can use during the model training time with that concept you'll want to group specific documents that have the same layout into a given collection this allows you to train a single model that's able to understand a wide range of specific different document types mapping to all the different specific needs of your different business functions or specific needs of your suppliers finally languages is an important feature we've just released an update in ai builder that allows you to support a wide range of languages uh with form processing there's actually over 70 different types of languages that are supported making easy easy to set up an intelligent document processing flow across a wide variety of languages and regions so there you have it uh that was a set of features that are currently supported through the custom form processing model but enough talking let's see this model in action i'll hand it over to you joe to show us how you can build our very own custom form processing model using ai builder thank you so yes let me switch back to my computer and see how easy it is to use ai and automate to to automate a process so here for this demo let's imagine the following scenario i receive every month invoices like this i receive hundreds of them and every time i receive an invoice i need to transcribe its information from the invoice into an excel file and then into an old legacy website and this is a process that takes me significant time every month it's a very repetitive process a very boring process and i rather do something more productive than just copying and pasting data from one place to another so let's see how we can use ai builder and poor automate to automate this this process so i'm here in the airbuilder website which is accessible from the portman portal you will find it on the left navigation bar ai builder and here we see all the different ai capabilities that we've seen in this presentation at the top we'll see the models that are customizable these are models where you can bring your own data and tailor it to your needs and here in this section we see all the pre-built models so these are models that are already trained and you can use them right away into your automations and applications we see that there's one pre-built for invoice so let's click on it and every time we click on one of these cells we get the same experience we get an introduction on the model to learn what it does and we also get a useful tried out section where we can see some samples to get an idea of what the model is able to do and i can even go ahead and upload my own invoices to try it out so here i have one of the invoices that i receive every month let's see what ai builder is able to do all right we see that it's able to extract already a good amount of information things we can expect like the total amounts the invoice id the different addresses even those line items so tables invoices so everything's looking good but one of the fields i need into my process is the shipping amount and i see that as of today the pre-built invoice pricing model is not extracting this field but that is not an issue what i can go ahead is just move and create my own custom font processing model so let's do that let's go into the custom section and create our form basic model i'm going to click on it and the first step is just to give it a name so let's call it for example webinar demo and click create and this takes me into a step-by-step guided experience that is going to help me train and build my own custom model the first step on this experience is to define which fields and tables i want to extract so if we go back to the excel file these are all the different fields from the invoice that i need to extract so let's just define them in this tab so i'm here back into the model i'm going to click add and here i can extract fields and tables in this case i only need fields so i'm going to click fill next then i'm going to go ahead and give it a name so the first field we want to extract is the invoice id and do the same i just go ahead and define the different fields i want to extract so second one will be the subtotal amount then i also want the tax amount we wanted the shipping amount which was the one not supported by the pre-built total amount of the invoice the due date and last the bill to address all right so i've defined the seven fields i want to extract let's move on to the next step and in this second step i need to upload sample documents to teach my model how to extract the information i want and the way we upload documents is through collection and collection is a grouping of documents that have the same template or the same layout for this demo we're using invoices from two different providers that they have two different layouts as we can see here so let's go ahead and create two collections i'm gonna create the first collection and for each collection i need to upload at least five sometimes five sample documents i can upload them from my local device from a sharepoint site or from an azure blob storage account for this demo i'm gonna upload them from my local device and here i have five sample documents for my for first provider all right if i have documents uploaded i can also give it a name so it's easier to reference so this is going to be the invoices from the contoso provider and let's go ahead and create our second collection and here again i need to upload five sample documents for this provider so here you have them let's upload them and they have been uploaded as well i'm also going to give it a name in this case is the avatum provider okay so we've uploaded we've created two collections and we've uploaded five sample documents for each one of the collections let's move to the next step now ai builder is looking through these documents to understand the structure and that will help me build the model great so the analysis is now complete and now i see the sample documents that i've uploaded so if i click on them what i have to do now is i have here the fields that i want the model to extract i need to indicate where are those fields so that way the model is going to learn which where they are so let's go ahead and start so to indicate it's very simple i just draw a rectangle around each of the fields and indicate to which field corresponds to so in this case this is the invoice id let's go for next one so subtotal is here so again i draw here a rectangle and indicate that it is the subtotal and i'm going to do the same for the remaining of the fields the tax amount the shipping amount the total amount the due date is here and last the build to address all right with tag of first document let's go ahead and move to the second document and here i'm going to do the same process some of the fields already tagged because we have that analysis process and a builder already understands a little bit of structure so i just have to tag those fields that are remaining so in this case the subtotal amount the tax amount the shipping total and the built to address so very easily i can tag this document so i will do this tagging process for all the sample documents that i have uploaded once i've done that i can move to the next step where i can go ahead and train my model and once i treat my model i will land into this page i've already trained a model with the same sample files and the same fields and is ready to be used what we recommend first is to try it out so here we have a test function where we can go ahead and try it out with samples that we did not use for training so let's go ahead for example and use this invoice which we didn't use for training to see what is the performance of a model so here we see that it's able to extract all the fields that we wanted including that shipping amount that we also wanted and we also get the confidence score and we see that confidence score is 99 it's very good so we can go ahead and use it if that was not the case if we were not getting good results or the confidence score was low i can always edit the model and add more samples to improve the model but in this case we're ready to go so let's click ahead and click on use model and i can use my model in two ways i can use it in a flow in port automate or in an app in powerapps in this case because we want to automate a process which is i want to write the data into an excel file and then i want to input it into a legacy website we're going to go ahead and build a flow important for this so i'm going to click new flow and this takes me to a sample template to get started so let's create this template and what i have here is an end-to-end flow that is ready to go the way it works is this flow is going to get triggered every time i send an email with an attachment and the subject ai builder then it's going to iterate through different attachments they have and it's going to run it through the form processing model that we just trained it's going to extract the data and it's going to send me the data back in an email so very simple though but right away i can try it out and then i can customize it so that's what i done i went ahead and i customized the flow to what i want to do here so the trigger is the same i receive the invoices by email so that stays the same i iterate through the attachments i run each attachment through the phone processing model that we train and then once the data is extracted we're going to add them into the excel file here and then for the legacy website since there's no api connectivity what i've done is i've built an rpa bot using protomate desktop that replicates what i usually do when i enter the data into the website so let's see this in action let's put the flow in test mode okay so now to see the flow work i need to send an email into my inbox so let's do that i'm here in my inbox i'm gonna create a new message i'm gonna send the message to me and the subject line is gonna be a builder for the flow to trigger and i'm going to attach one invoice okay let's send the email the email has arrived so now at any moment the flow will get triggered so now that the flow has been triggered is taking the attachment it's running the attachment through the phone pressing model and then the data is going to be entering to excel so if we go into the excel file we should see the data be entered perfect and then the next step is an rpa bot entering the data so let me switch to a vm and here we see that the rpa bot is entering the data into this virtual machine without me having to do anything so that's the demo we've seen how in a matter of minutes we've been able to train a custom ai model to extract data that we wanted from invoices and then using power automate we've been able to automate the process of entering that data into excel and then with an rpa bot to enter the data into a legacy website very easily as you saw there was no single line of code no need to do any complex machining operation is that easy and that is the power of the platform back to you bueno awesome thanks joe it was really great to see how you were able to not only create that custom model to detect that specific field that you wanted in your document but you also were able to leveraging power automate templates already create the premise of that end-to-end process to fully automate that document processing solution so that that's great in this next section i wanted to highlight another solution that we built to help uh users build out that end-to-end process now joe showed us that there are already a set of templates pyro automate templates uh that can be used and that can be actioned straight from the models page after the model is trained another concept that we've built is a full document automation solution that already has a certain amount of the elements that joe showed already pre-configured so that you can just take that solution you can tailor it to your specific needs that are in your company and you can build that end-to-end process to automate how documents are handled in your company so let's walk through basically the process of document automation and the different components that are included now the first one that joe showed in his prior demo is the ability to monitor a specific mailbox to check when you receive new documents that you want to process now using a set of either pre-built models or custom trained models you can then extract the information from the documents from the email attachments now the document automation solution also gives you the ability to have a bit more complex validation processes maybe in some cases uh when you're extracting the content you want to verify that the content is accurate that it actually maps to data that is stored in a database for example or maybe there's specific validation rules that you want to set up maybe if an invoice uh is over a certain limit you want a certain approval from a specific approver before you're able to process that invoice maybe you also want to check the accuracy of the different fields so for that what we did is we pre-configured a validation station that is built as a power app that's already included in this document automation solution that allows you to verify the content that is extracted from the automated process and you can approve validate that selection or you can choose to send it over uh for another maybe more complex approval process that's already built in as part of the solution and then finally there's a set of extraction export possibilities that are already pre-configured in the document automation solution and there's also monitoring tools that are able to enable you to monitor how effective and how many documents you've been able to process all right so there you have it for an end-to-end document automation flow next we're going to see another demo with joe who's going to walk us through how we can set up the document automation solution our starter kit to help you set up an end-to-end process for intelligent document processing joe is going to show you how to configure that and how to customize it for your own specific custom needs over to you joe thank you guys let me show you how easy it is to set up document automation and set up that end-to-end intelligent document processing that we want so yes for this second demo we're going to see how we can leverage the form processing model that we trained in the previous demo with the document automation solution to set up that end-to-end process that is going to be able to process attachments that come through email it's going to extract the data using the form pricing model we're going to be able to review the data in a validation station and once validated those data are stored in a database table that i can then export to any system of my choice so i have here my model let's go now to document automation documentation you will find it on the air video section within the broadband portal here so let's click on it and here on the page we see information about the solution as well as some helpful links to documentation and tutorials to get started the first time that you will enter this page you will see an button you just have to click on it and in a matter of minute you will have the solution installed the only thing that we're going to ask you is the maplex account you want to use for this solution in this case for this demo i've already installed the solution and now the next step is to open the application and this is the application where i'm going to be able to see the different documents that have been processed but first i need to configure to use the form privacy model i wanted to use so i'm going to go into the configuration section i'm gonna click here and here i see all the different form processing models that i've trained in my environment i'm gonna click this one next and here i see the different fields that this model is able to extract i confirm that this is what i want perfect save and that's it that's how easy it is to configure document automation so now i have set up an end-to-end process where every time i send an email with attachments to a specific input inbox is going to be able to extract the data show it in this application so let's see it in action let me go back to my inbox and listen let me send me an email again here i've configured to trigger the process when i send an email with the subject line builder and i'm going to attach some invoices let's attach this one and let's attach a second one from a different provider send so now i've received the email so now document automation should be picking up that attachment extracting the data with the phone processing model that we train and we should be seeing the documents appear here in any minute okay so the documents have been processed and now the status are in manual review so i need to go ahead and review them so i'm going to click on each of the documents and what i get is a side-by-side viewer where on the left i see the document that has been processed and on the right the data that has been extracted with the ibuilder so i see that every data has been extracted correctly i only get here a warning for the shipping amount but in this case this invoice doesn't have a gpa amount so i can validate it as a good good extraction i'll just go do the same thing for the second invoice here again i see that everything has been extracted correctly if there was some issue i could always modify it of course but in this case everything is is looking good so let me go ahead and validate it as well all right we've validated the documents and now these appear in a table interrupters so all these extracted data is here stored safely and structurally in database and now i can decide to export the data in any system of my choice and the great thing about document automation is that it's a driver solution so i can customize it to customize any of the components to any of my needs so these are different components so i could customize for example the validation station to with my company branding for example if i wanted to i can also customize different cloud flows that orchestrate the process of extracting the data from the attachments to the validation rules everything is fully customizable and that's the the great thing about about this solution that we provide so that's the demo so we've seen how with a phone privacy model we can with two clicks the document automation solution which provide us an end-to-end intelligent document automation process very easy to and very easy to customize back to you gwen thank you joe for another great demo to recap what we've seen you've shown us how we can customize an ai builder form processing model to be able to process a specific document type you've shown us how to include that into end-to-end process using a template and then you've showed us how to also and configure the document automation solution which is a complete end-to-end solution that allows you also to verify the information that is extracted from documents using the validation station that really is a complete end-to-end solution to process documents now that we've seen how our solution works let's go over some common use cases in the enterprise and some typical benefits that you can get from integrating this intelligent document processing solution in your company now we've already seen at the start of this presentation that there were a number of documents that are used across different functions and different industries here i've highlighted four common use cases that we see a lot across financial services to handle an account payable automated process for government to manage application forms for healthcare to manage medical results forms and for energy to manage production reports these are typical processes that include documents i think what's important now is that you look at your company the different functions that are using documents in the end-to-end processes and figure out how you can automate those using the solutions that we presented now let's go through the process again which is similar to what we saw already for the document automation solution but just specific to accounts payable as you'll see in a typical ap process you'll have different vendors that are going to send invoices that'll all have different formats to a central mailbox so what you would have to do is train a custom form processing model with different collections that map to all the different types of documents coming from your suppliers you would have to configure or tweak a standard power automate flow to monitor the mailbox and then to point to the custom trained ai builder model that you set up then what you have to do is set up maybe some validation rules either using the document automation solution that you were presented or maybe there's more specific or complex rules that you want to set up maybe you want to verify that the amounts of the different documents that were extracted actually mapped to fields that you have already stored elsewhere maybe you want to check cross-check that with the database that you have all of that is possible using the connectors ecosystem of power automate that has over 500 connectors that you can use to communicate with a wide variety of different systems so that process should be fairly easy leveraging power automate and then finally you can choose to take the data and then extract it and to any of your systems like an erp to really complete that end-to-end process and by setting up that automated process there's lots of benefits that you can have let's say your company that typically processes maybe 5 000 invoices in a given month uh that takes probably on the low end maybe three minutes for a person to actually read through that invoice extract the information log into the system and then and then type in that value in the system if you're able to fully automate that process end to end using power automate and ai builder you can save maybe up to 250 hours a month that's over six work weeks uh that you'll that you'll be able to save so as you can see there's a good return on investment here for sure and the idea behind all this solution is that you can stop focusing on routine tasks by automating these tasks there's lots of benefits that you can have first of all faster processing time obviously reduced processing errors when we when we look at documents by hand manually and we and we supply the information into a system there's high possibility of human error so we can remove all that through automated extraction and then of course improved employee productivity we can go focus not on these uh routine tasks but on on on more interesting tasks that require human intelligence so that's kind of an overview of all the benefits that you'll get from this solution uh in conclusion we hope that we've given you a complete overview of how to automate document processing with ai builder and power automate all you need to do next is think of the different types of documents that exist in your company to run everyday processes and how you can generate cost savings by automating those tasks now there are a number of resources here that i've learned to help you get started you can look at an overview of ai builder of the document automation solution that joe presented there's different learning courses where you can review again all this content and guide you step by step into setting up your own first custom form processing model or configuring the document automation solution and then finally we have other tutorials and resources to explain to you how to set up uh ai builder and all the details that you need to know about ai builder licensing i'd like to thank you once again i hope you've enjoyed this webinar thank you again very much joe for the incredible demo and we hope to see you soon with power automated ai builder [Music]