EDelivery Initial Made Easy

Edelivery initial

[Music] thanks very much jane and hello everyone i'm so glad you all are here to hear me and my colleague ryota talk about how you can get started with graph analytics as dean was saying we are on the product management team i am a product manager with graph and a couple of other technologies i am based on the east coast of the united states ryota would you like to introduce yourself um hi my name is ryota yamanaka a product manager covering asia pacific region and i'm based in bangkok thank you right so today i and riot are presenting from kind of opposite corners of the globe so our standard safe harbor statement so as jane was saying we're restarting this series we we have some product updates we would like to share with you and i also wanted to once again revisit the why graph question why are graphs relevant today why are they likely to be interesting for you in your analytics applications so i'll start off with that and then i will take a look at how we can model your data when your data is stored in a relational um in a database in oracle database and then talk about how you can write some simple pretty quick queries and some product packaging and shipping updates and then ryotar will walk you through some analytics and actually using the product and do some demos to show the uh show how the product is used so with that then we got get started with why graphs and wire graphs relevant today so to look at the y graphs question let's start by looking at what is a graph a graph as we all know is a collection of points such as vertices or notes and the connections between those points commonly called as edges so you hear you see here an example graph you have some nodes smelly gene and yarn and they're connected by some edges for different labels this is a simple graph and the graphs we'll be talking about today are property graphs which is a type of graph and what that means is one of the characteristics of a property graph is that you have properties um for every you can have properties for every node in every edge so here the node melee has properties name address and account number and the edge cache transfer has the uh pro have the has a properties date and amount and so on so when you think of graphs in your application i would like you to think beyond uh what we would consider physical networks like road networks railway networks and so on which naturally can be considered as a graph that is a problem which has been addressed by many different technologies there are many different ways of addressing the problems of routing and so on in road networks and railway networks and other physical networks so when you're looking at graph analytics here i would like you to look at what you would consider a more logical network that's a network or a graph of that can be created from the data that you have in your application this data can be human resource data where the graph can be the connections between employees or the connection between departments or it can be your financial transactions data where you have the nodes as user accounts and the cash transfer modulus edges between user accounts and so on and so forth so all of this type all different kinds of applications and that and the data they work with all of that often can be modeled as a graph and that helps you find new insights into your data so you model your data as a graph your existing data as a graph to get new insights that are revealed because you're analyzing the connections between data items and doing analysis based on the connections what are some examples of uh analysis that you can do when you're looking at your data from the point of view of the connections between your data items uh the classic or the more common thing is to look at how many connections does a note have or how many edges are connected to a node that we refer to as centrality and this gives you a measure of how important a node is so an important node in a graph is like likely to have a lot of connections um a lot of edges connected to it so if you're modeling your customers perhaps based on their social network behavior an important customer could be one who's connected to a lot of other customers and could hence be an influencer so that's just an example of how you look at the importance of a node and apply that in your and use that information in your business application um talking about important notes there are many different ways of identifying that a node in a graph is important the page rank algorithm is a very popular algorithm used to do identify importance of a node so here you're looking at the importance of a node not just by the number of edges connected to that node but the importance of the nodes that these edges are connecting from so you're just not counting only the number of edges connecting to a node but you're looking at where these edges are coming from and if these edges are coming from important nodes then this node becomes important so looking at it from an influential perspective it's a question of um it's not just how many people you know but who you know that makes something an important note and this has a lot of interesting applications when it comes to recommender systems while buying products and and a lot of a lot of other applications so pagerank keeps coming up again and again and then of course there is path analysis um this has again applications in a wide variety of use cases for example in manufacturing uh if you're modeling uh the different parts that make up a car a car is 30 000 parts there are different components and some components and so on so it's often interesting to or useful to find out whether there is a path between two components and how many hops there are between these two components so you can model your bill of materials in a manufacturing process as a graph and then do path analysis on that so again that there are a wide variety of applications that uh that you know that where you can use path analysis to get all kinds of insights into your data so this kind of hopefully gives you a flavor of the things you can do when you model your data as a graph and kind of do analysis on your graph based on that and here's an example from one of our customers which we like to talk about because it really very clearly highlights the value of using graphs a pasif is a payment solutions company and they use graphs to uncover fraud so what they do is they model all the user accounts as nodes and every time there's a cache transfer between accounts they model that as an edge and you can see here right away that the two figures here kind of very intuitively um tell you what is going on in the first figure you have a lot of um edges going in from one source node to the destination node so lots of money is being transferred in multiple transactions from this node to this node but interestingly they go through a lot of intermediate nodes this is to pay safe a sign that something fraudulent might be happening if money is being transferred from this source node to this destination node why is it going through this all these intermediate nodes do they want to hide something do they want to get around some maximum transferable amounts and so on and the figure on the left also you can see there are lots of vertices or nodes lots of money transfer being happening lots of edges but relatively small number of source and destination vertices so this again is something that would alert face safe that this is something they had to look at more closely so they would alert their business analyst to look at these transactions more closely so they have found a great uh a lot of business value in modeling some of their data as a graph and doing analysis based on that visualizing their data as a graph and so on so with that let me move on to the product overview so now that we certainly see in a brief introduction of what graphs are and how they can be used in business applications uh let's take a look at what we offer as part of the product you can store manage query and analyze graphs in the oracle database and it's built using of course the leveraging all the enterprise capabilities that the oracle database has such as manageability uh security high availability and so on and you can manage and we can work with data uh stored in your database using graphs as we'll see in some of our examples and from the beginning we have designed this product to be highly scalable we have an in-memory graph server that you can use for query and analyzing a graph you also have an in-database option uh and the in-memory option or the in-memory server is particularly useful when it comes to analytics because analytics like page rank and path analysis tend to be compute intensive operations uh where this is built to scale to tens of billions of edges and vertices we have collaborated very closely with oracle labs and some of the parallel memory structures for processing graphs have been developed or innovations developed by oracle apps you can store your graphs and then you have to query your graph so you need a query language to query graphs that are stored in your database or graphs that you're working with in the graph server for that we have the property graph query language and we'll be seeing some examples of that and then of course is the rd analytics which is the one of the key features of the product and the and we support over 50 pre-built graph analysis algorithms in the product so it's a java api you can use that directly in your java applications you don't need to write any code to use this algorithms and finally we have a visualization tool which is a lightweight work application to enable easy access and exploration exploration of the graph we also support third-party tools like cytoscape and terms higher and so on so the property graph query language you can see here has the first point is that there is a ability to specify a graph pattern so when you have when you represent your data's nodes and edges you need to have a way of specifying the nodes and edges in your query because you want to find out for example is there an edge in this graph that connects two people of type person two notes of type person with the works for relationship so if you're looking for such a pattern you need to have a way of specifying that pattern in your graph query and the property graph query language gives you that within the the familiar sql syntax so we have retained things like select where group by order by and so on everything else that you're familiar with in sql and provided this additional way to add graph pattern matching and the result sets of your queries are the results of your query or result set so you can use in other downstream operations on your data and we'll see some examples of pgql we maintain the property graph query language web page here we have the late the specification the syntax and all of that lots of information about this query language that you can find when you visit that website and the 50 pre-built algorithms that we have um span a lot of different algorithms and different categories that you can see here detecting components like often graphs fall into tightly connected clusters where the nodes will be more tightly connected within the cluster and more loosely connected across clusters and it's really interesting to identify these clusters because it's useful in a variety of applications for example telecom companies use this type of technology to determine churn analysis so if mellie switched her a phone provider maybe people connected to melee are also likely to switch their service so they need to know this ahead of time so they use uh clustering in a graph to identify who which customers are at risk to switch service and so on ranking and walking to identify importance of a node path finding and so on so you can see here we have comprehensive a comprehensive set of algorithms various flavors of the algorithms like we talked about uh shortest path your dijkstra's algorithm bellman ford so we support a variety of algorithms for that you can use in your business applications and finally the graph visualization tool so query analyze visualize these are the three key features and we are going to also show you some examples of how you can submit a pgql query here and then run your query and visualize the results in this graph in this browser in your browser so with that let me move into modeling your data as a graph so this is a question i get often people tell me that okay i like the fact that you can support graphs i like the what you're telling me about graphs but okay i have these relational tables how do i get a graph from these tables so starting with a simple example you have here a table accounts and a table customers you can think of these accounts as a bank accounts then you have a list of customers and each table has a set of columns like in the customer's case you have the customer name age location etc so intuitively typically each data entity can be a set of vertices so if you have a table of entities like a table of accounts or a table of customers those are typically can be considered vertices that is each row in this table in these tables will be a vertex and the column values would become properties of the vertex so when you have a table customers then you have a set of label set of vertices with a label customers which each row becoming a vertex in your graph and then what happens how do we now next look at edges so you have the vertices that's good how do you have edges so when you're identifying edges one is if you have foreign key constraints between these tables those automatically naturally become edges between the between the vertices but in addition there could be other types of relationship that that you want to add as edges in your graph and this typically would be things like in this case you can say the customer owns this account this is there's a relationship of between every account and every customer so that relationship kind of naturally forms an edge so typically you can create um edges like this you can identify that these are edges or you can just default to using the foreign key constraints to say that these are the relationships between my vertices and this is what forms an edge in a graph so looking at some other edges so let's say we had this additional cash transfer table where you're transferring money uh between different accounts that again is a relationship that can be that can be edges between accounts it's not an edge perhaps not an edge between customers not an account but an edge between the account vertices so you can see here that we have an edge from the cash transfer table you can see here that there's a cash transfer between account 201 and 202 a cash transfer between 211 and 202 so all of these have edges in the graph so now that i've sat down and looked at my table and i say that okay these are vertices these are edges and i've identified a few additional edges how do you create this so we have the ddl syntax as part of the property graph query language where you can create a graph based on syntax that looks like this you're specifying the vertex tables and you're saying that the customer table is the vertex table the count table is a vertex table and all the you're using almost all the column values as properties for these vertices and then you have the edge table that you're using that table uh owned by and you're specifying the source and the destination as coming in from these tables and again you have the transfer edge that with the source and destination coming from the accounts table so you can see here that you can very easily write this kind of a pgql statement to kind of represent what you have in mind for your graph and then all you have to do is execute this page equal statement and the graph is created and stored in the database once the graph is stored in the database you can run pjql queries such as this for instance you have the select clause you're selecting from this graph and here's the match that i was talking about when you're saying in this case you're trying to see whether there are circular money transfers which is another way of identifying fraud if money moves around in a circle and comes back to the original source then that is typically uh some maybe something fraudulent going on so here we see that a1 has transferred funds to a2 was transferred the funds back to a1 so maybe that's something that has to be investigated further and that will be visualized like that as ryotar will show you shortly and here is another example where you're saying that you're looking at or you're adding addition an additional half you're saying that a1 transfers money to a2 transfers money to a3 and a3 transfers money back to a1 so once again you can have a simple graph pattern and you can find all um nodes well this kind of transfer nodes and edges for this kind of transfer is happening so something that could be flagged that is fraudulent so let's look at another example uh this is this this one table where you may be modeling uh let's say an online e-commerce application where you're looking at which products are brought together so you have a set of products each with their own product id and then you have a list of products that are bought with that product so you can see here the products of id one two and 4 were bought with product with id 0. so simple table extremely simple table so how would you model a graph from this intuitively every product is a vertex that's the intuitive thing that you can think of every product id is a vertex and when two products have a bought with relationship perhaps that's an edge and you can label it portfolio they're bought together so you have you have a vertex and you have an edge it's a simple graph and you would create a graph using this vertex like this now we're reading off of the same table you can create views to make the syntax a little simpler to use so you're saying here that the products table is the vertex table so every row every euro with the unique product id becomes a vertex and then you have edge tables where you have the source and the destination referring to um so referring to the same table so every time they're brought together you have an edge between them so once you have uh that type of graph that has been created once again you can run pgql queries you have the select class you have the match clause here and i'm running very similar queries where i'm saying is there a two-hop distance between this and maybe the more interesting is the array um actually the first one is our two products but what together the second one is is there a two half distance in the sense that v1 was bought with v2 then v2 was bought as v3 then how many times has v3 bought with b1 and and things like that so if you're going to visualize this in the graph uh visualization tool it would look something like this this would be the are there circles uh cycles in the graph when you're just um doing a simple circular query or when you're looking at two halves uh you would get a set of triangles like this so if you want to analyze some of these graphs and now that we you created a graph you are running particular queries on this graph graph queries and then you want to analyze this graph this is what the java api would look like uh we have this pre-built algorithms as i was saying so you would just call the page rank algorithm on this graph or you would call the betweenness centrality algorithm on this graph or as riot i will show you you will be running clustering and a few other algorithms on this class so you can run any algorithm just by calling the api so if you're it so it's as simple as that you just have this statement in your java application so some of the architecture options when it comes to using property graphs so the graph is stored in the database your source data store the database so you have the single source of truth meaning the same data is used by other applications you can use the same data for your graph analytics as well so you have the data the database and then as i said we have a graph server so you can load the graph into memory into this graph so you can create the graph then load the graph into memory and run your pgql and graph analytics and memory this graph server can run standalone or it can run within a weblogic container like weblogic server or tomcat so this is the recommended configuration because some of the graph queries can be compute intensive so we recommend running queries and in particular analytics in memory now on the right you have a two-tier application if you don't have the ability to stand up a glass server and you really do not want a treated application you can run some of the pgql queries directly in the database we do not recommend running analytics and we do not support most of the analytics in the in the database so if you're doing analytic duty that you definitely need a three-tier architecture but for smaller graphs and if you really don't want to have a three-tiered architecture you can run some pgql queries in the database itself so these are your architecture options and there's some quick notes on product packaging and shipping but this has this has been um a different for from from since the time we last spoke with you on the ask tom series so we used to ship some of the property graph libraries along with the database in oracle home we no longer are doing that we have moved the graphs client the graph server and graph client packages into a separate graph server and client kit we did this primarily because the developers are going to be writing these graph applications they are typically going to be writing it on a machine that they have access developing these applications or a machine they have access to and having these libraries at on oracle home where they have to ask the dba for access to this libraries did not make sense so we just wanted them to have access to the libraries directly so the graph server and client libraries are downloadable as a separate product so we ship those libraries separately you can download them you can connect to the database using a standard jdbc connection that's all you really need to do is connect to the database with through jdbc create a graph load that in the memory start running your queries and analytics and and so on and the first version in this packaging the 20.1 was released january 2020 20.2 releases in imminent any day now and the graph server and client works with all databases 12.2 and later it works with the autonomous databases adw atp including adw free tier we hope to do a session in the future we will focus on how to use graph server and client with adw free tier uh database cloud service etcetera on-prem databases of course and we plan to release this four times a year so it makes it easy for you to update the latest graph features you don't have to wait for the next database release with that let me hand this off to royota um meli before we transition into ryota's section there are two questions on the q a if you want to comment on those briefly right so the first question is is there capability to export and import from a graph ml form yes there is i did not mention that because um we we don't see a lot of questions around uh graph file formats like that but we do support graphical melanin and a few other graph file formats and that's no documentation the second question is do we have support graph support in autonomous yes we do as i think i just answered that question can we use oml notebooks and autonomous to run graph queries not right now but that is on the roadmap right now you would have your own notebook for graph queries and your own notebook for and merging that is something that is coming in the future great thanks nelly so i think we're ready for ryota to begin his section okay okay um in the last half of this session let me explain how to quickly set up graph analytics environment using docker on your machine or oracle cloud and especially uh when you have oracle cloud three tier you can also set up uh your graph analytics environment on it and and also i'd like to show how to use separate notebook and graph visualization component for actual uh analytics use case so um if docker is installed on your laptop or cloud instance you can easily build a graph analytics environment in this step we recommend the three tier deployment shown here as it supports all graph analytics features such as query algorithm visualization and notebook in the current release for this deployment we need four containers oracle database graph server graph client and tempering graph visualization component is actually included in graph server and this should be your web browser which can access to graph server from your local host the quick installation procedure is written in this wiki here it is and as you can see if you go to uh docker installation step one uh it is very brief and i think you can complete this initial setup in 30 minutes okay then i'm let's start these containers on my laptop to do so um this is a command to run all this container using docker compose and it will start phone content for containers graph client server oracle database and separating on my laptop and all of them are starting now now we have several tables in the database and we want to create a graph from those tables as mary explained using create property graph statement and this create graph oh sorry create property graph statement should be issued by graph client and run on database so let's uh see the tables in this database now i think the database not yet up let me check okay well um while we are waiting for uh oracle database let's go into the next slide we want to check the how how we can write create property graph statement and it is my github page as well okay this is the create property graph statement which is pointing the original tables and and for for vertex and edges and defining those nodes and edges using the table names and column names of original datasets data source right and then for running this query we can open graph client always seems my laptop is a little slow but uh yes now i'm connecting to the database and as i said the database has several tables like those account customer merchant transfer and now i'm connecting to graph client to issue this create query okay so from this graph client we can connect to the database okay now it is connected and while pointing this create propertygraph pgql file we can run that create propertygraph statement and after running this statement you will see those tables which contain contain nodes data and also edges so these are internal tables which contain the graph itself so it was this step now we have a graph on graph database so we can retrieve the data graph data and expand that on graph server on memory so for loading from database we can set pre-road graphs on this graph server so when the graph server start those preroad graphs will be expanded on this memory space of this graph server so let's restart graph server to do so i can use docker compose command and well this is restarting graph server and separating containers so the graph server has yes these two containers has already restarted then i can go to check if this graph visualization is up or not well it takes a little time in the first time to connect so i can show it on my cloud instance once you successfully wrote the data you will see the nodes and edges on this visualization and also separate notebook is running here so this is the steps to create your graph on a relational database and retrieve the data on graph server and visualize it if you uh have already applied for uh oracle cloud always free services um everyone can create oracle cloud account which includes this free tier and use two compute vms and two autonomous databases for an unlimited time so oh this is the oh yeah brief explanation about the free tier and using one computer vm and one autonomous tv you can build an always free graph analytics platform in this way the setup procedure will be soon updated on my wiki on the same page here and in my experience it is a little tough to run graph server and separating both on the same compute vm um because of the the limitation of the memory uh which you can use uh the of the instance which you can use in always free tier but you can run graph server so that you can access to the graphs using graph visualization and it can connect to oracle database to get the graph data okay in the last section uh of this talk uh i'd like to explain uh and introduce one use case of graph analytics which is about customer 360 degree analysis customer 360 degree analysis is is widely applied in many industries and the data integration is the first step for this analysis well let's say in this small graph data it assumed to be an integrated data set of these data in financial industry like those bank accounts and account owners and transactions between those accounts and purchases by the people who own the account maybe we can get this information from the usage of credit card and this data integration concept is um applied in many industries such as customer 360 in digital marketing employee 360 in human resource analysis or product 360 in manufacturing in typical graph visualization user can select nodes and encrypt node and expand the surrounding information to pull all related information to specific this entity for knowing the more information about this node okay then i like to show this concept using graph visualization here in this case we can get only one account and the account number card number is xs x yy and 2 0 1 run then i can get one node here i want to see the label and let's get those connected node from this node using expand function then you will get some information like this account is transferring money with those surrounding account and also something is purchased from this merchant and another merchant by using this account and also this account is owned by john if you want to get more information you can expand two hops into hops and then you will get more information surrounding information of this node so this is the basic concept of the data integration and how we can see those integrated data on graph visualization also a graph database function support graph based query language such as epgql and pgql is convenient for detecting specific patterns such as find accounts that had an important outbound transfer of over 500 on the same day and its balance is less than 300. since such an account is used only for transferring the money and assumed to be less beneficial for financial services let's try this query run so we detected this case this account has got money from this account but the money is immediately transferred to another account on the same day and this node doesn't have much balance you can see the balance is that 200 dollars only so this is a pattern matching functionality of pgql and another example is to find a series of transfers that stirred and end at the same account such as a to b to a or a to b to c to a okay the first thing is simple saturation like this and another one is a triangle we can do this uh saturation detection using uh well for arbitrary numbers of hops using path query you can check the query on the document the last part is about [Music] using a separate notebook how we can write programs for more detailed graph analytics for more procedural analytics for and for running graph algorithms uh we need to use a separate notebook which is a multilingual interactive coding interface for example the question is which is the account with the highest page rank in this small network this is a money transferring network and we want to filter by transfer relationship to get this small uh graph and run pagerank algorithm to score the importance of those nodes pagerank is a well-known algorithm developed by google to calculate the importance and of each node considering the connections and their directions uh before running this algorithm we need to filter by a transfer relationship to get this small graph and we can run a page rank algorithm on japan notebook and query the result so let's see on separate notebook so this is a notebook interface and this example is already included in the demo if you install with this docker installation process and then the first step is to attach the graph on graph server okay run this code oh this one is get graph method to attach the graph oh by the way this is a groovy language it is a java based script language now the graph server interpreter supports this language but soon we're going to support python language as well then in the next step uh this is a running query pgql query which i have already shown and now you can get the result in table format like this we have already seen the pattern matching and cycle detection on the graph visualization so i'm going to skip these things and then this is an example of detecting important nodes so the first step is to filter by transfer relationship so that you can get this homogeneous graph now this graph contains only six nodes and eight edges then run pagerank algorithms and now you can uh get the result on this pgql query again looking into the new property node property called pagerank so the result say that no 201 is the most important node in this network and and maybe yes it is intuitively correct result the next example is community detection the question is how we can find the communities where every account is reachable from every other account uh it is also called as strongly connected component like these four red nodes are connected strongly because each of those node nodes can access to other members of this community component following those edges in their directions but this yellow node and pink nodes are not in the same strongly connected component because the arrow node can access to the red node component but those red nodes cannot access to yellow node because of the direction of the edge and the same thing for the pink node assuming you have a much larger graph you can detect this kind of strongly connected component as a community to do this again you can use this filtering to get this money transferring network and run strongly connected component crossover algorithms on this graph then you can find those nodes who are included in the same component as uh the one the node for two zero one account in writing some code snippet like this you can find okay you can find the number of components and the number of members for each component the final example is recommendation which merchants should be recommended to this account1201 based on the purchase history to generate this recommendation one simple method is to use random work the random walker always starts from this node two zero one and randomly moved to connected nodes in a certain hops and easy up here every time when london walker hop on the nose nodes it gives them squads then the landmarking is repeated many many times from this certain node one two zero one at the end closer nodes to this node have higher scores than those nodes in distance then this query fill the margin type nodes here and remove those merchants the account 201 has already purchased from these two so we can find which of those nodes should be recommended to this node to zero one okay to do this this time we like to get this bipartite graph so we can filter by purchased relationship then for random walking you need to add reverse edges [Music] using this code then this code snippet is running random walking then after random walking you can show the scores of those three nodes and then it shows that kindle store and azure books they will should be recommended with equal scores and abc trouble is more distant from the node to zero one right okay so um these are the uh small example using small graph which you can try with that document installation immediately okay then yeah i will switch back to nelly so yeah so we have if you can go to the next library um right so just a partial list of plant features we have um lots of plant features coming and we just put a subset of those features here graph server and client is will be available as an ocm marketplace image and just like we have the create property graph we have will plan to have insert update and delete syntax in ptql and we also want to make it make pgql available within the sql cl and sql developer tools and finally the native python client for data scientists uh we want to make that available today of course you can use j pipe and use java api but you want to have a native python client and that is coming and sql pgq is the standard that extends sql to how property to have graph syntax and support for that we are actively participating in the standard and we plan to implement it when it becomes available and i see a comment that they really like the ocr marketplace id area that that should be out that very soon imminent actually any day you'll have the ocf marketplace okay next slide please and here are some links you can go to on oracle.com to learn more about graph and uh you can download the graph server and client from oracle.com or from e-delivery and of course once you have ocm marketplace image you can use it on the cloud very easily next slide jane yeah so uh we have a very vibrant and active user group they are a subset of what was formerly known as the b way user group now being rebranded as the analytics and data oracle user community but this slide is just an invitation for you to check out their linkedin group their twitter handle or to send an email if you're interested in learning about the activities and seeing some of the technical exchanges that happen on the linkedin group as well as updates about events both virtual and in-person as well and next slide so with that uh i think we're ready to wrap up this has been a great session uh melee and riota so uh thanks for the great information i think we both have now uh shared product updates as well as some real uh hands-on uh information a reminder the slides from the session are up on oracle.com as well as uh you can also access ryota's github resources and those links have been posted in the chat as well we're planning our next month's session and we'll be sharing more in-depth use cases that are actually real world use cases for graph analytics in the financial industry and the date will probably be on may 28th so please check back the landing page for confirmation on that we'll be confirming the dates and times very soon today's session was recorded and you can access the recording also at the asktom landing page the url is again on your screen if you subscribe you'll get email updates about upcoming sessions and you can always submit feedback topic requests and questions at that site as well so with that i think we're ready to wrap up mali and ryota any other parting comments for our audience today well now that thanks everyone and i hope you try our product very soon yes um try out the um docker installation and i'm happy to have any uh questions and suggestions thank you excellent thanks everybody for joining and uh have a wonderful day we'll see you next time thank you thank you you