Add Looker-on Date with airSlate SignNow

Ascend name field

hello everyone today's top six list is why looker users love to use ascend my name is tom and joining me for the demo portions of our video will be my colleague sheil to get started give you a little context about where ascend fits in the looker ecosystem if you are a looker user you're very familiar with this slide that shows looker interacting with a sql database or data warehouse that's housing data from a variety of mostly relational data sources with ascend you open up the entire world of big data iot and streaming and web analytics as a send can ingest and transform that data before handing it off to looker or optionally loading it into the data warehouse or database to which you currently have looker connected to fantastic now off to the top six lists number six any data source anywhere if we can get to it we can ingest it we have native connections to common data sources and the ability to write small python snippets to bring in anything else now over to shiel for a quick demo now clearly that's not sheol but cookies are like data sources to us we love them and we eat them up number five pipeline visualizations ascend has a modern dag based gui interface that simplifies complex queries into easy to understand sequential operations through this gui you can also see metadata and state information about your transformations and data pipelines all at a glance this time shield for real thanks tom here we can see an ascend that a complex sql query can actually be laid out as separate chunks for example with our green cap data here we're first cleaning it up and then we're unioning together the incremental data set with the historical data set and by seeing it as separate components this gives us the ability to investigate and see what's going on that first we have these 4.68 million records that are all getting cleaned up and then adding that together with these 4.15 we get 8.83 million records so it looks like overall with these sql statements there's nothing wrong with our transformation we even have the ability to dive into the records of one particular sql node in this sql statement as we know that we're just cleaning up the green cap data we can see that we're adding something like a current time stamp and cleaning up some columns but in this end we have the ability to also start looking at exactly the records that we're working with this is the full set of data regardless of its million records or billions or trillions of records this is not a sample we can scroll down and look at the full set of data we can also even write queries against this intermediate data set thanks so much shiel number four data quality who doesn't have a problem with this with ascend you can easily explore and validate your data before handing it off to your reports or to your analysts back to you sheila thanks again tom even though we have a sense of the number of records that are flowing through the system this doesn't help us understand the quality of data luckily ascend is profiling the data all along the way so we can open up any one of these sql components and we can take a look at the data profile as well going through the different columns here we can see that you know we have some cab type some drop off day times but the fair amounts looks like we have some problems there we're seeing negative values on some of these record sets let's figure out how big of a problem that is by querying this component and finding all the negative fair amounts go here and query this component and we can go ahead and write a count star as the bad records where fair amount less than zero simply running that and querying this data as we're bringing it through the pipeline will give us a sense of how bad the data is and if we just want to discard these records still coerce them or do something else about it looks like we've got 9 000 of them out of a full set of 8 million that's not too bad i think for right now what we should do is we'll go ahead and discount these records so we're going to just remove them from the data set by selecting all the records that we do want and making sure that the fair amount is greater than zero we can double check that query by writing a quick limit we can see that ascent has already figured out the schema while it's still chugging through the data and we can see here now that this set looks pretty good let's go ahead and remove the limit and we can save that as a transform into our production data flow we'll call it valid cab records and by simply hitting create we can see that it's been added to our graph it's already started to run and soon it'll be up to date just like all the other components thank you on to number three adaptive ingestion what a cool term ascent solves a number of common schema problems including cascading schema changes into the data warehouse automatically we can also alert the data team with configurable levels of notifications when breaking changes are detected once again over to geo back to the data flow we can see that we've wired up this node to write into redshift and snowflake with ascend we can automatically keep these downstream syncs like any warehouse or even directly just straight into s3 for other machine learning or data science use cases up to date by simply adding one of these right connectors in fact with these right connectors we have the option of also keeping the schema automatically managed that in order if there's any schema mismatch we can go ahead and adapt a schema change even fixing up any type mismatches or dropping any odd columns that have maybe made their way in there let's check out what that looks like by going into the sql node i can go ahead and add a column that we want in our database such as editor we'll add that to our group by and i'll hit update looks like a send has already recomputed with that and we've added in a new editor and then ascent has gone ahead and started to restage redshift and snowflake adding this new column into the data warehouse and once these are up to date we'll have that new column as well as all the historical data filled in already if there's a fatal error in your data or in any of your code logic that you've written and you want to be notified about it you can go to data service settings and notifications and simply add a new notification where we can send an email or a slack message for anything that's going on in your ascent environment in the dataflow events or in the system processing events for anything that might be going into error state thanks again shiel number two data lineage we allow you to easily answer the question how is this field derived through the visualization of lineage that shows all calculations and operations done to the data we can do this upstream or downstream demo please shield here's the lineage of chasing through the sum total amounts one particular column of our daily write data we can see that it's coming from this line in this column which is actually coming from a sql function running a sum on the total amount and we can keep going through that way tracing all the way up to the input source data and as a reminder we can see that it's coming through the union data set as well as it includes the yellow cab data set from the first upstream source from the original raw source of the data we can see that the green cap data is using the total amount column but we actually have a small bug on the yellow cap data that is using the tip amount that probably explains why our our reports are a little bit buggy and we should get that fixed up asap similarly we can do a downstream impact analysis from here we can take the column tolls amount and if we're thinking of maybe changing the definition of it or we're thinking that maybe this this column isn't so worthwhile we can see how it's being used downstream in these particular transforms all the way into its final data state as to where it's dropped off and finally number one you get to play with cool stuff yes no matter what your level of technical skill we make the most modern capabilities accessible to you instantly play with python execute spark jobs begin your data science and machine learning journey with no knowledge of the underlying infrastructure or applications required your last demo shield with ascend if you do need further capabilities such as writing python you can also create a pi spark component in this one i'll show a simple example of a machine learning algorithm just using a more simple logistic regression model we can see here that without having to know all the underlying pieces of how to spin up a spark session to do machine learning or deal with any of those types of complexities that's simply all passed into us as the user and it feels like a familiar interface similar to a pandas data frame where you get a list of spark data frames inputted and we can start doing some transformations in this case we're identifying which columns we want to featurize on as well as then running a logistic regression example from pi spark with that we can see the output records and see how we were doing on fitting our model and training our model with this send interface you can intermingle between sql transforms and pi spark transforms to do python code in a totally safe space there's no way to cause any type of collateral damage of mutating any records outside of it or losing any data set it's a total safe space to just try whichever code that you need to try and get it productionalized and there you have it the top six reasons why looker users love to use ascend to recap number six any data source anywhere number five pipeline visualizations number four data quality number three adaptive ingestion number two data lineage and number one you get to play with cool stuff thanks for watching everyone come visit us today at www.ascend.io and sign up for a demo or a free trial