Send Populated Time with airSlate SignNow

Send populated time

hello and welcome back in the last episode i showed you how to write data to an sqlite database in this episode i'm going to show you how to optimize the flow of data to your database increasing the life of your sd card and in the process i'm going to teach you about contextual variables this is the flow from the previous episode i've made a minor change in that i am now pushing humidity data through to the database as well as the temperature in the previous episode i just sent the temperature across and this will be useful in a future episode now you may be wondering why do we need to limit the number of times we write data to our database no matter what medium you use to store data on whether it's a hard drive or in our case an sd card they have a limited number of read write operations that can be performed on it a modern sd card can manage about a hundred thousand writes to each sector if you are collecting data from your temperature sensor once every second that would be 86 400 times that data will be written to your database every single day or 31.5 million times per year now it's not as bad as it sounds because what happens with modern sd cards is they have a form of wear leveling and what that means is that it's not going to write to exactly the same sector every time it's going to spread the load across multiple sectors this is one reason to get an sd card much bigger than you think you'll need because it allows far more wear leveling to actually occur one obvious and easy way to limit the number of times you're writing data to your database is to send the data less often but i'm going to show you another way first let me introduce you to the concept of contextual variables in previous functions that we've written we can see that we're using variables but these variables here are only applicable within these functions i can't use that variable outside of this function it basically dies as soon as this function is run and completed a contextual variable is something that can be used outside of this function there are three contexts to concern ourselves with the first is a node context and we'll talk about that a little bit later the second is a flow context and this is visible to all nodes on the same flow so i could use a variable that i declare outside of this function within this function or this function and so on and then the last one is a global context now that means i can use it outside of this flow i could use it in this flow as well and any future flows that i create it's best practice to avoid the use of global variables wherever possible so we're going to focus in on flow contextual variables and node contextual variables let's create a test variable that is accessible throughout this flow we need to add a couple nodes let's add an inject node as well as a function node and that's just so we can call the function in the function itself we don't need to return anything but what we will be doing is creating the variable and to do that type flow because we want this accessible within the flow we're going to set and we're going to give it a name which in this case we'll call it test one and we need to give it a value in this case we'll give it a value of 500. we also need to be able to read that variable so let's create another function and we'll create an inject node over here as well just so we can actually activate it and in the function instead of flow.set we're going to be doing flow dot get and the name of the variable which in this case was test one message payload is equal to flow.get and that returns the message and that means we need to now also create a debug node which means we can read what that message is that should be it if i've done this right let's deploy this and see what we get so deploy we're going to put it on the debug side so we can see any messages so click that you can actually see some of that information if you go along to context data you can see any variable that are created within the context of the flow will show here which we can see it's there but let's see it in the debug node if i click this there you go message payload and the number is 500. let's do something a bit more useful than creating a test first let's delete this because we don't need it and we're going to change this function here instead of it being set to test one let's create another variable called buffer length the way that we're going to be limiting the number of writes to the database is we're going to store up a number of values and only send that data across after a number of times that we've actually collected the data so for example if we send temperature data once every second we could choose to say that the buffer length should be 60 which means that we'll only write to the database once every minute because we're going to collect 60 readings before we actually write that all to the database so let's use that number for now 60 we'll say done and that means now we have a buffer variable created let's give this inject node a relevant name so that it makes more sense with what we're doing and make sure that it injects once and that'll make sure that the variable is created we now need to modify the function that writes data to our database currently it takes data from our sensor and it creates this sql query it sets the topic to the sql query and it returns that message which then executes against the database and writes to it we're going to change that instead we're going to cycle over build up this sql query and only send it across once we've got 60 readings let's take a look at this code the first line is calling the buffer length variable that we created earlier so it's flow.get pulling that buffer length back and it's assigning it to a local variable if it cannot get this variable if for some reason we haven't specified what that variable is then it's going to return 0 and that is just going to make this function act like it did before it's just going to push data to the database every time it gets a reading from the sensor the next three lines haven't changed so let's move on to line number five this is an interesting one we're creating a variable called count and we're assigning it a value from a contextual variable which is context.getcount now we haven't yet initialized it if this is the first time the function is running however if it's the second time the function is running we actually have initialized it you can see down here within this if statement which we'll get to in a second so what we're doing here is we're creating that variable called count we're assigning it that value if it hasn't been created yet then we're going to assign it a value of zero because that's just going to start it from zero and all the count is doing is counting the number of times that this function has executed over here we're creating a variable called the sql which we're assigning the value from this contextual variable called the sql if that doesn't exist then we're assigning it a blank text value this is where the interesting stuff is happening we're checking whether count has exceeded buffer length if it has then it's going to execute this code here which essentially is returning the message which contains the topic of the sql statement that we've been building if it isn't bigger than buffer length then it's going to run this part over here which is basically building up the multiple entries from the mqtt sensor data that we've been retrieving if you want to understand a little bit more about sql i would suggest going on to w3schools and you can learn some fundamentals there but essentially this is just building out an sql query that we will then be sending through to our database and it will input multiple entries all at once i wanted to clarify something if you haven't worked with contextual variables before it might look a little confusing because i'm naming this variable here the same as my contextual variable these are two different things this variable here that will be destroyed when i close this function however when i go down here and i set the contextual variable the sql to the value of the variable that we've been creating over here then this is going to be available to me the next time the function runs by using context.getthesql so you can name these differently if you like i name them the same because it actually works out better for how i approach this code let's deploy this flow and see what we get let's have a look at the context data so we can get some idea of what's going on if i refresh this we can see that the flow context data we have buffer length at 60 which is fine because that's what we've set you can also see that test 1 is still there even though we've deleted it from the function but that's because we've set this right so we haven't deleted it anywhere we've deleted the function but we've actually set this in memory if we go and reboot our raspberry pi that will disappear buffer length will still be there because we are executing it again we look at the node context data if i refresh here we're not going to see anything because i'm not selecting the correct node this is the node we want to select i refresh on that and we can see that data is building out within this sql query we can see that we have two variables that have been created in context one is count and we can see that the count is at two it's at three now we've got three entries within this sql query now one of the problems that you might have is let's say i wanted to reboot my server halfway through a cycle so let's say we've made it a very big number let's say we made it 120 on the buffer length and that means we're waiting for 120 entries before it actually puts data into the database what you might want to do is have another inject and function node created so that you can manually make sure that it purges all that data through to your iot database now this is one reason why i used this whether count is bigger than buffer length because what we can do here i'm just going to copy the data from this function what we're going to do here is we're going to make the buffer length 1. now you've just got to remember to set it back again and we can do that in a few different ways we don't need return message but we can set that there and we're going to put that and let's do this we'll give it a proper name so we know what it's doing um purge data so what this is going to do is make sure that all the data that is sitting there in this sql query will actually be purged and sent through to the database so that you can then reboot your server so if i go back let's go back to the context and we can see what's going on here context data let's click on function we can see that it's still building out we've got eight items in there when i click purge data i'll refresh down here as well the buffer length and you'll be able to see that that's going to turn to one actually i've got a bit of a problem i need to deploy this before it'll work and what's going to happen even when i just deploy a new function it's going to lose all this data as well so this is another reason why purge data is quite useful so let's deploy this unfortunately we're going to lose those entries let's refresh this we can see that started again we've got one entry in there and this year we can see the buffer length is still 60. if i just wait for a couple more to come in if i click on purge data now we're going to set the buffer length to 1 which means it's going to force this count to drop down to 1 because the next time an entry comes in the count is going to be bigger than buffer length which means it's going to push the data through to the database as you can see over here this is just going to continue just sending data directly to the database until we redeploy or until we reboot the server so the other way to get around it if you don't want to deploy something if you press this button by mistake you can just press that and that now is injected and you'll see that this is going to start building again and well the value here let me let me show you what that value changes to changes buffer length to 1 and if i click on that it changes the buffer length back to 60. before we finish up i just want to show you what the sql query looks like once it gets built out so to do that we just create a debug node we're going to connect that function to the debug node we're going to change this to topic because we're sending the topic across to the iot database that builds out the sql query and we're going to deploy that and you'll be able to see that come through now let's have a look at what it looks like when we aren't building it out over 60 iterations so we'll just purge data and that should pop up in a couple of seconds and there we have it there's an insert statement that's been created that's what's been sent across to the iot database we can see there are a few entries in there because i clicked on purge data after it already gathered a couple entries so let's change it back to startup exec we're going to do that so it's going to be 60 entries and i'll speed up through the video to show you what it actually looks like after 60 readings have come through finally okay it took 15 minutes but there's our entries you can see how that's been built out you can build quite a large sql statement so obviously here it's cutting it off because there's a limit to how big this field can be in the debug but you can get the basic idea of what's going on so that there is now injected into the iot database and it's populated with 60 entries and it's only done one write operation versus 60 write operations during that time now you can increase this field over here the buffer length you can increase that to however much you want just bear in mind if there is a reboot on your raspberry pi or if you're deploying new nodes things like that then you potentially could lose some data so just be reasonable with what you set this to i hope that this video has been useful for you i've taught you a few new concepts in particular i think that contextual variables will be very useful for you and i'm sure you already have some ideas of how you're going to incorporate them into your own projects i've also showed you how to optimize your data flow through to your database which should actually allow you to take in more sensor readings more often without actually impacting the hardware down the line as always i will leave all the code and the flow data inside the description below if you have any questions let me know in the comments and i will do my best to answer until the next video thank you so much for watching stay safe and stay spicy