Sign Hawaii Assignment of Partnership Interest Computer

Document type sign assignment of partnership interest hawaii computer

each year microsoft research helps hundreds of influential speakers from around the world including leading scientists renowned experts in technology book authors and leading academics and makes videos of these lectures freely available boom alright so this talk is going to be about sort of give you an overview of project Hawaii and what it means to develop cloud enabled apps and then we will look at a video some of the other students that have been building apps on our SDK and then we'll do a little bit of coding and hopefully we'll get some interaction and you guys can sort of drive where we go with the code yes you too yeah yes and so the side that concludes the stuff that we're going to build in the future or that you guys can go and build right now if you want so as you know there's you know there's this growth proliferation of mobile devices kind of everywhere in our lives but their compute limited battery limited cetera et cetera bandwidth limited and then there's this growth of the cloud so the idea that we had was where the research had was to to build a bunch of cloud services and make it super easy for developers to access them so we picked for services to start and we'll talk about those in a second whoa I got this deck from a p.m. so that's why this that's why there's color and transitions I wouldn't be able to do this at all on my own so the so the benefits are as we talked about you know that the cloud gives you all this you can offer outsource or offload all this processing and composition in the cloud things that you would never want to do on the phone like image processing optical character recognition or speech to text or whatever right and there you go see the lightning bolt with the cloud in the sunshine so the so let's bring them together right the mobile device plus cloud with our SDK so here's the the for services that we that we ship in that are live right now that we shift in our SDK the Hawaii SDK first service is called relay which is basically a really lightweight pub/sub so publication subscription you create an endpoint and you publish to that end point everybody's subscribe to the end point gets messages and you create groups and you put endpoints inside of groups and you publish to a group and everybody who's in that group gets a message it's very simple then there's rendezvous which is basically anonymous name lookup so basically you have you have a human readable name and it gives you it it gives you somebody's endpoint that you can use with relay and so you know you can swap out those endpoints but the name say is the same as like DNS right so these are really lightweight services they're all hosted in the cloud and we have a an SDK sir an API surface for them on the phone and speech to text is what you think it is you speak into your phone and we you know we take that wave file at a certain at a certain sampling frequency send it up to our servers and it does its best it sends you back it top n confidence responses or recognitions yes sir we use a oh sorry the gentleman over there asked what's your name again Clinton yeah asked how that's different from tell me well it's it's not much different then tell me except that we have an SDK for you on the phone to use it our back-end we actually use the the windows 7 stack but soon we'll be using a different a different back-end to do this but they're all equivalent Microsoft there's lots of different speech recognition stacks so we picked one the mCRC team which is taking this on starting in a few days will will likely for their next Rev move it to a different service and then the last one is OCR we're partnering yes sir sorry probably about a year ago I seen with Windows Mobile they have the wiimote robot and that actually had speech-to-text assembly locally in the boss right so the question the comment or question was that we saw something called winmo robot which had sneak attacks on the device yeah yeah I'm not familiar with that one but it would be that would be useful because it's local but it may not be useful in terms of accuracy or multiple languages or composition so the advantage of having something hosted in the cloud is when you when you want to go and string these services together you don't want to do a bunch of round trips right because eats up lots of bandwidth and in CPU and you'll run out of battery before the whole thing is over so so if you host stuff in the cloud you can basically say well let's take you know let's take some speech convert it to text do some machine translation on it to another language do some text to speech and then send back a WAV file in a totally different language which is which is pretty cool but if you had to just on the client you're sort of limited to what you can compose on the client and your processing and bandwidth power on the client and the last service is OCR to take a picture of something and it'll send send the compressed version of that picture to our service and you'll get back its best guess on your on the text and again the these aren't you know industrial strength you know SLA services these are intended for us to play with and get an experience of how to how to build things and compose things for the cloud and how easy it should be on the phone 7 and they'll continue to get better with time so this is what we have right now so the leftmost column is the stuff we just talked about and then the mCRC team which is represented by g2 and victor where'd they go they're in the back strategizing the mCRC team is currently working on this the adaptive at profiling a trajectory prediction generalized image processing and matchmaking for games and then they're thinking about the stuff on the right no machine learning things location sharing so that you can sort of know where people are face recognition and and basically the theme that I hinted at earlier which is down here there's a service composition framework and that sign that sort of the Holy Grail they're going after which is being able to generically compose multiple services in the cloud from the client so again you don't have to do a bunch of round trips and you can basically say to to a cloud service look I want you to put these three or four services together I'm going to send you texts and all back you know I want an image back in different or whatever right an image back in a different language or for image processing yes sir then I've got some people on IM we're going to be raising questionable and someone just love me we're saying this is really cool they've never heard of this before it really excited oh this is good what this is going to cost for developers to use it's free obviously I was gonna say like a thousand dollars but it's free so at the end of the slide deck we'll have a link I think it's called high zap or something like that should it be a you guys can remember its top of my right now yeah there you go i zap so that's where you go get the SDK and if you have a phone if you're a student you can get a phone becoming a developer is free there's a special program for that and you don't have to pay the hundred dollars then you can unlock you could do devil unlocking your phone right apps for it publish it to the marketplace if you want to this SDK does not have a commercial licence for the apps I don't think I'm actually I'm certain it doesn't but you have to look at the competition for the intern competition find out for sure so I talked to Ben and g2 about the details there but but it's free so the downloadable tools are all free the entire visual studio is free for students the whole SDK for the windows phone 7 is is also free and available for your you know for your desktop and becoming a developer's free so this is not going to cost you anything except except time to invest in learning the tools and and building stuff and playing with it yes sir nice i'm not i'm not exactly sure what our I know that this SDK ships with a academic license which means that it's it's it's I we want you to use the SDK and the services to do research and public published in the marketplace and have fun with it but but I'm not sure you can publish the marketplace and go try to make a bunch of money off of the services primarily because we're hosting these services and they cost a bit of money to host them and Azure and we don't we're not going to keep them around for it for what we're going to keep them around we want to upgrade them to a commercial license later but not this summer are going to do it so right now the the SDK answer system is as game said with an academic license right now it's more in academic non-commercial use but in the next couple of months we would be updating that license to be commercial and the services right now are posted in Asia which means they're scalable you can actually have a lot of people hitting those that's not the problem it's just the license and we'll be updating it so so so essentially whatever you develop you can in the next couple of months push it out to the marketplace yeah thanks sergeant yes sir slowest Asians in in terms of the amount of the late in the second period of time yes along with is coming personal couple months does do you have a then to get visualizes over so then we say commercial it means you can use it for your own app but right now we do not have any plane to plan to charge anything they would still be free you would still be using them free of charge but you can use it to sell enough so that's your first question which was is there how how how often how fast can you hit the services basically our services are throttled to protect the back ends it's a it's a really generous throttle so it'll it will keep keep things fair and sort of avoid you know a denial of service attack that's the idea don't try it what's your name by the way well thanks Kyle note to self no just kidding so okay yes yeah so this project Hawaii is that going to require a phone update and if it does then how we can't publish it to the marketplace to an update comes up okay so the question is does does this SDK require a phone update the answer is no it doesn't it works on the v1 release and then the mini update afterwards and we have done some lightweight confirmation that does also work with mango but it's it's very well tested on on the v1 version of the phone so if you have a v1 version of film without mega don't worry about if you have it with mango it's most likely going to work just fine so it works both any other questions yes sir um so there's a whole sea are featured in bing so I'll write this one's different I believe the OCR featuring Bing is also cloud-based correct and this isn't that much different I'm not sure if the OCR feature being offers an SDK surface for you to write against or not and that might be the primary difference so the question was is that there's an OCR future in being and how is it different yes sir though I was just trying to comment there is no SDK for that future okay so so this is what I do he was we're told by guy in the back that there is no se que what's your name Chris hey Chris thanks okay cool so let's go back to where were we so these are these might give you some ideas for what you want to work on this is what's happening in the future for the SDK and the leftmost column is what we have available right now in terms of cloud-based services and the client-side SDK so a little bit more about the Hawaii project we're partnering with you know a bunch of universities around the world giving them phones and with their mobile classes and getting them set up to two so that's what we did last semester in the semester before getting them set up to build applications for the mobile phone device and then there's a bunch of a bunch of you know students picking the phone 7 and and working on Hawaii and lots of links down there below if you guys are interested which which universities and if yours is part of it you feel free to ask argument he has that list in his head here's here's some sample some sample apps I'm not familiar with all of these apps so I think you can get the gist there's a lot of them they're really diverse and some of them are really really interesting as well like this the flag down one I don't know what that one's about that looks really interesting and so a lot of a lot of students have spent quite a bit of time building some really cool app so it's it tells you that it's um it's relatively straightforward there's a lot of creativity out there and it should be a lot of fun and that's kind of what we're what we're really looking forward to seeing is what you guys will come up with and so some examples of what students are building now argument where do I actually click this the video okay cool so this is this is a citizen science one oh you know what I don't have audio hold on let me pause it the music is totally awesome too yeah do we have audio I think that's a mic let me try this no that's the headphones so we'll see what happens yeah innovation we were sort of distracting you guys at the whole sound thing so the premise here is they built as a citizen science framework backend and application for people to go collect scientific data for different scientific projects cool so all right so let's see so this is where you guys can get the the sdk from that we released today high zap just pretty straightforward easy to remember and then you can learn more about Hawaii at research on myself calm wack Hawaii are there any other questions before we get into some coding how interested are people in the the code aspects on the SDK and what the libraries look like are cool okay great all right we'll skip this then I mean there's nothing else but you know we won't talk about that that much more half ideas that you guys have had we were creating the services yes they thought it'd be really cool after people to create we had a I'm trying to think of them but one of them was like a menu app when you're in a foreign country this is one of my favorites because I travel a lot and I basically want to be able to go anywhere and look at the menu in whatever language and then basically understand it and have either overlays or have it come back to me and it could be specialized so it just understands menus really well and then takes the pictures or we also thought of an app that does optical like computer vision on the different images and basically says well this is you know kung pao and that's that's your vietnamese sandwiches or whatever it is what other what other ideas that we come up with when we were brainstorming favor yo speaking a language to give back the way you're spoken to the other guy right so you can have a conversation with a foreigner or another official you are looking the guy that's made me go right look at him so translation based ideas that are so you can have a conversation somebody doesn't share your language they don't share you you don't share their language and I'm sure your language other ideas were location-based advertising overlays or location-based like broadcasting and you want let's say you're out somewhere and you want to meet somebody that's interested in a certain class of things that you're interested in either belgian beer or gaming or whatever and you basically broadcast and its location based les bras guys there's games that came out of research there about taking your phone in this virtual augmented reality and running around not trying to run into each other's sort of Tron trails and leaving trails behind you on campus and things like that so there's lots of yeah which is I think a pretty cool idea to select a bunch of people running around with their phones and and the new mango phones have some some really awesome sensors in them that you can combine to do things like that like translation sensors orientation gravity sensors and and then spit spinning sensors so there's a lot of there's a lot of stuff you can do it's sort of the world's wide open any other any other questions yes it has we're gonna have it off the saudi see you can look like so the question was is a Microsoft machine translation language language translation service up and the answer is yes it is we don't have an API for but the API should be really really easy to do basically look it up you'll find the rest api is for it ticker code cut and paste it and you're kind of you know build build your API surface on top with it whatever object model you want to use it should be pretty straightforward I'm sure their samples that already do it it's very powerful okay so I'm going to unify all these guys like zero real estate here all right so what we want to work on I pick the optical character recognition app and i'm going to show you so what i'm going to do is I'm going to start by turning off the cool features i just added which is this one here so that there's no chooser okay and we're going to run it in an emulator come to Danny come on alright I'm telling it to deploy to the phone so how many of you are familiar with the development tools for phone 7 Silverlight the whole nine yards so development tools guys use them before silverlight okay phone 7 have you written your first hello world phones have an appt all right about it like a twenty percent of the room or so okay so i'll spend a little bit of time kind of orient you how how did you know how easy it is to build phone 7 apps so you create a phone 7 project which is in the new projects dialogue new project after you install all the tools which are free FYI and you come in here where did it go ah there it is so then there's a bunch of different phone 7 type of apps and they they come with a bunch of oil put boilerplate stuff and they're relatively straightforward to use and that's if you want to build a new app so we have we have an app now that's that's pretty baked and we're going to what what I'd like to do and you guys tell me if you think this is useful what I'd like to do is take it from where it is and then add an extra button and you can sort of see how easy it is to do that so we are going to oh the other thing is you can pick which where you want to go you want to go to the device or you want to run the emulator and so for now we want to run the emulator in debug and we're going to hit play and so it just deployed it to the phone and there's as you can see the phone up there alright so the first thing that happens it says hey your application ID is missing and so what's that so in the SDK we bake this this concept of an application ID which is basically a whitelist that you have to go physically request a special app ID stick it in your app and then we whitelist you in our services so we know who you are and then all the requests you make will go through and that way if somebody is you know pounding on the services are doing something that brings them down we can always drop that ID sort of it's not really authentication is basically usage yes sir yeah exacto if ID equals you know Kyle reject but now so so basically all the applications so you get these these four applications yes sir such applications have been denied II ended up kind of consolidating with the job ID one ID will get you you can use the entire SDK with one idea believe or is it ID tied to a service yes for one ID you can use all the services so you only need one idea you can use it for all of your apps so obviously it's not that strong I mean you can share your idea or publish it or whatever but it helps for what we needed to so here it tells you hey you don't have an app ID go to this spot and go get one so I did one I got one for us and this is sort of what it looks like after you go get the ID so you start out you start out with this and you say give me an app ID and you sign in with your live ID and you get this thing okay and then you go and you follow the instructions and we're solution Explorer boom you follow the instructions and you go to Hawaii client CS and near the top are the instructions again to you know go get this go get the app ID and you stick it in here right so that's what I did right so we'll stop the bugging real-time coding is always a lot of fun and interactive so you want to healing you want to keep it going otherwise people get bored right so here we go so here's the app ID and now let's see what happens so we deployed it and you have this button and it basically is the camera chooser so this is the camera chooser task and what it does is it brings up the phone's camera the person takes a picture it turns your applicator your application back with a pointer to the picture and then you can do what you want with it ok and now obviously this won't work on the emulator it'll say yeah it does its best but there's obviously no camera there so we'll go back wreck so now have have any of you deployed any of these applications to the phone before where did I put my phone awesome thanks okay so let's uh let's deploy an application to the phone could you give me the projector thank you there's a guy in that booth listening it's pretty awesome okay so here's the phone everybody see the phone alright so i just plugged it in and then what it did was it brought the zune client up if you can bounce back to my laptop so I can show them that'd be great so brought the zune client came up automatically and basically recognize the phone and solve a bunch of drivers and now knows that the phone exist so you need the zoom client up in order to deploy anything to the phone okay so we're going to going to kill this current session which is running in the emulator and we want to go to the device let's just do release just for fun and we're going to run it in debug mode so i can debug through it and step through the code or we could just deploy it and run it but i'll run in debug mode so can we go back to the no oh right so the other trick is you have to make sure that your phone is is open which means that it's not locked like the screens actually turned on there you go all right go okay can we go back to the projector please okay cool so as promised this is pretty awesome by the way I've never used one of these days promised there's the there's the the thing and you see these little R the lines on the side here that's that is I thought that was only available in debug Clinton what isn't that the case that these are only available in debug okay when did you buggers attached so this tells you frame rate memory usage things like that so a lot of good performance information so you guys can can keep your performance up all right so we're going to sorry soon is right right right right no you couldn't start connect WP kills it right I'm just gonna I'm just gonna be really nice to my phone I go to the start menu and started again shortcuts client light that's right so there's a little thing with when the zune clients connected you can't actually use an account which i forgot okay do you think this will work bloody we'll see so Lottie is one of my devs that romo to the most of this code so if it doesn't work okay yeah that's right it's all tested it ok so I'm going to want to take a picture let's just do it this way ok and going to say except and it's going to send it off to our service over 3g and we'll see what we get back bummer we have any other anybody I've got text anywhere again free and those works as I did a lot when we were in Paris showing this stuff off so converting a lot of you said it's on 3g right now right okay so we will fix this really soon actually I could you check this make sure this is on 3g so what I'm going to do without the phone is I'm going to can we go back to the computer please thank you sir yes sir the camera capture test there yes what about in mango when developers have the ability to then tap into the live camera stream if you can do whatever you want it for that like your service works with that just fine our services in character you just need to send it a well-formed image and it will send you back what it thinks it can recognize be up to the app to determine if the image was of high quality right what we do is we do resize the image to save bandwidth because the full-size image takes forever to send we resize the image to the size that our OCR partner told us works best for their service and and then and so that's what we recommend they do and it's actually in the SDK so grab an image you pump it in the skl get resize and send off if you want to play if you want to pre-process it that is which is pretty straightforward to do okay so what I want to do now is I want to turn on the chooser which will make this demo much much better if we're on 3g or Wi-Fi okay so I've cheated a little bit cuz in this for the sake of time because i have actually i'm almost done right i'm done at seven thirty i'm done at six i'm done at eight okay so i'll walk you through it though so what i've added is a photo to choose or task which is like the camera capture task so there's a set of these tasks you can look them up it's pretty straightforward and and then in the when the page starts up so this is the main page this code gets called as soon as the app starts this is the only and first page on the current app so basically what we do here is we verify the Hawaii app ID so that will bring you up the friendly message right and and then we we create this capture task which is a class that you want to create and keep then we basically subscribe to its events and when the when the tech capture task is completed it calls photo choose are completed right and then won the sorry is that intentional ok right ok cool so then what I've added is basically the corollary so you have a photo chooser task a new task and you add the event handler you guys ever seen how cool it is to add event handlers in your studio yes ok everybody knows I love to tell my developers when they complain that coding is really hard at tom all you have to do is just tab tab tab and your entire program is done I don't know what you're complaining about everything is built for you so let's see so this is the part where we verify kill that note and this is the on picture click so this is this is basically what gets called after we're done no sorry this this gets called when the the button that we're about to create which we haven't created yet when that button gets gets clicked this gets called as the event handler and brings up the photo chooser task which is pretty straightforward you just call show it basically put your put your program and I SCO's does its thing and a come and when it calls you back it calls you back with the image ok then let's go to the zamel and enable this button and again I'm taking shortcuts in the interest of time but we get done with this maybe we can do a little quick project so this is the button open image and then the open image ping is something that's part of my project so what I'm going to do is I'm going to take it out of the project to show you how easy it is to put in there actually not want to do that I just want to exclude from project okay so I have the image and email actually all right I don't want to waste your time on that so let's just want to go and add an existing item it turns out to be that the open image ping is here so I'm just going to add that now you can also drag it right if you have a bunch of images sitting on a folder you know add them to a project you can drag all in there there's a couple of things you can do with these resources you can add resources as a resource or you can add them as a as content and there's there's a difference content actually shows up in your virtual file system and a resource is actually part of your dll so that's that's the difference and and what I've done in the past is when I've had tons and tons and tons of resources that i want named and in the directory structure add them all as content and it just just makes your zap really big but it looks like it's part of your file system so it gets added by default as a resource which is totally fine for us right so now here's this button and I think we are good to go so I'm going to run it to see if the button actually shows up and I haven't forgotten anything really important right so there's the button this is the photo chooser button and when I click on it the code we talked about gets run and it lets me go and pick something I'm going to pick this five and it's going to use my Wi-Fi connection to go and and fail the location i do that you can get help i really can't go back to the bait related education short i worked upstairs well it's just something this library is indeed use this miles work or is this my word okay let's get another one right she's gotta go here where is Hawaii client it's either tab or cut and paste right that's how we program boom okay of course the silver may be down there yeah hopefully the server isn't down in that would suck okay you can also change the size of this guy it's not so big all right let's try again come on please work okay so what this five on the asphalt recognizes dot 11 ' space capital j which is very very close ok now another problem with not a problem but something you have to do with the emulator is if you want images on there you have to write an app to load a bunch of images on and so you can play with them or you can go to the phone so what I'm going to do is I'm going to go to the phone which has some images on it right now and use my Wi-Fi instead of the camera to go hit up some some recognition now that we've added this thing yes I will redeploy app because it has the new ID on it ok so I'm going to minimize this guy and this guy I wonder how I got d off from the figure that out later ok so I'm going to redeploy this app to the device or the release version and instead of just hitting f5 I'm just going to go and say yeah that's right I'm going to build deploy and it'll tell me down here how it's going ok so the deploy succeeded we're going to stay tethered and if you could switch me to the thank you sir yeah so client light we're going to pick a photo what even if I want to pick a photo wow dude no I don't want on tether because I want to use the Wi-Fi to do this for analysis okay and I picked the sample on purpose to show you what the reality is of programming on the phone with a complicated samples 04 boom ok let's try that awesome so there's a bunch of images that we took we're traveling around showing off our phone let's pick I love Paris I'm showing off yes okay that says I love Paris and there you go exciting so we have about 20 minutes left is there anything you guys are curious about that you'd like to do in the code one thing we can do is we can go and I can sort of show you the SDK the API surface for OCR and we can look at some of the others like speech to text and relay and rendezvous would that be interesting to everybody some thumbs up questions you do you don't do okay you want to see the really random vertebrae maybe said you don't I feel like too bad these are usable these on the okay so let me show you around the SDK so once you go to the site you download the SDK and install it and read the eula really closely come here and you'll have cloud services SDK for windows phone 7 108 right and inside here this is the this is a shortcut to the directory where all the samples are and then this is each each of these samples points you to the solution file for that sample let's just start in the directory and you have these are the samples here okay and then all the documentation and then this these are the client libraries so these are the libraries that you would write to the go do the hard work of not that hard but do the work and provide the object model to talk to the backend services let's just look at the OCR client library has basically the OCR services and and this is for this is for documentation and service agents and this is a base library that provides lots of utilities and services yes Ben what languages are supported just English right now and letter languages that use the same C sharp oh you mean give me a computer I c-sharp d-sharp my L thank you wanna ride in Al and BNP meeting so Santana English just English for now yeah and that's easy enough to expand once once we flip those switches and we expect mCRC team to do that right you two you're gonna you guys are gonna expand the language set for all these services that's right okay good we're back in the same that the bing app uses on here I don't I'm not sure comment speech is that the same as down speech speech uses yeah different there's four different speech stacks of Microso t's technologies there's stuff that is the backend of being there's speech that tell me uses there's a speech stack that's in the windows server 2003-2008 which is a totally different stack and then there's the windows 7 stack which is a client second that's what we're using because it's much easier to tap into I any there was another hand that went up yes sir what's the main difference between these reliefs of Hawaiian defeated what it was that's weak ok so this release of Hawaii adds a lot of stability moves the services to Azure adds the authentication not authentication whitelisting protection it adds some samples and and we did some back-end work to make sure that our services are up and running like logging and things like that so the Nina neither it just matter of us community yeah yeah just one thing I would like to point out yet soon running but you can run WP connect if you're compounding on your phone to get access you can so look into the debugger and have access to like your camera on can write your media library so the comment what a case of people that know that you don't have to have zune running you could use another application called WP connect which connect your phone to the debugger in the same way without using without you things you that makes your life easier so let's go look at some let's start with the OCR SDK so here you'll see that there's a client library which is a project that we point to which provides a bunch of services that everybody uses and then there is the and then there's our application which is the thing that gets deployed to the phone right here okay and then this is the OCR client library so let's go look at that where do you suggest I start Lottie what's the top level okay we can go from the drive right so this is this is that probably the primary method you'll use which is recognized image a sync it takes it takes your app ID it takes your a byte array which is your image buffer and then you give it a delegate it calls you back on that delegate asynchronously when it's done with the conversion that calls internally it calls another version of image result with a with a context object and this creates in this creates an eight agent and then the agent goes in and is responsible for processing the request and that's it there's basically one method call and you're done and if you want to dig into how this is all implemented all the sources there you can go dig into it but there's really nothing else I'm OCR is a pretty simple thing here's here's a bunch of bits tell me what you think is in there in terms of text so let's go and let's kill this guy and go and what do you guys want to look at next really really all right you're excited about relay so so we'll start with a sample because the sample has the right solution relay test client and they test client solution I didn't try this before so if it blows up lottie's fault ok now we all tried this ok so let's go look at the solution Explorer so here you go you see client base again which I'm going to minimize you see the relay test client which is the application and on the library and should we start with relay service ok so these are all the basically these are all the basically commands so you create a this is the the endpoint creation which is an async call this should look familiar by now right you pass in the interesting arguments and then you give it a method to call you back on when it's done and there's two versions of these this one has a state object what's the state object used for body ok so you want you bet let's say you're firing off a bunch of great clients and you want to know which one is which or you want to attach some custom data to each one and when they all come back asynchronously you want to know who you're talking to right so that's its really good to have context information on the callback this deletes the endpoint that you created this joins a group given an endpoint ID so basically if you guys remember you can create you can create a group and it has it has a name an endpoint ID and you can go and join that group so when you join that group you I you can you listen to all the messages that are published to that group and the group is is we have a group which represents the group right we have an endpoint which also represents the group is there correct or is the endpoint the thing that we're going to publish to or listen from which is the joins what I'm doing to you okay okay so basically but we're not getting back an end point right we'll get back to this not sure exactly how that one works should be in the documentation so this allows you to leave a group and all these methods are asynchronous and this this is the interesting one which is send send a message to an end point or sorry this is sending message from me and then a bunch of recipient ideas which are expressed the strings and that does it do a look up on the back end against rendezvous or those recipient ID is actually a bunch of endpoints okay so this is the string expression of a bunch of endpoints and here's the message you're sending and of course it calls you back when it's done and that's any other send message and this one so this goes and pulls messages to see if there's anything there waiting for you I receives a message and and this creates a group so you can go through all of these and kind of figure out how really works and go play with it pound on it and then use the I think we have two examples that use the that use relay and rendezvous you can play with those and see how they work and read the documentation yes sir in terms of your cloud services what are you using yes service for your relay we r you mean where is it hosted we're using Azure okay sorry said again by using a fabric use of Windows we're using blob did you are we using blobstore now to store all of our men's lights and messages no no daddy okay so we're using sequel Azure to store everything so you create an endpoint that gets slapped the sequel a jerk and groups and all that and the services run against that okay so I'm going to be sometimes it's a good point so a lot i said just remind me that we have we spent a lot of time writing how to sort of guide you through building these applications and something that would be instructive is the singapore demo which is the singapore yeah could could you project it there you go not protecting like a full screen right okay let's go check this out I'm gonna have to do smokey hi this is rajesh bother assistant professor how do i go full screen the Oracle sweetness speak with information systems at the singapore management university i'm presenting a demo application of crota owari seriously alright let's start again can you guys see that that's best big enough right hi this is rajesh balan assistant professor in school of information systems at the singapore management university i'm presenting a demo application for protocol is relay service in this application we have a video player that synchronizes between two different phones that are geographically dispersed what we can see in this demo is the left one is playing a video while the right phone is waiting because the left phone is actually playing parts of the video that right phone is already seen when the left phone catches up you will see that both phones start synchronizing at the same time this allows people who are geographically dispersed to be able to see videos together in a shared interactive manner they can also exchanged text messages with each other and they can also send audio chat to each other we will now see that if you look at the display down here you see that the left one is six seconds five seconds behind the right one and they have now caught up and both phones are in sync and everything is streaming through the relay service even though the video is located somewhere on the internet as I will show you now you can even send voice messages to each other this is a great video and that concludes the demo of the relay service demo application thank you okay so that was a really interesting use of reality to synchronize video by by publishing and subscribing to an end point yes sir in while you work on the desktop so can we maybe sync with a desktop app creation so it again to it that's not I'm not I'm not following like can I usually lay with like a windows program and then sink to that yeah i mean if if the windows program is talking to the real a client to the relay service yeah there's no reason why I shouldn't be so by using the rest so there's no problem you can easily communicate with them from a desktop now if you want to use the blank memory that you bro you may maybe you have to do it before when you go on and poor things because the sources should be the same which is the class library that you use is a fascinating for windows phone you would have to recompile it into a different class language for silverlight so to answer the question over recording question was can I can I use these the sdk on the desktop and the answer is yes you'll have to modify it or probably put it in a new project a desktop project and just try to compile it most likely to work it's done at 40 it looks almost identical and then you'll be able to write a desktop app that hits relay or OCR or whatever you want to hit and communicate possibly with phones as well because it's just a rest it's just a rest surface due to worry about to say something from anything right and it's just rest you can you want you can type your commands in the browser yeah on any client could you also hook the relay service up to the windows phone notification service do push notifications to other points so I don't think so the the push notifications is part of the phone 7 stack and it's something that you subscribe to with I believe with Azure or I remember seeing this in one of the early early decks and correct me if I'm wrong but it's basically you have to stand up for your own service ok to use it ok so you have to say hours I know there's no extensibility to the push notification kind of service so you're you what you're looking for if I'm reading you right is you you want to get notified when you have messages basically sure right we'll be able to push something and have another end we get notified right so I think the way you would do that is you you would have to stand up a service that sends out notifications and it's subscribed to the end points on the back end or you can pull which is not really recommended Oh question about the app ID thing zia the way you had it was like hard-coded into the application so and then usually how like yours became fee registered somehow if that happened and you had a bunch of users they would all just be deactivated right here you have to read update here yes so I don't know how I ID registered myself and it was just a theory that I got be registered I'm not really sure i may have refresh the browser or may have logged into a different live ID i'm not really sure but if there's an issue will definitely let you guys know is that we've tested it extensively and its really stable once you're in the whitelist year in the white list but yeah you're right that would totally suck if you gave a bunch of people your app and they popped out of the whitelist right so commercial academic licensing so if that does happen technically yeah there is no SLA but we want everybody to have a positive experience with it and the idea is that you you can learn and experiment with different kinds of things and there's these services that are out there for you guys to try and its really relatively straightforward with the SDK and the two link yeah I guess I was just wondering because like the way that the push notification perches you publish a device ID and then your server uses that to your service uses a lot to communicate so I mean when you do it that way if one person's being a jerk than that just experience with everybody right how be I'd be interested to see how you guys make that work we haven't we haven't really tried that scenario and I like the idea about push notifications and I haven't I won't pretend that I've thought people about I just know that it's a cool idea so a question back there is actually codeplex project for windows as a tool kit 7w a toolkit wp7 of perfection oh so the comment is there's there's a an open source tool kit for phone 7 and Azure called what's it called again it's w a toolkit wp7 ok w a tool kit that we do so so go play with it get up push notification tilt sensor so ok ain't gon create your cloud-based service and you're fine down to the web portal and assess all right that's probably has enough of Jeb Bush with another question back there was a person ever speech recognition ok so does it fit in a scenario where we actually need real-time speech recognition or it is still slower it's not real time they see but offset right it's time offset so you you speak you tell it when you're done talking and then it you tell it when to send it impacts everything up and sends it and a few seconds later comes back with a recognized speech yes sir that you didn't tell us when to start listening yes yeah you tell them so those three three buttons start listening stop recording and then send straight forth ok so lets you guys interested in looking at some more of the sdk surface or maybe speech to text yeah all right I I saw somebody nod so I you know went for it ok so let's go back to so here's a speech recognition test client so how many of you are looking forward to writing apps for the phone and like you're going to go get a phone sign up download the SDK play with it tonight instead of like going to Seattle and dancing come on raise your hands okay get perfect that's what I like to hear I at least a sixteenth of you okay it's a good start right so we've seen the client base before let's go look at the client library and this is the this is the actual sample app which won't look at right now but here's a speech service and a lot of this should start looking relatively familiar to you here's the get grammars a sink which returns to you the grammars that are supported I don't actually think we're using this right now so I'm going to skip this and then this is yeah it does it does work it gives you back a grammar and I think it's a really small set so you pass that grammar back to us and the reason we did this is so that we could grow the service to actually providing lots of different grammars without having to republish the SDK right and this is the recognized speech async you get the app ID the grammar you're interested in the speech buffer which you captured using the I think it's called a recorder task microphone task and then the on complete method and that's basically it that's the entire API so you know ten years ago in the dark ages you would have to be you know machine learning specialist and write five years worth of code in order to do any sort of speech recognition now you just show up in Seattle and there it is one line boom okay and let's look at the last one let's look at rendezvous actually knows rendezvous test client and this is the last one and after this do we want to take questions or just sort of know about and socialize ok I think we're scheduled to network and socialize and ask questions all of us will be available up here too to answer you guys questions and we hope that we hope that this gets you somewhat excited about this platform and realizing how much easier it is to to write interesting apps and hopefully your your creativity is going trying to figure out some cool thing you can do to win the competition and go to Hawaii guys know that the first prize actually physically goes to Hawaii right it's not like Oh welcome to Hawaii there's a room called Hawaii i will give you phones like the actual island FYI ok rendezvous storage service ok so basically this is you you give this guy name and it gives you back the on complete method gives you back the actual ID of that name that you can go publish to so think of DNS again here's the state object this registers a name so you basically want to you want to create a name that other people can can look up and then you register the name with your you register the name and you have not given it the endpoint the e's a separate API for that ok this is unregistered or delete a name and then this associates ok so you register the name you get it back an ID for the name then you take that handle and you say so she ate this name with this endpoint and it it's basically like a hash table right and that's basically it and disassociate ok so I know why we have an associate decision so that's so the name stays constant and you can always plug in different endpoints there in case you you change your group or you want to you you feel the turn it off you want to turn it back on or whatever so everybody has it's like having it's like calling somebody with their with their first name which is never going to change but their phone number always changes right because they're in Paris and have a different phone number or they they got google voice or they they got it you know they dropped 85 years ago and got verizon or whatever so you can always publish to goes go look up that person's current contact information which for us is an endpoint using the name and that's the whole point of this api and service that's basically it any any last questions oh cool hope you guys get pumped up and motivated to go play with this stuff and we're available been runs the zap fest program he's basically mr. phone 7 jutsu is jutsu and are Thurmond and Lodi are and myself are responsible for the Hawaii SDK and the way it looks in the next version of it so thank you very much for showing up and hopefully we'll see you around campus take care