How To Sign Oregon Banking Presentation

How to industry sign banking oregon presentation secure

hello everyone my name is Winton and today I'm going to talk about Helen maliciously secure cooperative learning for linear models and this is a joint work from Luca with Luca Joey and Jung from the UC Berkeley rice lab so let me first start with a use case which is urea use case I think some of you may have seen before and it's also needed by or as a rice lab our industry collaborators so here we have a bank who wants to detect fraud using machine learning however fraud is kind of difficult detect by a single bank because criminals try to conceal their illegal activities across many banks so really the ideal way to compute a model for fraud detection is to join a computer model on customer transaction data across many of these banks however this currently cannot happen right now because the banks cannot share data with each other because they're in competition with each other and also possibly to do some privacy regulations so one possible solution to this is to use a trusted third party where the banks actually transfer their data to to this trusted third party and then this third party outputs a model and gives the model bags to these banks however this is not very ideal because again from our conversations with both the banks and the trusted third party we learned that actually they have some various concerns from both sides so from the bank side they think that the trusted third party is rather expensive and they don't know how you know trustworthy this third party is really because we have seen many cases of third parties being breached and actually from the trusted xxx part of third party side they were also a bit concerned because now that they have all this data right sensitive data from the banks they kind of become a central point of effort of attack so they are very concerned for liability reasons if they are actually compromised so now the question we want to ask is how to actually allow these organizations to cooperate in the press of competition without sharing the data in playing so plain text with each other and without outsourcing to a trusted third party and we actually named the setting co-op position which is where you know the cooperative learning comes from which by the way is a real word it's this is a definition taken from the dictionary and we thought that along with the pronunciation guy you know if you're interested and we felt like this setting really captured or this word captures the setting really well because you know by definition means that you have collaboration between business competitors in the hope of mutually beneficial results and this competitive setting corresponds exactly to a really well-known research direction in cryptography called secure multi-party computation and the goal of secure multi-party computation or MPC is to have the parties emulate a trusted third party via cryptography but then without actually using the trust Authority right so the nice guarantee that it provides is that no party should learn any parties any other parties input beyond what is learn from the final result and this kind of gives you the security guarantees of utilizing an ideal trusted third party service now our bank is has heard about MPC and is very excited to utilize it to compute this fraud detection model he wants to make sure that its desire security properties are satisfied so from a bank's perspective right what the bank wants is to only trust itself and it wants to make sure that its data confidentiality should not depend on the correct behavior of other banks and the state of privacy is actually guaranteed even of all other banks misbehavior and here are one kind of ones who emphasize that our cooperative setting is a little bit different from the federated setting that was mentioned earlier in that we assume that in this setting the banks also do not want to leave the intermediate global models either so the desired properties from the previous page are roughly translated to the following threat model right so you have the secure computation you want to have the secure computation done among all these parties among the banks themselves for exam and we want to assume that this hacker who can compromise you know subset of these particles actually compromise p- 1 out of p parties and we also want to have a protection against a malicious attacker which means that the attacker can actually deviate from the protocol by you know trying to maybe execute a different functionality or substituting ecosystem puts into the computation however under this threat model we do allow the parties who input data of their choice and we also you know allow the parties to learn the final model because this kind of Wantley words and functionality of the of the training process so with this if we take a look at some of the prior work in this area are currently actually the framework that handles the requirements is a generic framework and pc framework like speeds however utilizing you know by doing training in using a more generic and pc framework is kind of challenging because of performance so we actually implemented stochastic gradient descent or SVD which is a commonly used training algorithm for lasso which is a you know simple regularize linear model using speeds and for in the setting of for parties hundred thousand samples of party on a data set of ninety features we actually estimated i would take three months to train this model now we couldn't actually run it and you ran out of memory way before and so to solve this problem we designed and built hella which is a training system that gives you this strong security guarantee protection against malicious hacker who can compromise a majority of the parties and we decided to support a more restricted subset of the machine learning which are regularize linear models but we thought like that still solves a very significant slice of machine learning and statistics statistics problems like cancer research genomics and financial applications and as it turns out trying to improve performance under such a strong threat model it's a pretty difficult problem so for Helen we really had to elaborate techniques from three different areas machine learning systems as well as cryptography for our solution and actually we found that with our insights were able to reduce the estimated three months to under three hours of training the same model in Helen okay so now I'm going to quickly go over give me you an overview of our techniques from these three areas so first the first problem of will fit what we faced was the fact that a common training algorithm like STD right scans the entire dataset and and actually requires so if we were to put that entire training algorithm inside NPC would require an expensive and PC synchronization for each iteration so we instead actually use an alternative approach so we use a different training formulation called a DMM that is crypto friendly and in our model a DMM is able to scan the data once in the beginning to produce a summary of the entire data and then during training it will repeatedly use this data and with this and the number of iterations as well as the NPC synchronization both are greatly reduced however adapting a DMM to secure computation efficiently it's actually so a bit tricky right so if we look at the way the summary is being computed we actually face a second problem because we're in the militia setting right the attacker who can compromise parties could actually deviate from the protocol then we actually need to make sure that the summer in creation is also proven correctly that you know the Hardys need to prove that it has been computed correctly unfortunately if we will put the entire summary calculation inside these cryptographic proofs then becomes pretty heavy way because the proof now has to take in the entire large training dataset as the input and that was scale in the number of samples of Part II which could be very very large like in the millions so we instead so um yeah yeah so on this so our second from the system side is to generate a create a scalable proof generation of these summaries using our first singular value decomposition to reduce the input into a much much smaller emphasizes and this can be done locally in plain text each party and then the zero knowledge proof that we use will take in the much much smaller data set and we prove in our paper that this doesn't the still provides the same security guarantees as before and so we made so yeah okay so finally if we take a look at the iterative training process that we used as a summary there's actually a third problem because the original 80mm formulation actually allows for nice decentralized computation but then when you formulate that inside of your MPC is centralizing the computation because you're requires ever going to participate for the decentralized computation or each piece of the decentralized computation from before so in how long we actually bring back the decentralization by using a mixed protocol that combines both home Orphic encryption as well as MPC together so that you know you can still do local computation and for the other parts that require global synchronization we put that inside of MPC okay so that is an overview of our techniques and now I'm going to quickly go through a Helens a workflow so the first stage in Helen is the setup when the parties need to come together and agree to do collaborating and also as you agree that they will release the result which is the model in plain text and then during this phase during the setup phase the parties also need to do a one-time initialization step to generate some parameters that Helen will utilize during the actual training in this case we are creating a threshold morphic encryption scheme that allows that gives us the strong security guarantees that we have we are since this is done once if you are at the beginning if the party and can be reused if the party configuration does not change we do is we simply put the key generation instead of MPC and and the key generation will give a public key to you party and actually split the secret key into share so that each party has a single share and the property this encryption scheme has is that you all parties can encrypt but then you need all P parties together to decrypt the ciphertext so now it becomes the input preparation phase which as you remember we have to create a summary at the beginning and this is that step so during this stuff we the party is locally pre compute and prove that these summaries are you know computed correctly using our insights to first of all reduce the input training data into a much smaller summary and the generate prove that it's been done correctly and each party does this separately in a decentralized way after this step we go into the iterator tree iterative training process and in this iteration are in the in each iteration rather there are two steps the first step is a local optimization step where the party is actually compute improve some local computation in a decentralized way using the summaries that they pre computed in the input preparation phase as well as the global way that you see that they received from the previous iteration where the initialized weights if you're in the first iteration so in this case what they do is simply they use home Orphic encryption to operate on their summary which is in plain text as well as the global weight which is encrypted and then each party does this in a decentralized way and next during the next phase for each during the free iteration is the coordination step where the party is coordinated together to compute a global weight from their local weights and this is done just doing or where or we're using MPC for this step and at the end of this step the global ways which is computed is distributed back to all of the parties and the parties can start training process again so once all the iterations are completely the parties then use their secret key shares to join me do you crypt the final weights and the final weights are given back to all the parties in plain text okay great now I'm going to go into our evaluation so in terms of the experiment setup we used for parties we experiment on ec2 and we put two parties the Oregon and two parties in Northern Virginia to kind of simulate this more wide area network setting the baseline that we compare to is SGD implemented in skis a generic maliciously secure MPC platform and we make some training assumptions in that SGD has to scan the data the input data once and for a DMM because we're able to as I mention before I want every insights that we can actually summarize the data in the beginning and then it early train on that we actually experimentally found the maximum number of iterations I would take for our data sets to convert which is 10 in this case and I'm now going to show accuracy results here but you can take a look at that in our paper the howling is able to achieve the same accuracy as the baseline model okay so I'm already going to show our evaluation on one dataset we do have more in the paper and this dataset is a sound prediction data set from UCI which is a machine learning or repository information you know open source machine learning data sets and for this one it has for this particular dataset has 90 features and so here's a graph on the x-axis we scale up the number of samples for Part II and on the y-axis we that we have the time in seconds which is a little bit hard to read so these are more human readable units on the right side and so this is the summated baseline which is in blue and as you can see it scales linearly because as I mentioned before HED has to scan more data as you increase the number of samples per party held on the other hand looks constant actually does scale up pretty slowly the reason it looks constant is that the cryptographic operation greatly dominates the plaintext operation in this case and while the plaintext operation scales in the number samples it is very very fast so and the crypto we designed in such way so that doesn't scale in the number samples which means that you know you see this you're kind of constant behavior and for a hundred thousand samples which is a pretty moderate size we see a roughly 900 X performance game okay so in conclusion Hellenes a maliciously secure cooperative training system for linear models and we're in general very excited about this direction because we think that enabling data sharing can create a lot of value and yeah that's it thank you so things questions so maybe I start with one first question so you basically work with linear models and I I was wondering whether you can extend your approach to also support nonlinear it's these functions maybe not models yeah yeah so I think some of our techniques can definitely be extended to more complex models I'm actually looking into extending into logistic regression for example are just a regression training because yeah so basically the local computation part can be translated into that kind of a lot of as well for new or networks I would say it's a bit more challenging good yeah I had something different than mine so maybe you know random Kitchen sings is this a technique from ten years ago where you can basically take a linear model and then apply some modifications to the features and you get a little bit of the power of nonlinear classifiers and my feeling is this could be perfect yeah yeah sure sure I I also a I think that would be done in the future they shoot snappy but you think right so yeah I think dial approach is definitely complimentary to our sitting as well so other questions okay so then let's thank the speaker thank you [Applause]