Sign New York Banking Permission Slip Mobile

Industry sign banking new york permission slip mobile

I'm excited to welcome to the stage James waters in New York great to be here I came here five years ago for the first time I was March of 2014 to talk to the New York Technology audience about micro services and platforms so it's really great to be here with the Kafka community today kind of taking a health check on where we've gone as a community and I think as jr. articulated there's an exciting next gear that we're all finding together what we're doing with these applications my talk today is basically that there's these two fundamental things that I think you need in your organization has technology primitives to help you and the two that I've come up with or everyone's gonna need spring boot and everyone's gonna need kafka and you might need other things but those are the two things I see everywhere when teams come to San Francisco to talk to me about where they're going as organizations they kind of come in and they give me the quick introduction and I'm waiting for it right they're like well we've chosen spring food and we've chosen Kafka and we'd like your advice from there so the talk today is a little bit about how we're making a big bet in that ecosystem and also the evolution of how we got there and how our thinking has evolved and how I think it fits into that transition we've already been making to continuous delivery and micro services the thing that our clients asked us to do our two primary things they want to learn how as an enterprise to start competing with those new up-and-comers that are attacking them in their traditional industry often or to accelerate their attack on incumbents and that centers around continuous delivery but the other thing is that these enterprises they can't really take on all these new micro services all these demanding new projects without thinking about a platform like there's just no way to go 10x the funding that they have and every team just goes to get off and get a billion dollars so they have to think of these platform primitives what they can get good at as an organization in a standardized way so these are two big things that our organization asks us for and why I really care about you know the right emerging technologies I got the whole way through all of my slides without saying the word paradigm shift it was really an act of discipline for me it's one of my favorite terms and anytime you're a technologist you're really excited to participate in a paradigm shift but that's sort of something that you let others say you've accomplished a paradigm shift but when I look at what's happened in these last five years if you look at how people used to do IT projects that would start you know and then 18 months later they get feedback from their customers or 24 months later I was talking to a bank in New York once they're like it takes us 18 months to change a button on an ATM but that was how the organization had been built does that sound crazy to people is that people seeing that yeah right like that was sort of the model and you'd have a big bang project everyone would fill their role in the swimlane and then you'd go execute sequentially that was sort of how it ran and the architecture of the organization also reflected that you had a monolith or all of your you know application infrastructure lived you had a monolith for all of your data lived what's happening is that these bonds are being changed by continuous delivery and so from left to right continuous delivery is radically reimagining what's going on and so you know instead of this old I'd sole process see ICD we used to think of just one virtual machine or several virtual machines for your project and then good luck and now developers are demanding API is to be productive immediately like that's the car that we were talking about and fundamentally the lower level infrastructure in the application has gotten changed as well used to have heavyweight Java EE and now spring boots everywhere if people heard of spring boot raise your hand if you've heard of spring boot pretty good I'd say we're about 90 percent and frankly the other thing where we talk a little bit about you'll hear all about at this conference is the expectations of what was a batch process that might have result in sending a letter in the mail and some of these applications it now needs to be a real-time application that connects with you on your mobile device so this I do believe is a paradigm change and we're helping organizations get through it the gravity of this situation is so so real that even the business intuitions are that we need a platform to enable these teams like you can't ask their teams to go through all of that transition that I just showed without some sort of enabling platform so even the folks at McKinsey now said hey you're gonna need a platform and largely that's the business pivotal has been in providing that and we made some progress I'm going to talk about our progress a little bit and then I'll talk about some of the you know the obstacles we encountered and how I think this new streaming ecosystem is going to be an exciting new addition to that what's with spring we've had to make a couple of bets on what the right distributed programming model for micro services are and our biggest bet that we made years ago was betting on Netflix OSS and we've made some progress there where even the Netflix team is then adopted spring boot as their core framework so we've got this great big web scale company that's now gone all-in and what we do is spring boot that's how they've done it and I would say we've largely solved the micro services with Java core framework problem you know in partnership with Netflix and I see the relationship we have with a Kafka ecosystem as another double down on what we're doing with this micro services ecosystem to make all of these new primitives easily accessible to developers we also gave enterprises a platform they were facing the need to go compete if you look at a company that's a disruptor in their space t-mobile is the only mobile carrier right now taking share and they've taken this aggressive approach they're using the pivotal platform you can just see some of the results that you get from this platform thinking they've gone from six months per release 10x two weeks when they ran in a project way when they wanted to scale one function in one of their applications it used to take seven months there was this one function an application they need to scale up and scale down and they ran a project they'd have to use you know hardware and stack it out now with our platform they're getting much greater efficiencies and these things come back to me like moments with them over dinner where they said hey it used to take us 500 people to keep that application alive it now takes for that that to me is worthy of the term paradigm shift so we've given people this platform and micro services and they changed what they're doing but if I'm here for one reason it's this slide because this is sometimes where you hit if you haven't reimagined the fundamental data infrastructure if you haven't rican sieved of how the teams are working together you can get the quick early wins and then hit the wall of yeah but the data team owns that yeah but that's a pretty pretty thick monolith for us to go to unpack and so you get these you get these wins and oftentimes this is a huge win for them they save tens of millions of dollars they deliver for the customer but they get stuck here my passion on Kafka and this ecosystem is largely about what we do after this is this familiar to folks like you might get some micro-services wins and then there's still work to be done so we really needed to help enterprises do a further fundamental rethink and there's this term convergent evolution convergent evolution is why a shark and a dolphin look a little bit alike because they're both competing and adapting to an environment that's similar and I think what was happening is that some of the fundamental work being done by the team at confluent and the Kafka ecosystem was a convergent evolution to this event based microservices world that we've been in at pivotal and so there's this fundamental ubiquitous language around events that we discovered to help people think of how they do micro services and one of my favorite quotes to come out of the coffee because someone may ha came and spoke at spring one she basically hinted that not only when you adopt a core infrastructure technology do you get a tool you also get a way of thinking and the biggest challenge for us is that the way some of those back-end systems think today when they think in terms of oh they'll be a batch process that in 72 hours will spit out a result that's a different way of thinking the way monolithic technologies were built were also a way of thinking and I'm very excited about this new way of thinking we have with microservices so what did we do confronted with having gotten only part of the way through a transformation of these organizations we turn to a tool called event storming and we've done hundreds of these app transformation engagements this happens to be with a born on the web company named StubHub and stuff up came to us they said hey we've got this good web application but the way we've architected it we really want a lot more velocity like we've gotten to cloud but we need to get to continuous delivery and I would say that's the trend it's we're not just trying to change the infrastructure we're trying to get to continuous delivery and so we help them unpack their monolith through this you know approach called event storming and we learned some things to hundreds of these engagement with enterprises this convergent evolution with the coffee you can system happen for us and what we learned is that the ubiquitous language that we can approach both a business person who's doing product management as well as a developer is thinking of these fundamental events we also learned that the closer we stick to these events and to these bounded contexts the better we keep the incidental coupling between services apart like the enemy as you start to do these transformations as you start to maybe decouple deployment but you know topologically you still have incidental coupling and making a change without affecting everybody else as a distributed monolith and the the number one phrase that the team told me to repeat to everyone here in the audience today is tell don't ask and that helps you limit the incidental coupling and so we see this as the next evolution of where we want to go with micro services we want to make the contracts between these teams more event-based more explicit and we also want to enable that next exciting use case and I think as we talked about that convergence of the data centric use cases as well as the next business application use case and these in these platforms is what's so exciting I'm going to talk a little bit about those next so we believe this is the new platform it's based on a new set of principles it's built for the world of that left to right where you want to do continuous delivery with autonomous teams and you want to have data enriched applications that are really gonna allow you to go compete they achieve arbitrary scaling a scope across enterprises one of the coolest things about building a micro service platform for me was how fast you could get onboard it like we had customers where they said we went from zero to forty thousand containers in nine months you know we've adopted 900 we've onboard at 900 developers in six months and when you have this scalable primitive at the heart of your enterprise that anyone can come in grab self-service go be productive go show their boss that's just an incredibly powerful thing and that's a platform so you might think of this as oh hey we've got these new autonomous teams they're working across the stream platform they're using spring boot etc I want to show you the most important line on this diagram though it's between the people and no matter what we do with the fundamental application infrastructure the number one critical to success is these teams have to talk they have to discuss how they're actually going to evolve these contracts between these micro services and the work that we do is really here just to enable that top arrow if you can take one thing away is that it's not just a technology solution the technology is there to reduce the toil allow you to be productive quickly but if you're unless you're having conversations about how you're evolving the product how the API is between those teams need to work and keeping you know awareness about the incidental coupling we're here as a platform to help enable you to have those conversations don't forget that part probably the number one key to success I've seen is you have everyone betting on a few common primitives and then everyone in a slack Channel talking about it because imagine one team one person in organization trying to go tell everybody here's how to do this new pattern versus suddenly everyone's using similar patterns and educating each other that's the number one key to success we've seen so this approach has had immediate impacts in the back half of this I'm going to go through them quickly give you some highlights so this is one of these you know days in San Francisco and you know a really big important Bank comes in and they say the classic thing we're using spring boot and Kafka and we start diagramming their architecture and what they want to do and they said a really important thing and this is the back to that critical success factor they said we no we need platforms to allow us to focus on the evolution of our micro-services that's what we want to focus on so now I got really interested I was like these are real players the second thing they did is they started talking about Kafka and the same way that Martin Clement talks about Kafka and they showed me their diagram where everything coming in externally was written a Kafka first and then this polyglot approach after that and they started saying you know we really count on Kafka for consistency strict ordering we can replay everything there's durability there's audibility and these are these exciting paradigm shifted moments where the technology has allowed them to reimagine how they think of data they got cross called cloud replication out-of-the-box as well this is happening all over banking like I really think in the banking ecosystem this approach of event based micro services is going to become the norm over the next five years as people take on new payment api's even new core banks very exciting use cases there there's also you know as Jay mentioned real-time inventory we do work with shipping and logistics and you know client came to us they said well right now every service has its own dedicated team and trucks and they do one service but we want to overlay other businesses on top of this what if we want to build that next service on that logistics infrastructure but today our data doesn't we don't have the data to allow us to do it so what we did is we worked with them to put RFID scanners into all their shipment centers 300 of them and then pull that back to a core platform and we can now compute in real time how much room there is left on every truck every day and were there opportunities for the other thing we did though is we really you know we prepared them for the unexpected or unanticipated use cases and now they've added additional business services to that logistics shipping the thing I'm going to point out in the next two slides is tell me where the data processing spring boot apps begin and end and where this more traditional business API is beginning at you can't tell on these architectures you know data processing to real-time applications are in this beautiful spectrum across the spring but--and kafka platform this one is a mind-blowing project we're working on right now and we have permission to talk just about enough of it how would you build a you know service to monitor a whole country like how would you do that in the old crud app my sequel world could you even do it I don't believe you could and this actually has far-reaching consequences you know beyond technology t at I won't touch but you know people came to us they said we want to monitor entire European country to keep it safe again the approach and the architecture here without going too deep into it is that you know between data ingestion stream classification and then ultimately building event centric micro services to take action at the end of the day what this application does is it watches everything going on in a country it ingests that data and it helps analysts curate and do connections between everything that's happening so it says oh if someone was just arrested here and there was previously you know an issue here and then we saw this on a video here oh hey take a look I think that's incredible but you couldn't do this in the old project way either just thinking it from your organizational structure if you tried to plan this out three years in advance you wouldn't be able to do it so they're constantly changing the models and the applications to generate these alerts and that's where continuous delivery in this new architecture are really shining you could never built this application before it's pretty exciting finally the power of working with legacy and enterprises is paramount things don't really matter an enterprise to some extent until you can show them how it helps them on their legacy because there's such an organizational investment in that they've got so much going on with it I'm really excited to see you know this is Express Scripts who's spoken at Kafka some before as one example but many others using this pattern where you use change data capture to allow that legacy to enrich the event microservices stream and so we've worked with organizations as part of app transformation to do these CDC like use cases where we say we're going to enable these microservices teams to use to access legacy data without going through the legacy interfaces and I think this is a huge promising area for the streaming platform ecosystem to really get inevitability because when you can do these new greenfield applications when you can build applications that could never been built before and you're applicable to legacy and you have these scalable operational illah T's that's really profound so what does that mean for pivotal what's our approach well we're all in we've been doubling down on our investments in spring Kafka there's some great demos that you'll see here today and you know the idea is that every spring boot developer and there's four or five million of those in the world 60 million downloads a month of spring boot we want to make it as simple as possible to add the streaming paradigm to their tool belt so if you want to add you know case streams right within spring streams you can do that and you don't have to know too much about configuring and managing on the interfaces to kafka you can just be a spring boot and developer I do that I think this is really powerful building around that of course we've got a caching product called PCC as part of that CDC style use case cache and validation is a big one we're gonna plum that right into our cache system we've got a great new thing called pivotal function service which brings this event centric approach to infrastructure management sure that infrastructure exists and not exists as part of you know event density that's something that we're doing tell you more about that we have service brokers that allow you to bind from your applications into into Kafka to do real-time provisioning we have pivotal kubernetes service and very excited about the engineering investments we're making here to make you know confluence platform run on pivotal kubernetes service I think it's gonna be one of the number one workloads that we have super excited about the idea that you have programmable infrastructure for these two you know distributed data finally we're doing a lot of app transformation if there's one thing that I think come talk to us about is how do you transform your legacy how do you think about decoupling in that legacy and we're using Kafka patterns more and more I was reading through last night the last thing I read was sixty pages of notes about everything we've done with Kafka and apt transformation so come talk to us about that so that's what I have today I think it's a really exciting ecosystem to take notes in to learn about this paradigm shift we're all in on it and I think it's the next evolution of what I started that five years ago when I came to New York for the first time to talk about micro services and so really excited about the future and I hope you have a great summit go see Josh long speak Josh raise her hand he is the energy of the entire spring ecosystem in one human and I can't wait for his talk thank you very much