Enhance your product quality with leads management for product quality

How to create outlook signature

hello everyone really excited to be here among such great speakers my name is Kwame I'm going to talk to you about managing product quality today my goal is really to inspire you to think differently about maintaining your software and we're gonna talk about how you can apply systems-thinking principles to understand software behavior over time much better so first a little bit more about me previous experience I've worked at a bunch of companies as an engineer a tech lead manager right now I do independent leadership coaching and technical advising a lot of the work in my career has been understanding what processes we're employing today and how effective they are and over time I've picked up a bunch of techniques that I'm gonna share with you today today I'll be using a tool that actually wrote are added to in preparation for this talk yes I actually wrote software in preparation for this talk so it's a tool for predicting system behavior nodes or amounts in quantities in the diagram that you see edges represent positive and negative correlations longer edges mean a delayed effect and this is heavily based on a tool called loopy that's used for systems thinking you should look that up too so the first principle I want to cover understand the understanding systems fully allows you to find the greatest leverage points which leads to greater impact I'm gonna share a quick story with you several years ago I was responsible for optimizing homepage performance for Facebook and I was managing the team that had the performance goal while 10 plus other teams were actually committing code to the home page and they just had engagement goals it was a really interesting experience so when I first joined the team my mental model for home page performance looks something like this so this is a technical problem we have some tech debt wheel and a bunch of optimizations and we'll hit our performance goal right wrong so after a while if thinking really deeply about this problem chasing engineers lobbying managers in examining a pattern of performance regressions my little model exploded in complexity so I began to see this problem is more a collection of feedback loops that span the technology stack and our culture and in order to improve performance we had to apply pressure to various parts of the system so it wasn't just optimizations it was things like tooling documentation t-shirts and Enter sumit all of these things believe it or not actually contributed to improving page low time in retrospect what I was actually doing was employing systems thinking and reasoning about these various feedback loops helped me understand all of the factors that contributed to oscillations and page load times companies are really machines right with all of these interconnected parts and if your company produces software the purpose of that machine is really to deliver value to users when we interact with these complex machines depending on how we approach the problem you know the solutions we try to land could have unintended side effects and that's what I'm going to talk about today most of the time we employ what's called event oriented thinking to reason about the world but event oriented thinking ignores the inherent interconnected nature of most companies employee event oriented thinking it's really easy to have a short-term effect on the system right but in the long term you're ignoring the rest of the system so whatever change you land is going to have some unintended consequences most likely at least if you're thinking about engineering problems so if you're an engineer or a manager you're encountering these causal feedback loops all the time and the more senior you are in your career whether you're a senior engineer or a tech leader or manager you should really be spending more time thinking about the system in less time thinking about local events so my goal here isn't really to prescribe specific strategies that you can use my goal is to share a mindset that I found really helpful in my career for teasing apart problems in different ways principal to working backwards from events to mental models helps you understand all of your options for manipulating a systems behavior so we're going to talk about some straightforward techniques that you can use for sort of changing your frame of reference and using different approaches to understanding problems so there are four different types of thinking I'm going to talk about and they all build on each other so I mentioned event oriented thinking and next we have trend oriented thinking which is just a pattern of events and we have system oriented thinking and the system is what gives rise to those trends and then finally we have mental model oriented thinking and the mental model is what gives to the system so as you tease the problem apart in different ways as we go down this pyramid you're having a greater level of impact overall and let's talk about how that works so event oriented thinking you ask yourself essentially what happened and how do I respond very simple when you're operating at this level again you're having that short-term impact but you're ignoring the rest of the system and sometimes that's okay if the sites down you should just fix it right trend oriented thinking you ask yourself what is the pattern of events what's likely to happen in the future and how do we adapt when you're operating at this level you're probably landing some incremental change the system but we can do a lot better here system oriented thinking how is the structure of the system guaranteeing the behavior I'm observing and then how do we change the system to guarantee the behavior we desire so at this level you're playing around the system it could be technical it could be social it could be organizational could be something else but the point is that you're trying to produce that sustained change in behavior as opposed to making that local change that may or may not work out the way that you want so lots of you probably very familiar with this but there's one more level I want to talk about here and that's mental model orient to thinking so what assumptions beliefs and values are giving rise to the system and that what mental models do we need to adopt or discard to make sure that the structures that we know are healthy arise organically when you're operating at this level you're basically hacking directly on company culture and this is really the highest leverage activity you can do because when you're hacking on culture then you're basically making sure that the right structures appear everywhere and you're not having to go around and you know examine the trends or you know tell people how to do the work there's a shared understanding and that's why it's so powerful now to be clear I just want to reiterate I'm not saying you should always use mental model oriented thinking if the site's down you shouldn't say well what mental models do we need to adopt today show this light comes back up right so it's really important to use that right mode of thinking often times it's multiple modes principle three and this is the last principle I'm gonna walk through several examples here to apply the concepts we just talked about so principle three employing systems thinking effectively helps you manage quality long term so first example a user is consistently complained about poor app performance and you're responsible for investigating what do you do so let's try event oriented thinking what happened and how should I react so you might dive in figure out why it's slow you found it on bounded query that only affects really big customers you fix it after a week great you made that customer really happy you've unblocked them on a really urgent issue and that is some impact but in the long run that's not that impact for it so how can we do better here trend oriented thinking what's the pattern and how can we adapt you might spend a little bit more time trying to figure out where a similar failure is happening in your stack so you dive in you do that you land a new abstraction that solves this problem in several different places so now you've solved the original pain point for one customer and figured out how to address the pattern which helps other customers who are in a similar boat so you multiplied your impacts but ultimately you'll probably hear about these issues in the future right you'll you'll continue to encounter these issues and land fixes ultimately I think we can still do a little bit better so what about systems oriented thinking with this you think about the structure of the system that would produce that pattern of perforations that you're seeing so in this example you realize that the perf cost for incremental feature is way too high due to some outdated architectural decisions most customers aren't suffering yet but you realize what you're seeing is really just storm clouds on the horizon and this could be a major block to your growth so what you're seeing on the top left is a tight feedback loop between new cut new features customers and revenue right so that encourages more new features but then there's a longer feedback loop between page load time quality and revenue right so in the short term we're incentivized to land all these features and then we're kind of blindsided by all these quality issues and this is what happens over time this was a really successful product at first that was starting to get traction and then all of a sudden it went into this death spiral so this is actually an example of something called growth and under investment this is a common pattern in systems whether it's technical social and you encounter may have some form of this what's happening is that you have a feedback loop around a goal that you're trying to hit but then there's another feedback loop perhaps longer running perhaps less visible that's working against you and then when your performance metrics start to suffer you invest less time in trying to hit that goal which hurts the performance metric even more which leads to less invested time and that is the cause of that death spiral this is something that's happening at every single company right so I challenge you to go and look for this pattern and evaluate whether or not it's time to try to unblock growth so let's see how we can do better in this particular system the structural problem is that the company is not set up to remove those blockers to growth quickly enough so in this case the blocker is performance and overall quality of the product so the three changes we can make to this system first renewed emphasis on quality worker increased emphasis second a much tighter loop between quality work and load time and the third thing is making sure product engineers are thinking about quality - and this is actually a well known fix to the growth in underinvestment problem a tighter feedback loop between the basically a tighter feedback loop around that second loop that's countering your performance metric and this is what happens over time in that system I just showed you so that purple line that you're seeing is actually quality so when it gets bad enough basically you start working on it again goes up and down because that's the blocker to your growth and the green line is actually customers and revenue so now we've unblocked growth and it grows at a steady pace awesome so that was systems oriented thinking so what if we use mental model oriented thinking you think about what mindsets leads to the system you're experiencing one work performance work isn't being prioritized maybe other quality work is being neglected - and you realize that maybe the startup mentality you guys started with isn't working for you anymore maybe your culture is the real blocker to growth maybe you and your colleagues need to fundamentally rethink the way you're measuring your performance and investing your time so again when you're communicating at the level of mental models your M increases a lot right so we've gone from trying to optimize this architecture to trying to understand what about our culture is holding us back as a business not just in in tech but everywhere right this is this is an exercise that's happening every day at every company so what we're using systems oriented thinking we talked about the ideal structure which included sufficient investment a tighter feedback loop and awareness among product engineers so the next question you would ask yourself is what kind of culture produces those structures and I'll leave that as an exercise you guys only so much time example to your app has several serious customer facing incidents over the last year you're doing retrospective diligently but it doesn't seem to be making a difference for reliability so we talked a lot about finding those highest points of leverage so our ideal retro process should be designed to do just that so my favorite version of this is called five why's the goal here is to identify the root cause for whatever failure experiencing you start with the observed effect you ask why and you keep doing it until you feel like you've reached the root cause once you find the root root cause you can talk about action items and next steps let's look at a quick example our site is suffering a two hour outage why we ship the bad config why a new hire didn't understand the config format why we don't have Docs why every team uses a slightly different format why we believe it's important to give teams as much autonomy as possible and their technology decisions interesting that seems like a really great thing to mean right so the thing that I really love about five wise is it's a great way for employing systems thinking as a group that exercise we just went through we were actually just exploring all the parts of the system that are proximal to the failure in other words what parts of the system directly are directly related to that failure and indirectly related and once we do that we actually understand all the options all the failure modes much better if you do this sort of thing diligently you'll understand not just what parts of the technical stack of contributing to failure but also what parts of your culture are contributing to failure and if culture is one of those factors that are limiting your growth it's paramount to understand when to change in case you're curious how that performs over time confusion and mistakes where I steadily but so does autonomy because goodness is staying constant at least is in the system may might sound familiar to some of you so I want to talk about what happens when you don't retro carefully and there are two systems patterns I'm going to cover the first is the fixes that fail pattern so in this pattern you encounter some problem symptom and you land a really quick fix but then the long-term consequences basically undo that work that you did so in this case we hear about some performance concerns we lazy load short term that helps long term it hurts app reactivity this is a very common thing in this particular in this particular system this is this is what we see a lot of effort but performance complaints don't end up going down that's the fixes that fail pattern and that's what happens when you ignore parts of the system sounds familiar so the second pattern my highlight is called shifting the burden aka addiction I'm going to shift away from tech and use a different example so let's use caffeine as an example so you're feeling fatigue you pick up coffee coffee short term you get that energy burst you get through your day awesome but you haven't actually addressed the root cause the problem it's a lack of sleep poor diet not enough exercise something else caffeine isn't actually a solution so you can take this replace caffeine with the useless tree factors or the other measures that feel good short-term but aren't actually solving the problem and if you just spent your time on the long-term solution it takes a little bit longer to see that payoff but ultimately just like putting down that cup of coffee if you sleep instead you'll feel a lot better and this is what how this system behaves over time coffee consumption goes way up dependency goes way up fatigue goes down but so do healthy lifestyle habits that could replace coffee and this is something that's happening to many of us today right okay so I'll close on this now once you all ask yourselves if you do you feel like you're retros or serving you well are they producing lots of incremental low-impact fixes what fixes do you feel like I've been failing you and finally what tactics do you feel like your org might be addicted to so we just covered understanding your systems for greater impacts we talked about employing different modes of thinking depending on the problem and finally we talked about the long term system dynamics of product quality that's my time oh well one more thing next time you encounter encounter a lingering product quality problem instead of asking how do I solve this problem try asking yourself what thoughts beliefs and values do we need to adopt to ensure the behavior that we desire Thanks