Add Support Agreement Byline with airSlate SignNow

Showcare byline template

hi everyone thank you for joining us for today's webinar roadmap to testing a design system with Murray Drake principle test automation engineer News UK before Murray kicks off we'll give a short intro in her role as principal test automation engineer news UK Murray is responsible for setting up the overall QA strategy and ensuring that the company delivers our high quality products to its end-users previously Murray was a test automation consultant working with different customers from various industries helping them speed up testing cycles and improving go to markets timelines Murray's also a sought-after speaker talking at software conferences and meetups such as software testing trends in 2019 digital women in tech in charge of Cyprus and applitools own future of testing 2020 virtual conference which took place last month in addition Murray is a Cyprus i/o ambassador and she also writes about techno Reid Raycom where you can find posts she wrote about testing and tutorials she shared related to test automation so without any further ado Murray the stage is yours hi everyone and thanks addy for the nice introduction so as a dimension I'm Murray and I currently work as a principal test automation engineer at new UK if you want to ask me any follow-up questions after this talk or you simply just want to connect I can be found on Twitter at M Cruz Drake and I've also added my LinkedIn ID here which is Murray - Drake so feel free to add me there after this webinar so today I will be sharing our experience about our roadmap testing at the sign system here at miss UK so I'm just going to talk a little bit more of information about myself so I'm originally from the Philippines but I currently live in the UK I have about seven years experience as a tester and apart from my a work role at NU so Eddy mentioned that I'm also a Cyprus ambassador where I talk about the different features at Cyprus offer to the tech community I'm also a tech blogger so you can find my tutorials or any or or any thoughts on testing and test automation at Mary Drake comm recently I'm also one of the organizer for the Cypress UK community meetup group and of course I'm also a mom and a woman in tech I try my best to inspire other women out there to pursue a career in tech and lately I've been sharing my story to different women in stem groups to raise awareness and support one of my most interesting fact that I like to tell people is I actually saw the great Stephen Hawking on the girl ship once which was quite surreal so I'm currently working at news as as I mentioned and first to anyone who does it know who news UK is so nice UK is a newest publishing company and part of the global news core brand they are responsible for publishing newspaper brands such as the Sun The Times and The Sunday Times and most recently they've also acquired wireless radio stations so first off I'd like to go over the agenda and give an introduction of what I'll be talking about today so first and foremost I'll be giving a quick overview about the tech reset which happened at miss UK I'll then give a talk about product platforms which is the team that I am part of and why we created the new scape design system I'll then proceed in talking about what our testing requirements were to successfully support the delivery of new skete design system and then I'll talk about a specific testing issue that we always face here at News UK I'll then give you an overview of what our testing strategy is as well as looking at our choice of testing tools and our testing workflow and lastly I'll share the overall applitools experience and talk about how it has met our testing requirements and the general feedback from our team so let's start from the beginning that's a Cree set if we look at News Corp's Global Strategy it's actually to break down the silos approach of our companies so that we can remove any unnecessary duplication amongst the different teams the basic idea is that services pools processes should be shared and reuse when it's time for us to respond to new brand ideas this creates a strain on the team's existing workloads because you essentially have to start from scratch on each new idea so for example if we have a new offering so such as Virgin Radio different teams will have to work from scratch to get an application up and running there is also a well known incident within the new stack called the fabulous pink incident so basically fabulous which is a well known brand Under the Sun wanted to change their brand color to a different shade of pink the different technology teams estimated that this change will take around six months because there is no single source of truth to make this change and we're only talking about changing the brand color here so just imagine the frustration from the business here now because of this tech reset a new team called product platforms was born so product platforms objective is to provide a library of shared reusable components that will serve as building blocks from which title teams and brands across news UK and also News Corp can reuse within product platforms were actually broken down into smaller teams with each team responsible for a different product in this presentation I'll focus only on the new scape design system team so we essentially want to remove any duplication such as the number of share bars that we have across the business as you can see here each brand has their own version of their share bar but essentially if you take away the teaming and all the other styling they look pretty similar the important question as well as to we really need to have 19 share bars now having 19 share bars mean that there is 19 code and assign repetitions which means that money and resource is wasted because we're just reinventing the wheel every single time there is also very little value in repeating work across the organization and what happens if there is a new brand at miss UK we definitely want to break the cycle of having our 20th share bar and this is why one of the core offering of product that forms is the new skate design system so new skate design system aims to address all the inconsistencies that have piled up over the years of having different solutions every development team has their own preference on how they would implement a shared bar for example however we saw that this leads to code and design duplication and oftentimes areas such as accessibility are often missed because of not adhering to a certain standard now the sign system it aims to create components that can be easily reused by different themes and it also enforces them to adhere to a set of agreed standards if we try to to link this to BDD or behavior driven development we are essentially trying to speak the same design language across multiple themes by speaking the same language we'll be able to deliver things faster and much more efficiently unlike the fabulous pink incident now having a design system would be cost efficient because we don't have to duplicate to work across different titles this is also related to my second point which is reusability new brands can also be delivered faster because they can use the foundations and the components straightaway it's also scalable and I like the analogy that our had a product platform scape on a recent meeting he talked about the sign system as similar to building a Lego you can scale it down to create small pieces or you can scale it up to build much more complex features it also promotes a standard way of working since there is a single source of truth and lastly it also promotes consistency across different title teams so just to showcase to everyone the different components that our title teams can use we're using storybook as you can see from the sample image here now all possible variants are listed as scenarios per component which shows the different use cases the idea here is that different title teams can use this different component instead and build it up to their liking so in here you can see that you can select different flag variations you can select different variations if you want it if you want an article stand forest or if you want an audio player you can also choose from different options on top of that we also have a new skate documentation site which already title teams can use so in here they can basically select what component they would like to use they can use the playground feature that we have and then at the end it will show a code snippet that they can import in their code base so in this particular example I'm just doing some customization on how I would like a button component to look and then once I'm finished I can just copy the code that's been provided to me cool so now let's talk about the exciting part the testing strategy since this is the first time that news UK have implemented a design system we need to make sure that the testing strategy is also well thought of because our product will be used by multiple title teams we listed down what our testing requirements would be and from these requirements we narrow down the types of testing activities that we would need as well as the different ice automation tools that can support our testing so in terms of the first requirement we identified that we need to we need to be able to test different components easily since new skate design system will mature and grow over time we will be delivering a lot of components so we need to make sure that for any code changes we can verify the different components easily since different title teams will use the different components cross-browser visual validation is a must for us so what we did was we looked at data from Google Analytics to see which browsers and mobile devices our users are coming from and we use this data as a starting point to decide which browsers and devices we must cover in our testing the third requirement that we widened divide is we need to be able to write visual tests with less maintenance in mind so it doesn't impact or delay our development and testing workflow so when we have visual changes to our component rather than maintaining the baseline images manually which can be quite lengthy especially if one change affected a lot of components so we want to be able to speed this process up relating to speed as well so since this is a new product we want to make sure that our build pipeline is high performing in order for us to provide a fast feedback loop to our designers and Uber's and therefore we set a time limit that it should be finished within 10 minutes since collaboration is very important in a design system we also want to integrate the sign review earlier on the process it order for us to catch any issues or any misunderstandings earlier on we basically want our designers to be involved and make sure that the visual changes matches what was in their design specification we've also identified that we want our components to be built with accessibility in mind on both the component and site level so all users can use our product and because some of our components are going to be complex and it will contain more functionality we need to be able to catch these functional issues earlier on and lastly we need to make sure that all relevant tests are written before deploying a feature so there's only popular engineers within the product platforms theme which includes myself already and so we want to be able to share the responsibility of writing the automated tests with the developers now from the testing requirements we've identified as a team what types of testing we need to do as a mass so this includes you know unit and and snapshot testing to test all the individual methods and functions we also identify that we need to do component testing test individual components which we broke down into three different activities so we've got functional testing visual and accessibility for our new skate documentation site so front-end testing activities must also include the same types of testing as component so functional visual and accessibility and lastly as I mentioned in the previous slide cross browser testing is really important because we need to be able to render these different components without any issues on different browsers and devices so while writing these different testing requirements there is one problem though so visual testing has been a really important testing strategy here at Miss UK other teams at News they're already using visual testing tools but these visual tests always end up flaky before working with project platforms I was already introducing I was already introducing visual tests on the previous team that I was that I was part of so let's just quickly talk about the limitation with functional testing so the limitation is that it's difficult or even impossible to catch all the visual changes that your feature might have in this example code snippet here we're basically checking if an element is visible or not however what if there is style changes you can definitely write more code to make sure that you cover these but then your code will be very lengthy and complex it's guaranteed as well that when you have a new tester or a developer on-boarded the moment that they saw that one test has a lot of code they won't even try to maintain that test so with visual testing we are validating what the user sees on different Buford's browsers or mobile devices however it's very time consuming when you have to do it manually so just imagine so there is an image here and someone told you that you know you have to test this manually every single time there are 30 differences and you can probably find them after quite some time but then this provides a slower feedback loop the most obvious solution would be of course to automate this now automated visual testing is not new with automated visual testing you have a baseline image which is your source of truth now every time there is a code change and then the visual tests are run a new test image will be taken the visual testing tool will then compare this test image with the baseline image and then it will check the differences at the end when it's finished it will report to us whether our test fast or failed the problem though and I'm sure that I'm not the only person who is experiencing this is with pixel excel comparison it's very sensitive to small changes even if you have a one pixel difference your test will fail even though in the human eye the two images look exactly similar if you look at this sample that's image here so if you just ignore the mismatch image in the middle and if you compare the two images on left and on the right some of the changes that were reported like it looks okay to me visually this this looks fine however because these two images were taken on different machine setups the tool has reported a lot of pixel differences you can run your tests on container such as docker to make sure that both the baseline and test image are generated using the same configuration but again from personal experience I still get a small subset of pixel differences and what if you have dynamic data in this particular example we have a change of data here but the overall layout on the rails is is it's it's it's basically similar there is no change on the layout so what you can do is you can probably set the mismatch threshold to be slightly higher so your test will fail only if it reached a threshold that you define the problem with this though is if you set the threshold slightly higher then you might miss actual - actual visual bugs because it will be ignored by the tool another problem so most of the existing open source tools for visual testing only runs on one or two browsers as an example one of the tools that we were using before which is called backstop j/s is a very popular visual testing framework however it only runs visual tests on Chrome headless Lee eye-spy which is a tool that was actually created by one of the teams here at Miss UK is another visual testing pool I Spy hooks into your selenium grid to run your visual tests on different browsers but still it's a bit limited so if you're using the official selenium docker images they only have images for chrome and firefox but what if you want to run your test on Safari now all these different issues will eventually lead to a flaky testing pipeline which then leads to visual bugs being slipped to production over time you lose confidence of your testing pipeline especially if these tests are always failing due to this pixel differences when your team loses confidence to your automated tests it defeats the purpose of having them in the first place and this is definitely something that we don't want to happen in our design system so just to quickly show you just some of the production visual bugs that we have deployed and I'm embarrassed to say this but you know I'm just gonna show you some of the bugs that we have deployed because of our flaky visual testing pipeline so if you look at some of the images here the first one was actually quite bad because the arrow icon was so enlarged on the tablet on on the tablet viewport so just imagine the shock of our tablet readers when they saw this bug the last image that you can see here it was actually a very recent bug that we slept the production and the issue was the real name was it was slightly indented to the right so now that we have established that visual testing is important how can we solve these different visual testing issues we know that we need to look for a different tool that would be able to solve this issues and at the same time we've also identified a set of testing requirements that we have to achieve in product platforms so one of the first tasks that I was given was to do a POC on visual testing tools during this time myself and my colleagues in narrowed down our choices between screener applitools and Apple we both engage with these different companies to understand if they're offering can help us and after some time looking at the different pros and cons of each of these tools doing example projects and they're moving it to the rest of the team we ultimately decided that applitools has met our requirements the most the next step for me was to create a business case and justify why it would benefit us as a peoples is a commercial tool I had to make a very convincing case as to why we needed and why open-source pixel-by-pixel visual tools are of limited use we also looked at different UI testing tools such as puppeteer selenium and Cypress after weighing all the pros and cons again for each of the stool ultimately we ended up deciding to use Cypress because of its great developer experience as well as this visual test runner personally I've had great experience so far with Cypress and we definitely wanted a pool that our developers will find easier to use because they will be helping us in writing the relevant tests so let's now look at the complete picture of what different testing tools the team have chosen to meet all our testing requirements cool so here are the other tools that we use to test our design system so apart from applitools and cypress we also use jess for accessibility we use ax specifically a plugin called Cypress ax and we integrated with our Cypress test so we can automate some of our accessibility requirements this means that on every component that the team builds we can test for accessibility much earlier on the process we also use Google lighthouse and wave which both gives a nice report on how accessible our documentation site is for screen reader testing we also use of our East voiceover tool what's great is one of our developers is an accessibility advocate and he shares knowledge within the team as well as across news on how to use the screen reader for accessibility testing so just to give you an an overview so on the next following slides we'll look at how we managed to integrate applitools in our project how we wrote that says as well as how we're doing the test maintenance so the first thing that we needed to do was to configure it to test our components and the good thing was because we did the POC already it didn't took us a long time to integrate it with our actual project now applitools has an integration with storybook so the setup was fairly straightforward we had to install ice storybook and export our applitools API key locally first and then create this config file with just 15 lines of code and then from here we were able to configure our storybook components to run on different browsers including IE 11 and safari in terms of the actual tests so applicants will automatically grab the components from storybook so all we needed to do was to write a couple of tests como and then run it as part of our circle CIE pipeline integration we also needed to configure applitools with our Cypress tests and the setup which was similar with storybook didn't take us too long so first we had to install the ice Cypress package and configure it to integrate with our Cypress tests which was fairly quick because there is a command that you can use to automatically do the setup called npx ice setup to run the full snapshots of the different pages that we want to test on different browsers we created a separate config file as well as the different test commands that we can then use when we run it in our CI pipeline now in terms of writing the actual visual test it also didn't took us long so as you can see here there is just over there is just 31 lines of code and with just 31 lines of code we managed to visually test nine different pages from our documentation site on different browsers and mobile device now let's talk about test maintenance so prior to using applitools whenever there is a mismatch on any of our images we would have to maintain the baseline images manually and this and and this maintenance actually took us some time so now that we're using up the tools we found that the test maintenance is very quick so you can see on the video here that we're using the grouping feature in applitools so you can group the similar differences and then from here you can easily approve or reject the changes in our visual test since the different snapshots are group once you accept one of them it will also apply the changes to the other images which made that as maintenance free really simpler for us how did we solve dynamic changes so if you look at this image really closely here we've got an audio player component where the track is loading if we use pixel by pixel tools this will be reported as a failure but with applitools we can add a region here like the layout region for example so with the layout region it will ignore dynamic content changes and it will only validate the overall layout so now we can avoid the situation where our build pipeline is blocked because of because of dynamic data differences now let's move on to how we did accessibility testing so apart from missile test accessibility testing is something new that we introduced in project platforms we know that with the design system comes with good accessibility standards so we have to make sure that we test for accessibility much earlier on if we integrate automation tools it it will help us catch issues and give early feedback to the team so by using cypress and cypress ax we have maintained with accessibility standards that could have been easily missed if we didn't had this and in terms of installation all we needed to do was install the cypress ax package and then import it to our cypress support slash index dot JSP here's just a a quick code snippet on how we write the test so basically for every component we have some default accessibility rules that we've defined and then on the next slide I'll show you how we actually run it all so I'm just gonna quickly play this video recording here to show you how we've automated some of the accessibility testing requirements so as you can see here each of our component is validated against a set of accessibility rules and if there are any issues you can see here that the tests will fail now when you open up your dev tools console you can see much more information as to what the accessibility violation is and then there's also a link to a URL which points you to some guidelines on how to fix the issue so of course it's not a foolproof solution because only around 20% of accessibility issues can be caught by automated tests so it's very important that we use other tools as well for accessibility such as such as doing the screen reader testing so you might have the Aria labels define or you have the different alt text attributes added in but you still need to check manually if these values make sense the PM is always pushing for accessibility so we don't exclude people with disabilities and most recently one of my colleagues did did a flash talk at use to raise more awareness in this area so we have the tools that we need to help us with our testing the next thing that I want to talk about now is our testing workflow as a whole so I like to start by saying that we're changing our approach so that quality is everyone's responsibility we want to move away from a model where testers are seen as the keepers and often get the blame if something was missed and a bug was deployed to production now starting from you know having early meetings we make sure that quality is baked in and we make sure that we add automated tests you know that needs to be added while the ticket is being worked on it's a team responsibility at the end of the day so we've also integrated this in our code review process so what we did was we updated our poll request guidelines so that relevant tests and documentation are added by the respective developers also by having a testing checklist this helps them look at areas that they need the check so as I'm mentioned on one of the earlier slides we have a requirement that our build pipelines should be finished within ten minutes so as you can see from the screenshot here we're on multiple jobs in parallel as much as we can to speed up the pipeline and provide that fast feedback loop we've also decided to have an approval step that you can see here on the component visual testing because as a developer when you create a pull request you normally have to push more than one commits and you don't necessarily want to run the visual tests on every commit however we've made it a mandatory check so that before you merge your pull request the component visual tests must be run and it must pass now when a pull request is merged to develop similarly we run most of the jobs in parallel at the moment we only run the documentation site visual tests when a feature is merged to develop due to our license restrictions however in the future we have plans to get this added on our PR workflow so we can also catch visual issues on our documentation site earlier on with circle CI s-- github integration we can easily see the results of our test so it gives an indicator to the whole team if a feature is so key to be merge we're also utilizing applitools github integration to manage any merge conflicts between baseline images for different branches this to save us a lot of time because if a pull request has introduced some new visual changes and you have to update the baseline images this will not impact the other branches baseline images so applitools will handle that conflict for us and lastly we've added slack integration so whenever our it has failed the team will be notified and will take action when needed who so because we're the first team you use applitools at Miss UK so far it's been a really good experience and you know it meets our testing needs in the last few slides I'll share what the overall team feedback has been so from development perspective applitools fulfills our testing requirements we're able to test our components easily on different browsers and it hugely simplified our visual testing process just to add some data to support this so we have around 760 component tests and our execution time is less than five minutes which contributed to our high-performing build pipeline this maintenance is also simpler which gives us more time to focus on other things and if we do face any issues at the post of our team is always happy to help and from the sign perspective so it enabled them to have a better design review process because they can focus on the fine visual details they also think that we have a great level of test coverage which is really great to hear and because we consider our design team integral in the review process whenever they review the visual changes on the applitools dashboard they can easily do it once they understand the basics of using it and finally they also agree that the AI functionality that applitools offer is a time-saver so I think with feedback is always good to hear what could be improved more because it could be a new feature that app titles might support in the future so the first feedback was with the app titles dashboard to make it more intuitive especially for new users this could be by a simple tool tips or you know adding more descriptive text the second feedback which i think is a really good one is to have the ability to set baseline images from the actual UI design so the sign poles like sketch which figma or even abstract could maybe have a plug-in with applitools so that the UI design is the source of truth and lastly as I mentioned on one of the previous slide we need to have more visual love on our documentation site because we only run them as part of our developer workflow cool so where are we now from accessibility view you can see here that looking at the new sketch site it doesn't have any errors that live in lighthouse and attack which means that we are on the right track from performance side it still needs improvement which is something that the team will focus on soon and as far as I'm aware we've also not received reports of any visual issues from our client so I take that as a good sign that our testing strategy that we have put in place works in terms of product highlights product platforms has supported the launch of times radio which actually went live last last month and this is the first product which uses our audio player component cool so having a solid testing strategy and defining what our testing requirements from the start has helped us support the delivery of new skate design system and if there's one takeaway from this talk it's to use the right tools to support your testing and that brings me to the end of my presentation I hope this was useful and I'm ready to take some questions now first of all thank you that was amazing Marie thank you so much so we do have questions but before before I dive into the question I just want to mention two things so first of all I see a lot of questions about is this being recorded and will you get the recording so I just want to I want everyone to be rest assured that we are recording the session and we'll e-mail you the recording plus Murray's slides by end of day Monday so so don't worry about that plus all your thank you notes to Murray I won't read them out loud but I will send outlook I will send all of them to her once the session is over so also know that you will get all of those too so thank you so first question what steps did you take to create the testing strategy that you currently have that you currently have for your design system yep so the one of my first thoughts when I moved this new team was to make sure that we you know we we create a cross-functional team so that we can you know avoid the stress being the bottleneck so the first thing that I really focus on research for tools that would really help with the developer experience so I focus on looking at the different tools like what so in the market I also you know obviously look at what open-source tools can be useful because it's always good to use open source on those first and then after we have selected you know the different tools we did a lot of pocs just to make sure that it has you know met our requirements and the team are all aware of you know all the different new tools that were going to introduce and in terms of like what the different testing activities it's it's it's really good to get everyone involved and one of our core core requirement was to make sure that we embed accessibility in in mind and that's why we you know decided to integrate accessibility testing much earlier on if you have you know data to back if you can see in terms of your your traffic like which browser your users are using it's also really good to utilize all this different tools so you have a an idea as to which you know browsers to focus on because a lot of pulls out there you know they support you know a lot of the different versions of different browsers but with this automation like you have to strike a balance you have to you know check Twitter for example the topmost browsers that you have the support so in our case you know it was Chrome you know like Firefox and on the mobile devices we have to make sure that we have support in those different viewports as well so back up your your research with data and then get you know get get everyone involved do a lot of you know demo so at least the responsibility is not just within your your shoulder and you'll be surprised as well that you know some of members of your team especially developers they have really good inputs so always take them into consideration when you start you know creating a testing strategy I hope that answers the question um next question why did you decide to use Cypress jazz rather than the selenium webdriver JavaScript library yep so I've I've used both boots and you know both tools they have their own strengths they have their limitations as well but one thing that I've noticed is just from personal experience I find that developers they're they they have more positive experience in terms of using Cypress because the set up is much easier they can debug the tests you know much much quicker however this doesn't you know say that you know selenium is is bad in terms of the setup it just takes more you know more time so because there's only two QA engineers in our team we really have to convince everyone that they should write all the as well and I find that Cyprus really takes you know all the requirements that I was looking for in terms of getting the developers on boarded and so far it's it's working great we know that there are limitations you know with Cyprus but however because we test most of the functional that's on the component level so far it you know it just managed to automate what what our needs are how long did it take you to set up Cyprus with storybook so in terms of setup so it didn't actually took us a lot of time so when we did the initial sort of the initial pilot for the design system project we had to or we had to look at you know alternative ways on how we can actually write Cyprus tests on the component levels so we definitely you know get everyone likes opinion on how to get it integrated and I think it's fair to say that in terms of actual timeline I wouldn't say it took us that that very long because once the team were able to figure out the actual integration writing that says you know didn't took us long the documentation site or in Cyprus was was quite helpful so that gave us a lot of like benefits and because I was already using Cyprus before joining the team I was able to share some of my knowledge as well yep so it's it's not the very straight like a straightforward answer because I couldn't remember the the actual side but I just remembered that they did it the best very long so the next question is interesting we got a lot of different people asking the same question so I guess a lot of people are interested in that so before you integrated a plea tools or before News News UK decided to integrate a plea tools how did you how did you carry out visual testing did you use another tool did you join manually what was your process before that before yeah so because the actual onboarding you know that we experience with applitools it took a longer time than anticipated because you know contracts had to be approved from both side and you know we were already developing components earlier on so we had to use a different login pool called Cypress image snapshot but so the good thing was it also integrated with our Cypress tests so in terms of setup it wasn't difficult however we did face some challenges along the way so that's why I mentioned you know um docker in in one of the previous slide because our team had to you know make sure that we're on the visual baseline images using docker container and locally it was just taking a lot of time however we just you know proceeded with okay it's better to have some visual testing in place rather than none at all so even though we face some limitations we just still you know use that just so that we still have coverage on the visual side and then if you look at our testing checklist so I've added it on one of the previous slide I've created some guidelines that because prior to doing the app it was integration we didn't had any sort of disability on ie 11 so I just had to write a note there that okay if you're creating a new component or making changes on the dock side make sure to test on ie 11 and um and at the same time I'm also doing a lot of you know some exploratory testing on different browsers just so I can report this issues much earlier on so it it did like that word it did slow us down that's it at the beginning and like sometimes we would have to rerun you know the visual testing pipeline cuz you know we'll have this like small like pixelated differences but we just accepted the fact that this is just for a short term long longer term will will will switch to applitools and all this different issues that you know buffets will be will be eliminated and and it did so that's that's the good thing now so we have a lot of cypress questions I heard Cypress is difficult to run on Jenkins see ICD is that what you use or do you use any other tool or do you have any recommendations so I was using Jenkins before but I haven't tried integrating the Cypress test because now we're using sirico CI pipeline but I believe if you're using the Cypress docker images you should just be able to use it in your Jenkins pipeline I did had the previous Lunch and Learn session to show how you can integrate this your your Cypress test OASDI pipeline but it's with using circle CI so I can't really answer it much from Jenkins perspective cool I hope that answered the question yeah so next question can a plea to integrate with other testing frameworks aside from Cyprus yes so the good thing with applitools is they basically integrate with almost all of the testing frameworks out there so integration with selenium webdriver IO if you're using even mobile testing frameworks like app you espresso xui tests they they have they have some SDKs for those so if you go to their their documentation site you should be able to see all the different plugins that they have do you use manual tests as well in addition to the automation in order to make sure you check the functionalities from the user perspective or do you completely rely on your automated testing yeah so we don't rely 100% on the test automation side we use it mainly to support or it has taken activities so I'm I'm so before when we were finding you know a lot of issues though that our automation who couldn't find exploratory testing really helped me in that sense and in terms of accessibility testing we still have to do a lot of it manually because only 20% of the automated accessibility violations can be detected so you can't just rely on the test automation you know side however what work is it gave me the confidence just to you know you know not be like hands on every single day to support me skate design system because we also have other teams within product we didn't project platforms that I have to do support so just by having you know the test automation like strategy in place I have more confidence that okay we won't be releasing you know any visual bugs to our clients and at the moment you know as I mentioned we haven't like receive any reports however from time to time we do get you know like the small odd bug that oh that you know that we we would have got if if we were doing some manual testing so basically we use both both approaches we don't just rely on on one over the other is it possible to run both mobile and web tests using Cypress using Cypress um so Cypress you can set the different viewports if that's what you're asking because Cypress is predominantly a tool for testing web applications but if you just want to verify that you can set different viewports you can do that with Cypress as well you just need to call sy that that I think it's I that viewport if I'm not mistaken and you just have to pass in you know the viewport size or the actual mobile emulator string and then depending on you know what's your test spiral decision approaches you can either use the side press dashboard or if you're using applitools you can just you know I'm going to figure it in your aptitudes that config dot JS file and specify all the different browsers or mobile devices that you want the earth has to be run against we are out of time so I'll take just one last question so you mentioned to tool three tools that you POC to in addition to a play tools why did you choose Apple II tools from the other two tools that you tested yeah um so when we were doing the POC with screener so during that time they don't have support yet for Cypress I do believe now that they have integration for it but basically before when we were doing the integration we didn't want to maintain a separate like selenium test framework in order to run our aptitude test and with hepl it gave us you know confidence that you know we can also support different browsers however we found that I think it's still using pixel by pixel different so we don't really want to use another tool that muses pixel by pixel because of the different issues that I mentioned at the very beginning so I found that with applitools we can run it on different browsers we can run it on different mobile emulators we can also use advanced AI algorithm such as the layout region for example the weather like the one that I showed where you can just ignore dynamic data differences so because of these different like perks and I forgot to mention that you can also raise you know issues directly from applitools and it will create a JIRA tickets so if you're using JIRA so because we want to be able to have you know a pool that not only supports our a testing but also supports you know our designers requirements that's why we opted for for applitools so unfortunately that's all the time that we have for today we will run a minute long I apologize about that before I let Murray say her goodbyes I want to thank everyone that joined us for this live session and of course thank Murray for this insightful session I saw that you have a lot of other questions that we didn't have time to to address your time constraints so you can obviously connect with Murray on social media and and continue the conversation there I just want to remind you again that the recording and the slides will be emailed to you by end of day Monday and I hope to see you in our next event thank you yep so just want to say thank you again to everyone and I hope you found this useful so as a dimension you can connect with me on my different social media links at Twitter I'm most active there now so you can add me at mq streak and also unlinked and you can add me there so thank you and have a good day and hopefully you have a great weekend bye bye