yeah cool whoa yeah hi everyone my name is Hamed I’m a product manager at Google app which I’m gonna be telling you more about it and joining me is Michael Bailey who is a senior staff engineer at MX and I can express he’s kind of a celebrity in the Android community really like very well known very well respected and his opinions are great follow him on Twitter if you’re not and he’s gonna be telling us about how American Express used Google cloud test lab to which use their testing infrastructure overhead and improve their testing practices I hear an echo is that Senate anyone else hear the echo yeah okay okay can everybody hear us anybody not be able to hear us good all right I think we should stay a little bit far from Yosemite yes stand back a little bit all right so why are we building this why are we why do we care about a product like Google cloud test lab what we want to make sure off is we want to make sure that developers build better apps we want them to improve their app quality and building better apps helps you increase your user base and increase your revenues eventually so we want to make sure that you build better apps but the only problem is it’s very difficult to ensure a higher quality for your apps on Android because of the fragmentation in the ecosystem so many different devices many different API levels different locations locales network conditions things that might go wrong very easily and it’s very hard for a developer to account for them and as we see here 84% of the users will not open the app again if they encounter two early-stage crashes 84% and this means that usage might decline very rapidly over time and you end up losing a lot of users a lot or a lot of money we also looked at some of the classification of the one-star reviews on Google Play and we realize that 56% of these 1 star reviews are due to quality issues 56% is that people actually tend to punish lower quality apps more than they tend to actually write good fields 56% of these are quality issues but actually if you look at the bright side from this chart this means that actually all these things are that developer put in control there are things that you could have could have fixed before releasing your app to production and could have avoided all these bad quality reviews so why aren’t we doing this as developers why are developers not doing this and making sure that their apps are compliant to a certain level of quality before releasing them well it turns out it’s actually a very difficult thing the testing practices that we do are not optimal we tend to do a lot of manual testing or even if we do automatic automatic testing we end up spending a lot of time money resources on things like the image on the lower left corner over there we have to build this sophisticated infrastructure to support all our testing we want to make sure that we test on so many different variations of devices form factors and we end up with a lot of investment in engineering time and resources we also look from our research at the time that developers spend on average building versus testing their code and realized that 75% of the time on the developer spend building an app is actually spent on testing and debugging 75% this is time that could have been easily saved and could have contributed back to actually shipping more features into your app so this is why I wanted to build Google cloud test lab so we we talked to hundreds of developers we interviewed a lot of different users from small companies to large companies from large dev shops small or even one person building an app on the site and we wanted to come up with the product that helps developers achieve these goals of building better quality apps so we build a Google cloud test lab with three main guiding principles which I’m going to talk to you about them the first guiding principle is making sure that we have a scalable infrastructure that takes that the cost of infrastructure completely away from developers and let them focus on just building and writing their apps while Google take takes care of the rest so Google takes care of building your infrastructure we build these racks of devices in our data centers we manage them for you we just let you focus on building code writing code and nothing else and we take care of all all of this for you so these are actually physical devices put in our data center and you can what a test my app on 5 samsung devices to Motorola phones and we run the test for you we also build the virtual devices which are a full version of Android running on Google compute engine so this is just an Android version running on a virtual machine and you can spin up hundreds or thousands of them and run your tests on them it provides you the most cost-efficient on-demand

scaling the second guiding principle that we built the cloud test lab with is the automation so we wanted to make sure that we understand the current testing practices from developers and try to meet them where they are that’s how we call it so we built the Robo we understand that most of developers actually don’t have automated test scripts and they do a lot of manual testing and we wanted to make this easier for them so we built the Robo tester which is an app crawler it launches your app on a number of devices and it just basically tries to interact with your app as if a normal user would interact clicks buttons swipe and when it finds a crash it gives you a reproducible set of steps to follow to in order to reproduce this crash and for people who actually like write instrumentation tests which is something I highly encourage everyone to do for these people we allow you to run all your tests on Google cloud test lab and on cloud test lab you can run and matrix of combinations of tests a matrix is a combination of dimensions the dimensions can be a device in the API level and orientation a locale all of this together you combine them and then with one click you can launch your test on hundreds of different combinations and then when your tests actually run we provide you the results that allow you to investigate any failing tests and help you go back to a better state in your app we provide you with a logs screenshots the videos the stack traces and things that help you investigate a failing test very quickly and going back to a better state and then the third guiding principle that we built a cloud test lab with is how it we wanted to integrate with your workflow so we started with Google Play Store a place where every developer is going to in order to publish their app and we build the integration with the Play Store so every new app that gets submitted to the Play Store alpha or beta channel gets an instance of the Robo run on them just to give them an idea hey this is how your app would perform on these devices it might have crashed on this device this is something that you’d need to worry about fix it and then continue with the publishing process and then we also built an integration with Android Studios where every developer writing an Android app is using Android studio and from within Android studio you can use cloud test lab the same way you would use your local emulator or your phone so you can launch your test on your emulator or on your phone or on cloud test lab on plenty of devices we also built a web interface where you can just easily go upload your apk configure your device matrix and launch the test from there directly the last one is my personal favorite which is the command-line interface which allows you to add cloud test lab to your continuous integration environment so every new build that you have for your app has to follow a certain level of quality standards and this is actually what Michaels gonna tell us more about and how Amex benefited from this one to make sure that every new mx mobile app is held accountable accountable to a certain quality standard Michael thanks I man all right so CI as Ahmed said is just one use case for CTL you’ve got Robo you’ve got the Play Store integration but really today I’m gonna tell you about our workflow how we do software development and changes go through the workflow and how we used CTL to add it to that process so before you add any tool to your workflow you kind of want to think about it and this is how we think about things kind of philosophically what challenges in my facing right so you have a workflow you’re trying to get value out to customers what are the challenges that I’m having doing that right and then as any new tool you see you go to Google i/o you go to all these things what tools are gonna help me solve the challenges that I have some tools will help you solve challenges or do solve challenges but they may not be the challenges that you have so think about the challenges you have then look at the tools that actually solve those problems and then think about is there something simpler you always want to try to get the simplest thing that can solve the problem so that your engineers aren’t burdened with maintaining a complex solution new team members don’t have to maintain a complex solution so you want to make what you’re doing simple and so that’s another thing to consider when you’re evaluating what tools you’re going to add to your workflow so what was our challenge this is a challenge we have I imagine that pretty much every software shop has a similar challenge but we have a code change whether it be a bug fix or implementing a new feature or a new thing that our UI designers wanted us to do and we want to know is this code change a good code change is it worthy of shipping to our customers right and that kind of comes into play two fold does the cold code change accomplish the desired outcome does it fix the bug does it implement the feature and conversely does the code

change introduce any negative side-effects does introduce a new bug does it break some existing thing that was working does it slow down your app right so you want to know you want to have evidence that this is a good code change that’s worthy to ship to our develop our customers typical solution is you have a code change which comes in the form of a pull request or a change list whatever you might call it you build it and you test it right so most people have some sort of solution like this if you don’t you should and this solution usually looks something like this you have a source control system we happen to use git and then you have a CI system we happen to use Jenkins but the things I’m going to talk to you about today are sufficiently general that you could probably plug them into whatever system you happen to use and when you make a code change in a git system you’d make a pull request that pull request goes into your version control then your CI server sees hey there’s a new code change your CI server picks that up it builds that and inspects it and test it and then it tells your source control system this is a good change this is a bad change it provides feedback if it’s a bad change or if there’s a problem you make another code change and then again it goes back to your continuous integration system so this is how this feedback loop goes until ultimately you have evidence this is a good code change let’s merge it ship it to customers so I’m going to talk to you today about our workflow at American Express and how CTL is a part of that but there is a page out there on the CTL website that kind of gives you some instructions on how you might integrate it with different version control system or continuous integration systems there’s a link out to how you might do it with circle CI I think I’m it’ll be sure to keep this page updated when new things come out and you can integrate with new things absolutely yeah definitely if there’s also anything you want us to add like any continued continuous integration environment you’re passionate about let us know and we’ll be happy to work with them to add their documentation and publishing on our sites cool so check out that link alright so what we do is we call pre-tested pull request right so before the pull request gets merged we want to know that and they have collect evidence that it’s a good change so you can see here this is just the very beginning stages of a pull request in a system called Atlassian stash which is a git repo management system you can see there’s a merge button up there that merge button is grayed out and it has a little triangle next to it that means it’s not ready to merge it we don’t have all the evidence but it’s a good code change this is how it starts out right so once you open that pull request we our Jenkins server will create a job a series of jobs actually specific to that pull request in this case the the branch and get is called feature CTL Talk so it created this job specifically after that pull request is opened to run those changes on cloud test lab so how does Jenkins know when I open a pull request that it needs to create these jobs and do these things we use the thing called the job DSL plugin this is a Jenkins plugin that allows you to provide a snippet of groovy code and the output of that groovy code is a list of jobs that need to be created and basically that groovy code gets run periodically maybe every minute and it says here’s the job so when it sees that oh a job got added between one run in the next it goes and creates that job if the next run of this code produces less jobs and knows oh those jobs are no longer important that goes and deletes those jobs so the jobs on your Jenkins get created and get deleted automatically in this code snippet you could see it’s just looking what are all the branches I’ll just create a job for every branch in our workflow we actually look at what are all the open pull requests but pull requests is open we build those we don’t build every branch because generally when we what we want to build is what somebody is opening a pull request for so now you can see here we have one piece of feedback on this PR it says one build failed there on the side that means the CI server created a job something went wrong it reported it back to stash and said something went wrong so maybe you make another code change and then it will again build that thing start all the process over again report back here you can see okay maybe I made a code change it’s going and rebuilding everything it’s showing the clock icon which means things are in process things are building I’m gonna get some feedback hopefully everything goes well and now I have some green checkmarks means everything passed you can see here I have two builds one is for cloud test lab that’s our instrumentation tests the other is just

you know Gradle W check which runs your static analysis it runs your lint it runs your JVM desktop unit tests so you want to get both running and you could have even more builds so now if we look back in stash you can see I have two green builds I’m good to go but you notice the merge button it’s still grayed out and it still has a yellow triangle next to it so it still hasn’t collected all the evidence it needs to know that this is a good change you can see I have one reviewer there that reviewer needs to give me a thumbs-up right he needs to look at the code give me comments I need to make it good enough code so that one of my fellow engineers is willing to give me the thumbs up I also have one open task there so he’s looked at the code and maybe he said oh hey you need to change this or you need to do this before I’m willing to give the thumbs up so I also have to go and make sure I complete all the tasks check in the code before I the merge button becomes blue and I can merge so if you look here now I have a green checkmark from my reviewer it says all open tasks resolved I’ve gotten my green builds and the merge button is blue I can click the merge button and I’m good to go I can move on with my life go to my next code change right collect the evidence so when you’re going through this process right we have lots of developers doing lots of things and they’re distributed the remote so we want to be able to keep everybody in the loop on what needs to be done to get these code changes merged so what we’ve done is we’ve integrated our CI and our git repo management in with HipChat so that it kind of keeps the whole team who’s distributed up-to-date on what’s going on hey this build failed you can go and see the details by clicking open oh this build is now passing again which is great so everybody can take us a breath of relief when a pull request gets open you know like all your co-workers can know they need to jump on it and start reviewing it and giving you comments somebody approves it everybody could say hey this person’s doing great the reviewing pull requests this one got the thumbs up and then also it will check in general is this mirja but when did that button become blue and york’s ready to merge so you can go and click it and get on with things and it’ll tell you right in your jenkins this pull request is now mirja bold the builds are green you’ve got your thumbs up you resolved all your tasks go and merge it I’ll tell you that right in our gift chat room so now I’m going to go a little bit lower level and just the commands that you need to use in order to integrate cloud test lab with your CI system so when you’re writing Android tests you’re generally writing two types of tests you’re gonna write your unit tests which go in your slash test folder in your Gradle setup those tests are tests that run in a general Java VM on your desktop or on your CI server these tests are great because they were uncle’ and you can run them on any CI server because that’s it’s just Java and running in a Java VM so they’re easy to setup they’re easy to run on your CI when you can run those tests they’re great you’re going to test the individual units of your app but when you want to see if your code actually runs in a real Android environment and you want to see that those individual unit tests that you’ve tested actually come together to form an app where you click a button and this things happen you’re going to use espresso or some tool like that to write instrumentation tests and then you need a Android environment to run them in and that’s where CTL comes into your workflow so I’m gonna take a step back and kind of tell you where we were about a year ago so this is a picture from our Palo Alto office in the corner in a closet we have some Jenkins slaves we have lots of USB cables USB hubs lots of devices and this was you know it works when you have a small number of test small number of Engineers but this type of setup can quickly kind of collapse under its its own weight this is exactly like why we wanted to build a cloud test lab like we really don’t want anyone to get into the situation where they have to build something like this so we wanted to like let go we’ll take care of this for you and let you focus on just writing your code right here your app and making making sure it just is a better app and that’s it so yeah we we ended up spending a lot of time you know having engineers going an onion plugging device three plugging devices figuring out why this device was failing was the battery dead the device actually you know just overheat and and died because these devices aren’t meant to run continuous integration 24/7 it’s not the original design so the batteries will heat up and eventually you’re gonna have problems you have to maintain you have to go buy new devices when new devices come out so you’re gonna run into these problems ADB is flaky USB is flaky I got to keep them

charged and you don’t want to be spending your resources on managing this and it just kind of collapses under its own weight once you get a lot of tests a lot of developers a lot of pull requests needing to to run on these physical devices and ultimately it’s limited so if you get more pull requests they’re gonna start stacking up because you don’t have enough resources to test all these things concurrently so one pull request has to wait for this pull request so the devices can open up it just gets to become a maintenance problem very quickly so we were doing this we had this setup we had in their cloud it started on my desk if it eventually went into the closet it was working but then I saw at i/o that hey they’ve got this Google cloud test lab thing maybe this could solve a challenge that we’re having maintaining this thing so they said hey it’s early access I signed up on the site early access I met some guys at Google i/o as hey can you guys get me early access I bugged him and tell finally they said sure and we were able to become their first customer they actually I flew to Palo Alto I’m from Phoenix they came to rappel out the office they sat down with us they showed us the CTL thing and we’re like hey this could definitely solve a challenge we have so that was last July we’ve been working providing feedback all these things since then to integrate this into our system so why CTL attacked about physical devices managing them yourself can be a big burden sometimes you would still want to run on physical devices so it’s great to have Google be able to manage that for you right but also when you want to scale ultimately physical devices are somewhat limited there’s only a certain number of them if you want to run on CI you’re probably going to want a virtual devices and on running on Google compute engine there’s a lot of server space so you could spin up a lot of virtual devices and parallelized things to really improve your workflow so you’re not gonna have to deal with ADB and I’m gonna go into some of the commands that you’ll need to run here so run tests on CTL we talking a lot about this but how do you actually do this now that hopefully you’re convinced that this is something you want to do so there’s a Google Cloud SDK you go you download this this will give you a command line tool that you can invoke the Google cloud test lab from but first you need apks to run how do you get apks you run Gradle W Gradle W a symbol debug a symbol debug test this gives you two apks it gives you your app baby k it gives you your test apk which contains your instrumentation tests all right so now you have your two apks now you’re gonna run g-cloud beta test run android you’re gonna say here’s my app APK here’s my test APK here’s the devices i want to run on i’m gonna run on a nexus 4 in the nexus 5 these ones happen to be virtual devices I want to run on API 19 API 21 now notice that you kind of get the Cartesian product of all the different configurations you specify so I have two devices with two API so just this is gonna give me four test runs it’s gonna run everything four times it’s gonna run nexus 419 nexus 421 nexus 519 nexus 521 and give me all the results back now by default your test Runner in your test APK is gonna go through all your tests and run every test by default and that’s maybe fine it might be what you want once you get a lot of tests you may want to manage how you run which tests you run it shard them out and run them in parallel so you can specify a subset of your tests to run whether it be by package whether you just give it a list of class names that you want to run using the test targets command alright so you issue this command it’s on the command line you’re going to start seeing streaming output it’s uploading your apk is the cloud test lab first then it starts streaming your results the raw results to a Google Cloud Storage bucket eventually it’s going to say hey I created those virtual devices things are running you’re gonna start seeing the results at that and eventually after it’s run it you eventually get the results hey it passed on all the devices that’s great when it doesn’t pass or even when it does you’re gonna want to have take this information take this results and put it into your CI server so you can fail the build and you can inspect and see if something went wrong what went wrong right so the first key is this first URL here this is a link to a web UI for Google Cloud Storage it’s similar to like Amazon s3

it’s just a file store so you if you click on that you’re gonna see something like this the most important thing for the CI use case is the test results XML it’s a j-unit XML so anything that knows how to look at a Ju in an XML will understand this output now for the CI case in the happy path you just need this test unit XML or the test result XML when things go wrong you’re gonna have additional information here that you’re gonna want to look at you’re gonna see the logcat information so you can log as much as you want it’s gonna capture all those logs so you can see what went wrong when did it go wrong the instrumentation results if you go into that artifacts directory you’re gonna see an mp4 video file of the UI on the device as your test was running so you could see like oh this dialog was popped up so no wonder our espresso couldn’t click this button right sometimes you just need to dive in and see why your test failed like that so how are you gonna get this test result XML down on to your CI server there’s another command that you can get at this URL called GS util this gives you typical Unix commands that you can run against a Google Cloud Storage bucket you can list the bucket using LS it’ll give you a list of files that are there you can see P and you just say CP this URL this Google Cloud Storage URL to my local file system so your CI server can just copy down all of the test results it supports file globbing so you can just put a star in there and get all of your test results so now you have copied down depending on how you run your test you might have a hundred J unit test result XML so what are you going to do with that on Jenkins there’s a thing called the Jenkins xunit plugin and this is you know works with j-unit works with in unit or whatever unit testing framework that outputs the standard J unit XML it collects it it aggregates it it tells you what tests you ran it can fail your build if tests failed it gives you this history over time of what builds failed what tests failed on what builds this build had this Vinny tests succeed in you know this next build had this many more tests so basically this plug-in kind of sucks in all those results so you can do useful things with them so if you use Jenkins look at this plug-in use a different CI system look for a similar plugin so the next thing you’ll see when you get the command line results of running the g-cloud command is an ever your URL that’s going to take you to a web UI this one’s a little less useful for CI but when you’re digging into things and like a wide might s friend fail you’re gonna want to click on this and you can drill into the test results a little more you’re gonna see something like this what test failed what runs devices did it run on you could click on each device so I want to see oh it failed on the Nexus 5 level 819 device I can click on that and I can dig in for the specific test results for those devices as well so it’s kind of after the fact when your CI server says hey it failed you’re gonna go and look at this stuff right so one other use case that we’ve done on CTL is using this Facebook screenshot test library so this test library basically allows you inside of an instrumentation test to paint just one view so you could say here’s of you I want to paint it and then it allows you to compare an own set of good images against what your latest build does so that if your code change introduces any UI change for a specific view you can fill your build and say whoa this UI change is different than what we were in expecting now generally you might have no changes which is great maybe that’s what intended or you have two cases either you specifically introduced a change I wanted to change light blue to dark blue and then you’re just going to go rerecord your screenshots and make sure that everything changed appropriately that’s the case where you meant to change something then you have the opposite case where a screenshot has failed and you’re like well I didn’t think I was changing anything on that part of the UI and then it caught something very early that like oh I changed this but in the theme and I didn’t realize it was gonna affect this screen over here or this view over here and then you can go in and submit another code change to the fixes that problem so it doesn’t affect that view and then your screenshot test will pass again so in the happy case where it’s like hey I meant to change this thing light blue to dark blue because that’s what my UI designer wanted you can now not only submit the code change in your pull request you can submit the image change in your pull request so that anybody reviewing the pull request can

say oh I see the code change but now I see visually this view used to look like this and now it looks like this and you can approve that as part of the pull request in fact if you want you can even have your UI designers go look at the pull request and say like oh yeah that’s the change I asked you to do and they can approve your pull request by looking at the image because your UI designers probably not gonna want to look at code so in stash which is what we use it gives you this nice view where you could see a side-by-side image of old and new you can see it’s just a blue color change light blue to dark blue you could fade between the old and new you can kind of see like scrub between the old and new you can see a pixel diff like these are the exact pixels that change and so your UI designers can go in and see this and just give you the thumbs up and say hey you did what I’ve asked for or like well now that I see it that’s not actually what I wanted and then have that conversation earlier rather than later it’s before you merge the the p.r so this is how we’ve integrated CTL so far into our workflow how it’s helped us solve some challenges that we were facing managing physical devices trying to scale up this PR workflow so definitely go and check out CTL and see how it can work with your workflow thank you so much Michael so final thing if it’s one thing I’m gonna want to leave you with is that make sure that testing is that some isn’t something that you do at the end of your workflow make sure to have it at every step in your workflow that’s release test often whether it’s on cloud test lab whether it’s locally whether it’s with any other cloud testing providers just make sure that you have testing on every new bill that you have for your app and then definitely sign up for a cloud test lab from here and also make sure that you opt in the Play Store offering by no means this is just something that gives you kind of a safety net before releasing any new version of your app it tells you whether it at least launches on these devices or not so definitely there is something that check it out if you’re not quite listed on it already stay tuned it should happen really soon but if you are then make use of it upload your app to the alpha and beta channels and make sure that you use this Play Store offering and then yeah remember that the more testing you do then you you gotta build better apps and then eventually this means happier users and more revenues that you gotta make and yeah that’s it thanks thanks much so I think how much time do we have to take questions we have about seven minutes seven minutes to take questions so if you can come up here and tell me the question is hard to hear in here and I’ll repeat the question yes if anybody has a question wants to come up here and I’ll repeat it and then use is there a way to configure cloud test lab to use a corporate VPN oh so in the corporate network so you’re running a APK on cloud test lab and it needs to make network connections back in so one way to do that like as of two days ago we had our the api’s sorry the IP addresses of our devices published so one way to do that is to whitelist this IP range and let them hit your internal staging network that’s one way to do it and I will say in general when you’re especially if you’re doing in a continuous integration environment as much as you cannot hit a real server during your CI you’re gonna have a much better time so we use wire mock to mock out all of our network requests so none of our CTL tests actually during the pull request they can talk to real servers but generally none of them during the CI process actually talk to real servers because it just adds a whole nother level of flakiness yeah to that and if it’s blocking your developers from merging pull requests you have to make sure it’s not flaky how many physical devices are available on cloud test lab yeah so we’re definitely adding more and more as of like this moment I think we have about 15 but this is like really low we’re adding more and more like you should expect by May you should expect a lot more than that and then until the end of the year even even like double that more than double what we will have by May so stay tuned but as of right now the easiest way to know what devices are supported just like go check it out open the either from G cloud you can run a command to list all the devices or from the web interface just see what devices are there yeah it doesn’t require it requires all the yeah the video of your test executing on the phone on the day

yeah yeah yes yeah so that’s actually one of the results that you get to get the logcat you get the screenshots and you get a video of your test yeah so the question is do you have an SLA for submitting the test and getting their results back it’s a great question we haven’t published essays yet something we’re still working on it might be a bit challenging or and it might take a bit longer on physical devices in particular so stay tuned for that but like so there’s three parts right – like that introduce latency there is the overhead time which is the time we take to set up the device install your app launch your app and then there is the test run time so this is how long your test runs and then there’s also like the queuing time right so there is X number of devices in the system your test was submitted but it’s like number five in the queue so the queuing time is the hardest to control this is what prevents us from kind of publishing any SLA s especially on the physical devices side but on the virtual devices and we can easily like publish lines for overhead and test time is something you can control so that’s that’s great yeah so what the question was is it the robo test can it go in your through your sign-in flow is that the question yeah so as of right now we’re working on actually adding the sign-in especially with a Google sign-in and we’re gonna be adding more different sign-in flows like Facebook sign-in or others but we’re also working on other alternatives like one approach that we’re also like starting to work on is allowing you to upload a bootstrap file so just give us a script that passes the login screen and then we take your app from there so also stay tuned on that this is some this an area where we’re investing a lot and we’re like very interested in yes so the question was do you have any plans to support iOS devices and simulators iOS is a very interesting question so initially we had these plans right now we still have the plans but they are not as highly prioritized as as they were and the reason is we we believe this like fragmentation problem is affecting Android more than iOS and we really want to make sure that we get its rise on Android first and then start like getting into into iOS but it’s something that we definitely want to support at some point it’s just that it’s a matter of priorities so so the question was when you’re setting up your own physical device lab you run into things like the device went offline or you lose connection so how stable are the physical devices on CTL yeah so the whole idea is is like the whole idea the whole reason we wanted to build this cloud test lab is not letting you worry about this like device stability issues and let us take care of them so that’s right now like in terms of stability in terms of like raw numbers like we’re at like 99 point something percent stability on these devices but we keep on refrying we try on another device if this defy device fail in this way like we try to hide it from you so that you you get like a test that runs and we worry about like hey the device went off line will be gonna send someone to reboot it or things like this so the quick so the question is could this work with other CI systems besides ink Jenkins and he says specifically build bot so I can start to answer this one so the things that I gave you in terms of the cloud test API if you can install the cloud test SDK or the Google Cloud SDK and if you can write a script that calls the g-cloud command and looks at the results you should be able to use that information to create a set up specific to yours so really nothing I showed here I just happened we happen to use Jenkins was really all that Jenkins specific it’s just a command-line interface that you can invoke and that’s why I didn’t go too much into the details because different people use different CI systems so so I think if I understood the question

correctly it was around the Facebook SDK and whether or not that’s only for instrumentation tests or could you use it during normal app usage to capture screenshots is that the yes so that’s actually the intended usage of the Facebook plugin is only in our instrumentation test it’s not something that would ever ship it or should assuming your Gradle is configured correctly whatever ship in your your actual app how much time we have left we’re we’re almost there maybe yeah we can take one more question okay so the question is what about multi device use cases where you want to test it like I have an app that is a collaboration device this ties to talks to this device and I need to talk the