The Cloud Grey Blog

Test.ai + Appium Element Finding Plugin Improvements

If you weren't at SeleniumConf 2018 in Chicago, you might have missed the talk by Jason Arbon (with a cameo by yours truly) on work I did in conjunction with test.ai to bring Artificial Intelligence to finding elements with Appium. Here's the video in case you want the full scoop:

In the last months, we've looked for ways to improve the performance and accuracy of the plugin, and there are two recent changes worth mentioning:

  1. Images are now classified in their greyscale format rather than color RGB. The purpose for this intentional loss of information is twofold: first of all, users are typically able to recognize icons by their shape alone. Color is obviously important, but most icons can be reasonably classified with only reference to shape. Because of this, there's no real reason to include color information as it makes the learning space bigger. That is why, second, the original model was trained on greyscale images. Using greyscale images in the plugin thus improves prediction accuracy.
  2. The plugin now takes advantage of TensorFlow's ability to classify multiple images in parallel. While the image classification step was never the long pole in the tent (so to speak), this improvement just about doubles the classification speed.

There are still ideas we have to make this plugin better, for example figuring out how to avoid getting every single element and its screenshot from the app using XPath, as this is incredibly time-consuming!

If you haven't yet tried out the plugin, be sure to read the Appium Pro article on the plugin to learn how to get started. And then, make sure you have the most recent version (2.1), which contains the improvements listed above. Happy testing!

Jonah Stiennon Joins Cloud Grey

Jonah Stiennon

Cloud Grey is pleased to announce that Jonah Stiennon (GitHub, LinkedIn) has joined Jonathan Lipps (Cloud Grey's founding principal) as the firm's first partner. Jonah has a long history with Appium, having been one of the first people hired by Jonathan to work full-time on Appium at Sauce Labs, in the initial days of the Sauce Labs Open Source team.

Despite having taken a few years to pursue other opportunities, Jonah still sits in the top 10 Appium contributors by commit count. More recently, you may have noticed Jonah contributing some posts to the weekly Appium Pro newsletter (for example, an epic guide to Getting Started With Appium For iOS Real Device Testing). We, and the Appium project, are excited to have someone with such history and context rejoin the industry and get hacking on Appium again.

It's not just the Appium community at large that will benefit from Jonah's involvement, however. Cloud Grey's clients will also be glad to hear of the additional bandwidth we will have for trainings, strategy consulting, and other services. For Jonah's part, he's excited for the opportunity to put his open source prowess to work, and directly help solve client problems as part of Cloud Grey:

"I'm excited to be re-entering the Software Test Automation industry, and looking forward to working with Jonathan and Cloud Grey. This is a big step for me personally, and I look forward to applying my skills and knowledge towards leveling up all the QA and software test professionals who comprise our community."

Everybody please give Jonah a hearty welcome! And if now is the time you've been waiting for to get some help putting your mobile automation in order, don't hesitate to contact us.

Test Talks Debut

TestTalks

Last week I had the pleasure of recording a super-fun interview with Joe Colantonio on his Test Talks show. We talked about Cloud Grey, Appium Pro, and a host of specific Appium-related topics.

Check out the full episode and don't forget to sign up for Appium Pro!

Hello World

I am thrilled to be announcing this brand new consultancy. For a long time I have seen the need for greater professional assistance in the world of Appium automation, and the time has come to step out and make it happen. The idea behind Cloud Grey ("Appium services") is simple, and will no doubt expand and grow over time. Regardless of what it becomes, I am confident that it will enable me the time and space to continue contributing to the Appium project.

There's a lot to be excited about with Appium right now. New features, new docs, new drivers, and AppiumConf right around the corner. Of course, the world of mobile automation is as confusing and volatile as ever. I hope that Cloud Grey will help Appium users to further understand the bewildering array of choices faced when starting out automating their mobile apps.

I am particularly enthusiastic about the fact that Cloud Grey, even before our official launch, has joined the JS Foundation. The JS Foundation is the non-profit foundation that hosts the Appium project, along with a number of other important projects in the JS ecosystem. Given that Appium relies completely on this ecosystem, and the same is true of Cloud Grey, it's only right to support the ecosystem financially.

Last, but not least, we are also launching Appium Pro, our new weekly Appium newsletter. It's designed to share interesting tips and tricks for using Appium more effectively, along with any important news from the community. Go on, subscribe now!

Thanks for reading, and don't hesitate to contact us if you have any questions, or want to see particular kinds of content in this space. The best way to keep up with this blog will be to follow our Twitter account. Happy testing!