486 Releases in 14 weeks

"I thought I knew how to move quickly. I thought I knew how to guerilla user test. Wrong and wrong." 

Last year I joined Aedit, a New York-based company focused on destigmatizing surgical and non-surgical cosmetic procedures(from Botox to boob jobs and everything in between) as a technology consultant. The near term goal was to launch three products: an iOS mobile application (which would become The Aeditor), a web-based marketplace for doctors and candidates, and an admin portal. The Aeditor is a ‘selfie’ app that allows people to apply a cosmetic procedure—like a lip lift— to their face and see what they would look like after.

Build a web app, a mobile app, and an admin—no problem. I’ve got this. I’ve done this hundreds of times. Piece of cake right? Um... no. I hadn’t built anything like this; something that required such an immersive experience, something that required hundreds and thousands of users to springboard the tech, something with such an intimate user experience.

What was different about this app? Why did it need to such an enormous amount of feedback? Let’s start with the basics. Camera-based apps are unique because:

• Cameras differ from one iPhone model to next (different screen canvas sizes, varying sensitivity to light, front facing vs. rear facing camera limitations).
• It’s tough to simulate real users, you need actual bodies testing the camera, the lighting, angles, etc. It’s a much more intimate experience that necessitates testing on real hardware as opposed to an iOS simulator.

Furthermore, our application had some specific constraints and requirements:

• We needed to develop facial recognition and landmark detection technologies. This involves detecting the face and then detecting the landmarks within it (eyes, nose, lips, ears and hair). Then we had to account for people smiling, wearing glasses, tilting their head, hair in their face, etc. All of these nuances can throw off the facial recognition and landmark detection and needed to be accounted for.
• Morphing needed to be realistic. We are not Snapchat, Facetune, or a makeup app with flower crowns. We have a responsibility to create morphing tools that accurately represent true-to-life cosmetic procedures (think rhinoplasties, lip lifts, eyebrow microblading).
• Morphing had to look good on everyone. Yes, everyone. Did you know that the fastest growing demographic in the cosmetic surgery space is people 65 and up? Neither did I. We had to account for all ages, ethnicities and skin types.
• Our team actually had to build two apps. One for our engineers to use in prototyping the facial morphing (we call this the ‘Labs’ app) and one user-facing version for prospective procedure candidates (our ‘Production’ build). We needed to perfect the tech on one while paying special attention to the UX on the other.

Accomplishing all of this would require an extremely tight workflow. One that supported distributing real time updates to our beta groups. This workflow needed to give our team the ability to pivot and move at lightning speeds, so we chose tools that would help us move quickly:

• Clubhouse for product management
• Google Forms for internal product requests
• Typeform for user surveys
• Github for code management
• Bitrise for continuous integration/deployment
• Mailjet for email notifications

All the tools are great on their own, but we needed to integrate them to work together. We set up my favorite project management tool, Clubhouse, and created a process that gave the entire organization visibility into the status of every initiative. 

In the above you can see how product requests turn into an actual release. Tickets start in a product request lane and flow from left to right moving through design phases, development stages and eventually into ‘Deployed Live’.

Employees at Aedit are encouraged to use our Google Form to submit product requests. I integrated the Google Form via App Script and the Clubhouse V2 version of their API. You can check out my code in my github repo. From there, product requests are approved and turned into design or development tickets.

Once the development team picks up a ticket, the automation kicks in. Clubhouse has a nice integration with Github where tickets are automatically associated with feature branches and flow from ‘Ready for Development’ to ‘In Development’ to ‘Code Review’ (PR opened) to ‘Deployed Staging’, eventually finding its way to ‘Deployed Live’. Having all of this automated keeps the product stakeholders in the know so they can test, schedule and plan accordingly. We don’t need to rely on humans moving cards and we can set up automated notifications (in the form of emails) when cards change lanes.

When code is merged to the staging and master branches, not only do the cards move but our awesome continuous integration (CI) tool, Bitrise, takes care of running automated tests, making builds and emailing them out to the team. If you haven’t used Bitrise before, it’s a CI tool on steroids. What sets Bitrise apart from other mobile CI/CD tools is the time they have spent developing a community that regularly contributes and open sources their steps. Need to integrate with an email service? Someone has done that. Want to put your builds on an Amazon device farm? There is a step for that too.

The Bitrise “steps” we used are fairly straight forward:

• Check out the code from Github. We use git submodules because we share the facial morphing tech between our ‘Labs’ and ‘Production’ apps. Bitrise supported the submodules with no issues.
• Automatically change build numbers. We needed a way to map feedback to builds, so we turned on a step that automatically changes the build numbers that are surfaced in the settings section of our app.
• Run XCode tests and create an archive to cover some basic automated testing.
• Deploy our dsym to Crashlytics. We have crash reporting in our app and wanted to automate the process of uploading the dsym.
• Store our build artifacts in S3.
• Distribute the new build for installation via email.
• Email the team with the new build to install.
• For production builds: deploy the release to iTunes Connect via Fastlane with all our store assets managed via Github.

One of the common complaints I often heard from teams was, “I get so many updates; I don’t know what's new.” This was another problem that Bitrise could help with; they offer a custom script step where you can run Ruby code that you have developed yourself. So I wrote a custom Ruby script that does the following:

• Hits the Clubhouse REST api and gets all the stories in the Deployed Staging lane using this search query; /search#project:aeditor%20state:staging
• Takes the Clubhouse stories and formats them into HTML markup
• Takes any label that starts with ‘http’ and wraps that in an a href tag
• Uses Mailjet’s templating to send out a formatted email that contains:
• A link to install the build
• A list of every story from Clubhouse that is included in the build, and a corresponding link to view the Clubhouse story
• A link to any screenshots or assets, all controlled from the labels (we regex on http in the labels and link out to those)

So what about feedback from the beta users? We implemented a process for collecting that too. Every time we tag a build, we inject the Typeform link from our survey into the email template (pictured above) and send an email to our beta users. We have been sending builds to our beta users on a monthly basis, so the number of stories in each email can be a little lengthy (we search for stories via the Clubhouse API from the last tag date to current date). Nevertheless this has proven effective and has actually helped me write user stories more effectively (the title of the story no longer has 1 audience).

So what were the results? the process helped? Our goals were to setupautomation throughout the organization so that we could iterate quickly on a real prototype and I believed we have achieved that. We have:

• Sent out 486 builds in 14 weeks.
• Collected feedback from everyone inside and outside of the organization in an automated manner enabling us to make changes in real time.
• Rebuilt our ‘selfie taking’ experience three times (I’m not talking about small adjustments either—complete teardowns). This was only possible because of how quickly we could get the app into our users' hands.
• Retrained our machine learning model (dlib) numerous times, using the pictures users have submitted, accounting for hair in the face, poor lighting, glasses and a number of other factors.
• Dramatically improved the user experience of our production build.

So what’s next? We have a ways to go with Bitrise steps, such as Github releases and Microsoft App Center Testing, which we could incorporate to improve our automated testing and enable us to run suites of tests on real devices. We could improve our build times (currently 20 minutes) by making use of different Bitrise caching strategies. And of course, real time training of the current dlibmodels would be fantastic.

The Aeditor is available in the iOS app store today (US only). If you have any suggestions or want to learn more specifics about the steps drop me a line.