Jae Salavarrieta - HoloLens: Drone Mixed Reality App

My Role: UI / UX Lead Designer Company: Aerobo Date: September 2017

Tools: Invision, Cinema 4D, Unity, Photoshop, Illustrator

Deliverables: Strategy, Storyboards, Wireframes, Prototyping, Production Assets,

IOS app test, Hololens MVP

The Objective

My role was to develop novel ideas for 3D UI's for augmented and mixed reality applications mainly on a MVP/initial concept scale and not as a final product. Create high fidelity 3D user interface components and design assets to be usable in Unity. I worked with the team to understand all aspects of the project to determine high level goals and to know the limitations of all possible technical constraints. Also, to create a variety of iterations to achieve compelling, working, interactive demos.

Background

Due to my contract with this project, I am unable to speak about or show all the design components, specs, and fine details surrounding this minimal viable product, but still wanted to share some insights into my process to highlight my experience in this field for other potential work.

With AR/VR/MR increasing in popularity, there is a growing need to find ways to intertwine their functionality into our daily lives. Drones are another rising tech that has been booming in the filming and photo industry. There are current mobile apps that can set a predetermined path for the drones to execute, but only on a 2-dimensional level. This project sets out to add in a z-axis component to bring more flexibility to the drone's path.

Getting Started

Looking at things from a different perspective.
The parameters of this project began from an isolated Hololens M.V.P with a timeline of 2 weeks for design. Developing a Hololens app that can set a manuel adjusted path in a 3-dimensional space of an existing structure and have the drone execute a set path proved to be challenging. Here were some of the design challenges:

• What was valuable for the user?
• Who are the primary users? What is their goal?
• Where will the demo be held? What areas will this be used in?
• What are the restrictions in not only design, but technical aspects as well?

This is only the tip of the iceberg of a long list of questions one can start with for a mixed reality project. Projects in AR/MR environments require a solid initial plan that also allows for flexibility and adjustments along the way.

Iteration Is The Key

Creating for every possible outcome, makes happy teams

After tons of research to guide the product direction, I went through many user journeys and possible affinity diagrams where most of my time was spent on paper and post-it notes. I was moving quickly between paper and digital storyboards, prototypes, testing, and getting feedback from my sole developer. From each iteration, I learned something valuable. With this level of complexity, I find this is best done in conjunction with all team members from start to finish.

I focused my early iterations on:

• Type + color combinations for spatial
• Functional 2D/3D interfaces
• Motion ( including trigger animations, interactions, states)
• Audio Cues within the 360° space
• Locations of all UI & scanned object (spatial UI)
• Gestures

For the first phase, I was originally focused on just delivering a basic user flow. Spending most of the time tweaking the user interface to be pleasing to the eye yet simple to understand with little to no onboarding. I experimented with different styles and positions that a HUD can be viewed without obstructing the users view from their primary task in creating a path. For the first design pass, this for me are "safe" zone components so we can complete testing, while I also plan for the second design pass that can be incorporated after demo phase.

Product Design

Prioritizing the features based on research

Despite concerns, what started from an isolated demo in Hololens now expanded to a publishing app to connect to an actual drone for live testing and an isolated iOS app, still all on an MVP level. For business purposes, having two access points to the app expanded the audience of users. With a mobile app now in play, the best way to shine is to learn about the current landscape and other app strategies. Tying in user research and competitive analysis when making decisions on any features is the best way to determine what holds the most value in MVP.

IOS app - What To Consider:

• combination of both the Hololens and Publish app capabilities

Design Decisions

Be considerate of human factors

Since there weren’t many competitors, research showed there was a clear opportunity to create an original, visual, interactive library for the interface interactions. This would set the foundation for any similar apps and tech moving forward, such as when the hamburger icon was copied and then became standard. At this stage, the path represented a 3D timeline; each “button” contained multiple stages and triggers to let the user know the current status. Recreating it also in 3D (size, polygons,texture), then apply effects (unity, after effects), create an animation tests/storyboard, collect audio, transfer to unity, and then test.

Animations / Texture Tests

3D / Spatial Interface Tests / Concepts

Initial Test Prototype Draft

UI concept in action. Several of these concepts are prototyped live through the device to test things such as placements, timing, colors, sizes, motion, and usability. (same password)

Reflections

Why is there never time to do things right, but always time to redo them?

“First time quality third time around…”

This is the type of project I am very passionate about, especially because it includes all things encompassing design (not just 2D). It was a big learning curve from mobile AR, but only really on the technical aspects. Some things I have learned are:
. . . . . .

• Consider the human factors: like size of fingers and spatial objects in relation to height for Hololens viewport
• Test in different environments, especially where the user would use it.
• More visuals and less copy for overall experience. This is crucial since there are extra cues available like sound and voice you can incorporate.
• Teamwork is more crucial than ever in this project, you need people to push you and vice-versa.

You may also like . . .