Mockup of a person using viewing UI floating while using AR glasses
In this article we’ll learn a bit of the history of UI, how it was born and how it has become what it is today. Then we’ll learn a little about how UI has adapted to other digital products such as smart watches. Finally, we’ll take a look at the state of UI in VR today and how it is evolving in terms of technology and Design Systems.
The first UI in history
Xerox Star’s UI example
The first UI, as we know UI nowadays, was created by Xerox Palo Alto Research Center, aka Xerox PARC in 1973. They developed Xerox Alto, which was the first personal computer to use windows, icons and menus. However it wasn’t a commercial product. Later on, in 1981, they launched Xerox Star, which was a commercial product. Both influenced future developments like the firsts Apple personal computers, Lisa and Macintosh. To put a little context, January 1st, 1983 is considered the birth of the internet.
Apple Macintosh UI example
UI evolution
Technologies
Logos of HTML 5, JavaScript and HTML 3
After these first UI inventions, brilliant brains started to work on this and standards started to come up. As it is usual in tech, the first to arrive is engineering, it owns the tools to create functional things. Design, cannot expect engineering to make these things work and be delightful at the first attempt. So, HTML (Tim Berners-Lee, 1993), JavaScript (Brendan Eich for Netscape, 1995) or CSS (Håkon Wium Lie, 1996) are the basis for what later on will be our beloved Design Systems.
First Design Systems
Before Design Systems, it was about pattern libraries. The need for patterns was not something new. So different libraries were out there to use, but the process was something like this: This set of inputs from this library, I will create the buttons, I do like these sets of icons and so on. Designers and designers struggled to maintain consistency. Upon the need of achieving more consistency, we went from designing interfaces to use design systems.
An example of YUI buttons
An example of Bootstrap buttons
An example of Material Design components
Designers join the game, product design tools
Obviously, pioneer designers took part in the design of the mentioned Design Systems. However, in this section I’ll talk about the designers who use these Design Systems to create products, not the ones who designed the actual Design Systems.
An example of a UI kit for Phostoshop
So, in order to create easy to use and delightful products, designers need a tool to work with. It started with Adobe Photoshop or Illustrator, which provided the tools to re-create the components visually and work from there, but they lack features to maintain consistency.
Logos of Photoshop and Illustrator with an arrow pointing towards the logos of Sketch Adobe XD and Figma
After that, in 2010, Sketch was launched, and it was revolutionary since it was built especially for product design and it included all features needed to maintain consistency across the components and styles. Learn more about Sketch history here
Later on, in 2015, Adobe launched Adobe XD. A tool with the same purpose as Sketch, helping designers on creating and maintaining digital products. Learn more about Adobe XD history here
A year later, in 2016, Figma was launched. A tool with the same purposes but cloud-based. Learn more about Sketch history here
At this point, designers already have the tools to work properly on digital products. These tools provide designers with design, prototype and testing features.
Beyond mobile and desktop
Sketch, Adobe XD and Figma have the capacity to adapt to more than mobile and desktop apps, they also can be used to design UI for other devices such as smart watches for instance.
On the other hand, due to the size of smart watches, different design systems watch-focused have come up.
Watch using Wear OS and showing a messaging UI
Wear OS from Android is the equivalent to Material Design for smart watch apps
Watch using WatchOS and showing a workout app UI
WatchOS from Apple is the equivalent to Human Interface for smart watch apps
UI in VR
An example of a woman using a VR headset and interacting with the UI in VR
It is obvious that the industry has understood that a design system is required for a product to become mainstream. Designers need an organized set of components to create.
Technologies and devices
Unity and Unreal logos
As I mentioned before, engineering comes first. VR apps are developed using 3D engines based on C#. These are Unity and Unreal. Worldwide known as the main tools to create video games. This is the first interesting point, how the world of apps starts to mix with the gaming world. This fact brings on the table 3D modeling tools such as Cinema 4D and Blender, just to mention two popular ones.
Three of the most popular XR devices: Oculus Quest 2, HTC Vive and Hololens 2
As for the devices, there are several players out there, similar to the smartphone market. Each one with its own resolution and properties. Here you can check out the most populars.
Design Systems and frameworks in VR
As for Design Systems as such, the market in VR is not, by far, as mature as it is for mobile and desktop. However there are some solutions that can be used as a starting point and inspire what is coming.
Unity UI Toolkit
A screenshoot of the UI Toolkit in Unity
UI Toolkit is the newest UI system in Unity. It is based on standard web technologies so it allows you to define reusable components and styles. However it is far from being a Design System as such, it just gives you the tools to define your design system and maintain it.
MRTK
Example of a menu in MRTK
Mixed Reality Toolkit is a framework from Microsoft that has a predesigned set of components and interactions. It has a View Only Figma file too.
Example of a menu in MRTK 3
MRTK just launched (2022) MRTK 3, which introduces a big improvement in terms of UI.
Oculus Integration
Example of a menu and buttons in Oculus Integration
Oculus Integration is an SDK from Meta that provides predesigned sets of components and interactions.
As we can see, the market is still engineering focused. For now, as a designer, it is difficult to jump into the opportunities that VR offers to UI without Unity or Unreal knowledge. However it is not impossible, there is information about how to design UI for VR out there waiting for you, but, be ready to break your 2D mindset.
An example of some UI components of FloatGrids
In order to make this transition easier, I designed and developed a little Design System for VR. It is available in Figma and Unity. The idea was to close the gap between engineering and design and at the same time provide some tips for designers when it comes to designing UI for VR. You can check it out here
UI tooling for VR
As for the tooling, the history repeats itself, but with some learnings. So far, neither Sketch, Adobe XD nor Figma are ready to provide the new features that UI in VR projects require. However, as it used to happen with Adobe Photoshop and Illustrator back then, we still can use them to design UI for VR.
However, there are some XR dedicated prototyping tools such as ShapesXR, which has MRTK and Oculus Design Systems available to use and prototype with them.
Conclusions
Imagine being there when the first design systems were being conceived. As I see it, this is where we are now in terms of UI for VR.
Engineers and big players such as Meta, Google, Microsoft, HTC and Apple are working on delivering easy-to-use and delightful experiences for both VR and AR. In terms of VR Meta is clearly taking the lead. I wonder what Apple is working on, rumors say that they will launch its headset in 2023. What is for sure is that its device will come with a VR/AR tailored Design System. However, I wonder if it will actually be a headset, they are known as launching products for day-to-day use, will they launch actual AR glasses? Just normal glasses meant to see better or protect from the sun but enriched with AR features? I don’t know, but what I foresee is that the XR world will become wild in the following years.
Thank you for reading! 💜