Category : iOS

AR/VR Design Design Thinking iOS Technology

Designing for Vision Pro: the coming spatial experience

As the real and virtual worlds continue to converge, Apple has introduced Vision Pro, a device designed to deliver immersive mixed-reality experiences. Vision Pro enables endless possibilities for exploring human-to-system interactions and spatial dynamics. Users can interact with the digital world more naturally using the mixed-reality headset. It overlaps digital content in the real world, allowing users to simultaneously see and interact with both worlds. Vision Pro is slated for limited release to the public next year, but Apple has already given a demo to select developers.

Vision Pro features two high-resolution micro-OLED displays that project images directly onto the user’s retinas, creating a “screenless” (screen without a screen) experience. The headset has 12 built-in cameras for tracking the user’s head, eyes, and hands, a LiDAR scanner for depth perception, and an Apple M2 chip with 10 CPU and 32 GPU cores.

Vision Pro is designed with accessibility at the center, using gestures, voice, and combinations of these input modes as critical enablers. This was showcased in the launch event video, where users could interact with the headset using gestures to browse the web, watch movies, and play games. Designing for Vision Pro requires understanding these input modes and providing solutions for users to have a seamless journey.

Apple Vision Pro Gestures Feature

Source: MacRumors Apple Vision Pro Gestures

Designing for a spatial experience

Design principles for traditional screens may not work for the future of spatial experiences, which are more immersive and interactive. Designers need to adapt these principles to consider the unique features of spatial experiences. Spatial experiences allow users to interact with the environment more naturally, so designers must create interfaces that are easy to use and understand.

Vision Pro’s spatial capabilities allow designers to create visually captivating and user-friendly interfaces that feel intuitive and natural to navigate. This focus on user-centric design enhances the user experience across various applications. Below are some key aspects to consider while designing for glass-style UI:

Iconography

To create a seamless user experience, Apple has reimagined the feedback mechanism for icons on the home screen. When a user looks at an icon, it expands as if it is being hovered over. Designers design icons with subtle depth by adding specular shadows and highlights.

To design a great icon, start by creating multiple layers. The system uses these flat layers to create a three-dimensional effect. The icon should be converted to 1024×1024 pixels, with transparent foreground layers. Do not use large regions of semi-transparent pixels, as they will blend with the shadow behind them.

Consider these additional recommendations when designing icons:

  • Maintain a uniform color scheme and style throughout your icons.
  • Make sure your icons are clear and easy to understand.
  • Use high-quality images and graphics.
  • Test your icons on different devices and screen sizes to ensure they look good everywhere.

apple vision pro icons for spatial immersive experiences

Source: App icons | Apple Developer Documentation

Glass panels design

Apple has introduced glass material to create a more spatial and lightweight user experience. This material allows users to see what is behind a window, such as other apps or people, without feeling suffocated.

Therefore, when designing an app window, it is essential to avoid using solid colors. Too many opaque windows can make the interface feel heavy. Instead, use a lighter material to bring attention to interactive elements, such as buttons, or a darker material to separate sections of the app.

For example, if you want to design a lock-up with a lighter button, you can place it on top of the glass material. Or, if you’re going to create more contrast, you can use a darker cell behind the button. However, it is crucial to avoid stacking lighter materials on top of each other, as this can impact legibility and reduce contrast.

Consider these bonus tips for designing with glass material:

  • Use glass material sparingly. Too much of it can make the interface feel cluttered.
  • Use glass material to create a sense of depth. For example, you can use a darker glass material for the background and a lighter glass material for the foreground.
  • Use glass material to highlight essential elements. For example, you can use a lighter glass material for buttons or other interactive elements.

apple vision pro glass material design for spatial immersive experiences

Source: Materials | Apple Developer Documentation

Legibility

Typography

The font-weight can be slightly increased to improve text contrast against vibrant materials. For example, on iOS, regular weight for the body text style must be used; on this platform, a medium can be used. And for titles, instead of semi-bold, bold can be used. This makes the text more legible, even when displayed on a vibrant background. System fonts, which are designed for optimized legibility, can also be used.

Vibrancy

Vibrancy is another crucial detail for maintaining legibility. It enhances the foreground content by bringing light and color from the background forward. On this platform, vibrancy updates in real-time to ensure your text is always legible, even when the background constantly changes. Vibrancy can indicate hierarchy for text, symbols, and fills. Primary vibrancy can be used for standard text, and secondary vibrancy can be used for less critical text.

Pointers for using typography and vibrancy:

  • Use a heavier font weight for text that needs to be legible, such as body text and titles.
  • Use system fonts or other fonts that are designed for optimized legibility.
  • Use vibrancy to brighten foreground content and make it stand out from the background.
  • Use primary vibrancy for standard text and secondary vibrancy for less critical text.
  • When using custom fonts, make sure they are designed for readability.
  • Avoid using small or lightweight fonts, which can be challenging to read, especially on large screens.
  • Consider using a darker shade for the pop-over background to make the text more legible.

apple vision pro legibility typography vibrancy for spatial immersive experiences

Source: Typography | Apple Developer Documentation

Colors

When designing a glass-style UI, use white text and icons on a colored background. This will make the text and icons stand out and be more legible.

  • Use system colors whenever possible. System colors are designed for legibility and look best on a glass background.
  • If you need a custom color, use it sparingly and make sure it contrasts nicely with the background.
  • Avoid using dark colors for text or icons. Dark colors will blend in with the background, making the text and icons challenging to read.

Here are some additional tips for designing glass-style UIs:

  • Use a light overall color palette. This will help to create a sense of spaciousness and airiness.
  • Use transparency and blur effects to create a sense of depth.
  • Use shadows to add dimension to the UI.
  • Use gradients to add interest and visual interest.

apple vision pro colors design for spatial immersive experiences

Source: VisionOS – Apple Developer

How Vision Pro can transform businesses

Businesses of all sizes and across some industries are excited about the potential of Apple Vision Pro, which is still in its early stages of development. The technology has the potential to transform operations, improve customer experiences, and boost overall performance. Still, which industries will most successfully adopt it remains to be seen.

Entertainment

  • Create immersive gaming experiences that blur the line between the virtual and real worlds. To illustrate, game developers can use Apple Vision Pro to create realistic first-person shooter games where players can interact with the environment lifelike.
  • Create interactive storytelling experiences. Filmmakers can use Apple Vision Pro to create 3D movies, transporting viewers into breathtaking cinematic worlds.
  • Provide real-time translation of foreign language text. For example, language learners can use Apple Vision Pro to get a real-time translation of foreign language text while traveling or interacting with people from other cultures.

apple vision pro application for spatial immersive experiences for entertainment and sports

Source: How will Apple Vision Pro VR influence industries | Merge Development

Education and training

  • Provide students with interactive learning experiences. For instance, teachers can use Apple Vision Pro to take students on virtual field trips to historical sites or to conduct experiments in a safe and controlled environment.
  • Offer virtual field trips. For example, students can use Apple Vision Pro to visit museums or other educational institutions without leaving their homes.
  • Provide real-time translation of foreign language text. Concretely, language learners can use Apple Vision Pro to get the real-time translation of foreign language text while taking a class or reading a book.

Healthcare and medical

  • Provide realistic surgical simulations and training scenarios. In other words, doctors and medical students can use Apple Vision Pro to practice procedures without risking harming a patient.
  • Offer remote consultations with patients. For example, doctors can use Apple Vision Pro to consult with patients in remote areas.
  • Visualize and analyze medical data. For instance, researchers can use Apple Vision Pro to visualize and analyze medical images and data to understand diseases better and develop new treatments.

apple vision pro application for spatial immersive experiences in healthcare medical industry

Source: How will Apple Vision Pro VR influence industries | Merge Development

Real estate and architecture

  • Give potential buyers virtual tours of properties. Real estate agents can use Apple Vision Pro to give potential buyers a 360-degree view of a property without meeting them in person.
  • Collaborate with clients and stakeholders on 3D design projects. Architects can use Apple Vision Pro to collaborate with clients and stakeholders on 3D design projects in real time.
  • Visualize furniture and decor in a physical space. For example, interior designers can use Apple Vision Pro to visualize how furniture and decor will look in a room before making a purchase.

Meetings

  • Join virtual meetings from anywhere. For instance, remote workers can use Apple Vision Pro to join virtual discussions from anywhere.
  • Collaborate in real-time and share information. For example, participants in virtual meetings can use Apple Vision Pro to collaborate in real time and share information.
  • Track customer behavior and improve the shopping experience. Concretely, businesses can use Apple Vision Pro to track customer behavior in a retail store and use that data to enhance the shopping experience.

apple vision pro application for spatial immersive experiences in workspace education and meetings

Source: Apple’s Vision Pro: Revolutionizing Industries Through Spatial Computing| ELEKS

Finance and banking

  • Visualize financial data. For example, financial analysts can use Apple Vision Pro to visualize financial data to understand the market better and make informed investment decisions.
  • Help clients track their spending. For example, personal finance managers can use Apple Vision Pro to help clients track their spending and reach their financial goals.
  • Visit virtual bank branches. Concretely, customers can use Apple Vision Pro to visit virtual bank branches to conduct transactions or speak to a banker.

Retail and e-commerce

  • Try on clothes and accessories before making a purchase. To illustrate, shoppers can use Apple Vision Pro to try on clothes and accessories before purchasing.
  • Provide personalized shopping recommendations. Retailers can use Apple Vision Pro to provide personalized shopping recommendations based on a shopper’s past purchases and browsing history.
  • Offer in-store navigation. For instance, businesses can use Apple Vision Pro to offer in-store navigation to help customers find the products they are looking for.

apple vision pro application for spatial immersive experiences in retail ecommerce fashion

Source: Apple Vision Pro and The Future of ECommerce (codilar.com)

Vision Pro: the flip side

Innovation can be exciting, promising a brighter future with endless possibilities. However, carefully considering the potential consequences of new technologies is essential. While innovation often brings benefits, it can also come with risks, such as privacy concerns, surveillance risks, and impacts on mental health and social isolation. Balancing progress with responsibility when developing and using new technologies is essential.

Comfort and ergonomics

Early reports suggest that the Vision Pro is well-built but slightly uncomfortable to wear for extended periods. The headset is front-heavy due to its metal construction, which could make it difficult to wear for long periods. Additionally, the headset’s weight distribution and heat management could further impact user comfort. If the Vision Pro is not designed to be comfortable, it may limit its appeal to consumers and businesses.

Privacy at stake

The Vision Pro raises fundamental concerns about personal privacy. The headset’s ability to project floating screens onto our vision while observing our environment could collect vast data about us. This data could include our eye movements, facial expressions, and surroundings. The potential for this data to be used to track our movements, monitor our behavior, and even identify us is a serious privacy concern. Establishing robust safeguards and ethical boundaries is essential to protect individuals’ privacy in the digital age.

Mental health and social isolation

While technological advancements can enhance our lives, we must also consider their potential impact on mental well-being and social dynamics. The Vision Pro’s immersive AR experience could be a double-edged sword. On the one hand, it offers captivating virtual overlays of reality that could tempt users to immerse themselves in a captivating digital realm. On the other hand, this allure could come at the risk of isolating individuals from their physical surroundings and authentic human connections. As we increasingly detach from the present moment and substitute genuine interactions with virtual experiences, the potential for social isolation and its associated mental health consequences is serious.

apple vision pro designing for spatial immersive experiences

Source: Apple Developer Forums

Apple’s Vision Pro hasn’t even been released yet, but the company is already planning a smaller and lighter version of the headset and is deep into work on follow-up products.

Conclusion

Apple Vision Pro can redefine how we interact with technology in a human-centric way. Its spatial capabilities allow designers to create visually appealing, user-friendly interfaces that feel natural. This focus on human-centric design will enhance and transform businesses of all sizes and industries by enabling immersive, interactive, and personalized experiences in various applications, from gaming and entertainment to education and training. While Vision Pro can be a powerful tool for good, it is essential to remember that it is also a new technology with potential risks. Our collective responsibility is to ensure that Vision Pro is used responsibly in a human-centric way.

Read More
Andriod iOS

Functional Programming Goes Mainstream

Functional programming is a programming paradigm where applications are composed using pure functions, by avoiding shared mutable state, by avoiding side effects and by being declarative. We can contrast this with Object oriented programming where data and behaviour is colocate. Procedural programming is an imperative style which groups algorithms into procedures that tend to rely on shared mutable state. Functional programming allows to compose functionality using functions. Composition is a simple, elegant, and expressive way to clearly model the behaviour of software. The process of composing small, deterministic functions to create larger software components and functionality produces software that is easier to organise, understand, debug, extend, test, and maintain.

Pure Function and Side Effect

In functional programming, we will be building abstractions with functions. Functions serve as black-box abstraction. In this style we prefer pure functions. A pure function is one which given the same input, always returns same output and has no side effects. Side effect is application state change that is observable outside the called function other than its return value. Some of the ways a computation can introduce side effect are

  • Modifying any external variable or object property (e.g., a global variable, or a variable in the parent function scope chain)
  • Logging to the console
  • Writing to the screen
  • Writing to the network
  • Inserting a record into a database
  • Querying the DOM
  • Triggering any external process
  • Calling any other functions with side-effects

Side effects are mostly avoided in functional programming, only controlled side effect allowed, we manage state and component rendering in separate, loosely coupled modules.

For example, the below simple function to compute square root of a given number is pure as it gives same output no matter how many times it has been called with an input, it is devoid of any side effect and all computation is dependent on input value and return value.

/*
    Algorithm:
    sqrt(2)
    Guess       Quotient (Number / Guess)       Average ((Quotient + Guess) / 2)
    1           2/1 = 2                         (2 + 1)/2 = 1.5
    1.5         2/1.5 = 1.3333                  (1.3333 + 1.5) / 2 = 1.4167
    1.4167      2/1.4167 = 1.4118               (1.4118 + 1.4167) / 2 = 1.4142
    1.4142      ...                             ...
*/

function sqrt(x) {
    function average(a, b) {
        return (a + b) / 2.0;
    }

    function square(x) {
        return x * x;
    }

    function abs(x) {

        return x > 0 ? x : -x;

    }

    function sqrt_iter(guess) {
       return good_enough(guess) ?
          guess: 
          sqrt_iter(improve(guess));
    }

    function improve(guess) {
        return average(guess, x / guess);
    }

    function good_enough(guess) {
        return abs(square(guess) - x) < 0.001;
    }
    return sqrt_iter(1);
}

sqrt(2)

On the other hand, consider the below function, checkPrice , it is not pure as it depends on the value minimum which is other than its input.

let minimum = 21;

const checkPrice = price => price >= minimum;

One of the important thing to notice that functional style of programming is amenable to equational reasoning. That is, we can replace the the algorithm to compute square root with an equivalent algorithm and since it is a pure function, rest of the program is not affected. Thus it is a black box abstraction.

Another hallmark of this type of programming is avoiding mutation through assignment. Every mutation creates a new value and hence assignment statement is not needed at all.In fact, the seminal book on computing “Structure and Interpretation of Computer Programs” does not use assignment at all!

Name Isolation Techniques

The parameter names passed to a function are unique to that function. The names of the parameters and local names declared inside a function are called free variables. Names of the functions declared inside a function are unique and isolated to that function. The functions sqrt_iter, improve, good_enough etc in the above example are such inner functions. Parameters to the outer function are made available in inner functions. This is called Lexical Scoping .The parameter xof available in all inner functions. Lexical scoping dictates that a name is looked up in its enclosing functions till one is found. A function can also access a variable value outside it but in the lexical scope, even after the outer function has completed execution. This is called a Closure and is a powerful technique of encapsulation employed in functional programming.

Thus a function effectively forms a module, there by achieving information hiding and name space.

Recursion and Iterative Process

Functional style uses recursion instead of explicit looping constructs. We can model recursion to achieve an iterative process for efficiency and tail call optimisation (that is, if the recursive call is at the last position i.e, tail position, stack frame can be reused) makes recursion optimal. For example, below function to compute factorial of a number is not optimal. It results in stack overflow for large value of n.

function factorial(n) {
    return n == 0 ? 1 : n * factorial(n - 1);
}

On the other hand this recursive version where recursion happens at tail position is optimal and is equivalent to a loop.

function factorial(n) {
    return fact_iter(1, 1, n);
}
function fact_iter(product, counter, max_count) {
    return counter > max_count
           ? product
           : fact_iter(counter * product,
                       counter + 1,
                       max_count);
}

Similarly, every recursive function can be recast to produce an iterative computing.

Higher Order Functions

Functions that manipulate functions – can accept functions as argument or return function as value are called Higher Order Function. Higher order functions serve as a powerful abstraction mechanism. Higher order functions are possible because in functional languages, functions are first class. By first-class, we mean functions are like any other values :

  • They may be referred to using names
  • They may be passed as arguments to functions
  • They may be returned as results of functions
  • They may be included in the data structures

In the below example, makeAdder is a higher order function as it returns a function as value.

function makeAdder(x) {
    return function(y) {
        return x + y;
    };
}

let plus5 = makeAdder(5);
console.log(plus5(10));
let plus10 = makeAdder(10);
console.log(plus10(10));

In the below example, Array.prototype.map is a higher order function as it takes a function as argument.

const cars = [
  {
    make: 'Maruti',
    model: 'Alto',
    year: 2010,
  },
  {
    make: 'Hyudai',
    model: 'i10',
    year: 2018,
  },
];
const makes = cars.map(car=>car.make);

Function Composition

Given two functions f and g which take x as argument, the composition of two functions, f ⊙ g , is defined as g(f(x)) , for functions f, g and h as f ⊙ g ⊙ h as h(g(f(x))) and so on. We can define a variadic function compose in Javascript which performs function composition as described.

const compose = (...fns) => x => fns.reduceRight((y, f) => f(y), x);

Let us define a few functions:

const inc = x => x + 1;
const double = x => x + x;
const square = x => x * x;

We can define a composition as

let composition = compose(square, double, inc);
console.log(composition(5));

which is equivalent to the below function call, but more readable and easy to reason about.

console.log(square(double(inc(5))));

Function composition holds good the mathematical property of associativity, that is

compose(f, compose(g, h)) === compose(compose(f, g), h);

If you notice carefully, function composition takes functions with only one argument. If there are functions with more than one arguments in the composition, we can convert them to one argument functions using a technique known as currying . The concept is simple: You can call a function with fewer arguments than it expects. It returns a function that takes the remaining arguments. When you supply the last argument, the actual computation takes place.

const curry = (
    f, arr = []
  ) => (...args) => (
    a => a.length === f.length ?
      f(...a) :
      curry(f, a)
  )([...arr, ...args]);

function sum(a, b, c) {
    return a + b + c;
}
console.log(sum(2, 3, 4));

let curriedSum = curry(sum);

console.log(curriedSum(2)(3)(4));

Declarative vs Imperative

Declarative programming is way of computing by declaring what something is instead of spelling out how you get it. In imperative programming, we spell out how computation is done in explicit steps via detailed control flow statements. Declarative programs abstract the flow control process (the how gets abstracted away), and instead spend lines of code describing the data flow: that is: what to do. Functional programming, which prefers expressions over statements, uses declarative programming.

Below example of quicksort implementation in Haskell is an epitome of declarative programming

qsort :: (Ord a) => [a] -> [a]
qsort [] = []
qsort (x: xs) = qsort smaller ++ [x] ++ qsort larger
                where
                    smaller = [a | a <- xs, a<=x]
                    larger = [b | b <- xs, b > x]

Consider the below code snippet:

const cars = [
  {
    make: 'Maruti',
    model: 'Alto',
    year: 2010,
    units: 150  
  },
  {
    make: 'Hyudai',
    model: 'i10',
    year: 2018,
    units: 300  
  },
  {
    make: 'Hyudai',
    model: 'i20',
    year: 2015,
    units: 150
  },
  {
    make: 'Maruti',
    model: 'Dezire',
    year: 2012,
    units: 200
  },
];

// Declarative, functional
let totalUnits = cars
    .filter(car=>car.year >= 2015)
    .map(car=>car.units)
    .reduce((acc, cur) => acc + cur, 0)

// Imperative
let totalUnits = 0
for (let car of cars) {
    if (car.year >= 2015) {
        totalUnits += car.units
    }
}

In imperative style, one has to follow along the control flow to deduce what is happening in the code. The operations does not stand out. This becomes problematic in large code bases to reason about the code.On the other hand, in declarative style if one knows the meaning of higher order functions map reduce and filter , the meaning of the program stands out and it becomes that much easier to reason about the code.

Functional Programming is not new

In 1930s, two mathematicians Alan Turing and Alonzo Church produced two different, but equivalent universal models of computation. Both models could compute any function computable by a Turing machine, meaning, given input n, there is a Turing machine that eventually halts and returns f(n). A Turing machine is a mathematical model of computation that defines an abstract machine, which manipulates symbols on a strip of tape according to a table of rules. The computation model invented by Church, which is based on function application is called Lambda Calculus. Lambda calculus is the mathematical underpinnings behind functional programming.

Lambda calculus was hugely influential on software design, and prior to about 1980, many very influential icons of computer science were building software using function composition. The programming language Lisp was created in 1958 by John McCarthy, and was heavily influenced by lambda calculus. Today, Lisp is the second-oldest language that’s still in popular use. However, pure functional languages were notorious for being inefficient and hence efficient imperative languages like C grew in popularity. Functional Programming was largely relegated to academic world. But, with the advent of modern computing hardware making functional programs fast enough and functional programming’s ability to tame the complexity of software system caused by shared mutable state, a new interest has been rekindled in FP. Rise of the popularity of Javascript, which is a functional language at its core, has been another reason for the raise of popularity of functional programming.

Indeed, functional programming has become mainstream and it is an interesting time in the area of programming paradigms.

References

  1. Mostly Adequate Guide to Functional Programming – https://mostly-adequate.gitbooks.io/mostly-adequate-guide/content/
  2. Structure and Interpretation of Computer Programs — JavaScript Adaptation – https://source-academy.github.io/sicp/
  3. Composing Software: An Exploration of Functional Programming and Object Composition in JavaScript by Eric Elliott
Read More