A Comprehensive Overview on VR(Virtual Reality)

Posted on by spiderlili Posted in AR/VR/MR, programming, Resources, Technical ArtTagged ,

Here’s an overview of the various platforms, tools and products as an introductory guide to the world of VR.

What is VR?

VR(Virtual Reality) replaces the real world around you with completely computer generated simulation of a 3D environment. VR is often experienced using a headset with a screen inside, which can track the movement of the viewer’s head to simulate an immersive 360-degree environment. Special equipment such as gloves fitted with sensors and hand-held controllers provide additional ways to interact with the virtual world.

The goal of VR is to create simulated experiences that are indistinguishable to all of our senses from reality, giving us the superpowers to recreate experiences that we otherwise would not be able to in the physical world. This could be visiting Mars in the year 3020 or traveling inside the human body to learn about white and red blood cells.

A full VR headset combines HMD and head tracking. Examples of popular VR headsets include Samsung Gear VR, Oculus Rift, HTC Vive, Merge, Google Cardboard and Daydream.

Further Reading: What is VR

vr20montage20abstract20-20togo28129

A Brief History of VR

Early 1900s: VR was referenced in sci fi as an engaging experience for all the senses

1962: Sensorama was the first VR prototype to be released. This is a mechanical device that attempts to engage the user’s sense of sight, smell, sound and touch.

1982: Atari created a VR research lab.

1990s: The first VR boom with the first consumer headsets primarily for arcades

2010: Oculus Rift prototype marks the start of modern VR.

2014: The cardboard platform was developed by Google engineers David Coz and Damien Henry. It was introduced to the world at Google I/O in 2014.

2016: HTC Vive release for home entertainment.

Further Reading: VR’s long, weird history

snesorama_2

Essential VR Terminology

HMD(Head Mounted Display)

A headset with 2 screens(one for each eye). Not all HMDs track the head’s movement.

Head tracking

Tracking the rotation or/and position of the user’s head.

Tactile feedback

Feedback which you can feel such as vibration in the controller. this helps to immerse the player by making it seem like they can feel what is happening in the VR world.

Field of View

The extent of the observable environment at any given time is an important aspects of virtual reality. The wider the field of view, the more present the user is likely to feel in the experience. The human eye can perceive a field of view of up to 180 degrees without even moving the eyeballs. Even more, the eyes can see up to a 270-degree field of view, given that the eyes are fully rotated.

Further Reading: VR headset comparison chart

Photospheres / 360 Panoramas

A technique where an image shows the entire environment around a point in space. Users viewing a 360 panorama can look in any direction, either by dragging on the screen with their finger, or rotating a device that can track their motion (usually a VR headset or a mobile phone with a gyroscope).

360 Degree Videos

Videos that let you glimpse the action from all angles and directions — not just where the camera is pointing. 360 Degree video can be delivered via social media (Youtube, Facebook) without the need for a stereoscopic viewer as is required by VR (webVR is the exception).

It’s worth noting that360° video is a fixed perspective from the camera point of view, meaning you can not navigate around your environment unless the camera itself moves. With VR, you have more mobility to navigate and explore environments.

High quality 360 video can be very bandwidth intensive, only devices with a good network connection will be able to reproduce a reasonable 360 video to its user without extensive loading or reduced visual quality.

Further Reading: The Best Encoding Settings For Your 4k 360 3D VR Videos and Free Encoding Tool

Alpha Channel Video

A technique where a video has the set background removed so that the foreground elements (usually the actors) can appear on an arbitrary background later on. Visual effects techniques like green screen make it easy to remove the background from live video. In the film and VR industry, the background is often replaced with a dramatic, computer-generated scene (such as Pandora, an alien planet in Avatar).

Further Reading: The VR Glossary

Major VR Hardware Options

There are two major options for VR:

  1. Mobile VR
  2. Roomscale VR

Mobile VR

Advantages

  • 3 degrees of freedom(head rotation in any direction) – this involves stationary (seated or standing) experiences that measure the rotational movements along the X (roll), Y (pitch), and Z (yaw) axes.
  • Low cost
  • Untethered – no cables, no PC,  can move around freely and use it anywhere,

Disadvantages

  • Has limitations of mobile devices – lower graphics quality than PC.
  • No positional head tracking – have to rely entirely on rotation, this can have the possibility of causing motion sickness.
  • No guarantee of controllers – some headsets have controllers such as Google Daydream, Samsung Gear VR and Zapbox, but there’s no compatibility between the different types of controllers. Be prepared for the lack of consistency!
  • Challenges of mobile development: needs extra effort in initial project setup to make your project run on mobile devices. These challenges are different depending on whether you’re building for Android(requires SDK, JDK, NDK, check Development Build in Unity, set minimum API Level to 19) or iOS(need to register as an Apple Developer). Depending on the platform, extra setup may be needed to make your device recognisable by the headset(such as Samsung Gear VR).

Google Cardboard

Google Cardboard is a VR headset developed by Google made of cardboard material, with an magnet on the side(which was unstable and requires a magnetometer sensor) and two optical lenses. You have to insert your smartphone inside it. the google cardboard app splits the smartphone image display in two – one for your left eye and one for your right eye, into a stereoscopic 3D image which has a wide field of view.

As of 2017, Over 10 million cardboard units has been shipped, 1000 VR compatible apps has been released and 160 million downloads.

Google Cardboard V2 supports phones up to 6 inches and the magnets are removed. an input button was added to the top left side. instead of getting a magnetic sensor, google cardboard had a mechanism that touches the screen when the button is pressed.

81bktmewqwl-_sl1500_

Google Daydream

Google’s high-quality VR platform which is made of lightweight fabric rather than cardboard.  Daydream has a view controller which can be used to interact with the VR headset.

Google is working with partners like Qualcomm and Lenovo to develop Daydream headsets that are completely free of phones, PCs and wires.

41i9qwnsoql

Samsung Gear VR

Powered by Oculus, Samsung  Gear VR transforms your Galaxy smartphone into a portable VR device. Users can hang out with their friends using Oculus social features or record and watch their own 360° videos with Gear 360.

The Samsung Gear VR controller can be used to interact with the VR headset and to increase mobile immersion.

how-to-set-up-and-use-samung-gear-vr

Roomscale VR

Roomscale VR has three main options:

  1. Oculus Rift: Slightly less expensive
  2. HTC Vive(Steam VR): Slightly higher quality
  3. PSVR:  Less immersive than the Oculus or the Vive but it is cheaper and has the advantage of being the only console-based VR system so far(36 million unit install base of PS4s)

Advantages

  • 6 degrees of freedom – positional head tracking as well as rotational tracking. 6 Degrees of Freedom (6DoF) headsets allow for walkable experiences around a physical space. It measures translation across the X (surge // forward, backwards), Y (sway // left, right), and Z (heave // up, down) axes.
  • Low latency – when tethered to a powerful computer, the graphics display can be updated very quickly
  • Higher graphics quality
  • Tracked controllers with tactile feedback – both Oculus Rift and HTC Vive comes with controllers.

Disadvantages

  • More expensive
  • Requires a high end PC with a VR-Ready Graphics card
  • Difficult set up – a lot of wires and cables during the process, takes up more room

Further Reading: The Benefits and Drawbacks of the Different VR Hardware Options

 

Useful VR Tools and Frameworks

Further Reading: The Big List of Virtual Reality Production Resources

Useful Links: Optimising Performance in VR

Best Practices for Creating Assets for VR

  • It’s important to optimise art for comfort in VR. Use low-poly models and low rez textures or solid colours when possible. This helps maintain high frame rate and good performance on various headsets.
  • Anything that needs to be interacted with, read or observed with in detail should be between 2 – 3 meters from the player’s starting location because this is the most pleasant distance to focus on.
  • Add walls and visual barriers to help improve frame rate and performance, by reducing draw calls to distant objects.
  • Use physics to bring the environment to life and make it feel real to the player.
  • Use stationary large objects such as floors and walls to make the player feel stable and grounded, in order to prevent motion sickness. Avoid moving walls to make the player feel disoriented.
  • Proprioception is a great tool for VR developers. This refers to your awareness of your body and bodily positioning. It is the reason you can touch your nose with your eyes closed. Players are likely to remember where the objects are in the scene if they can remember them based on their body(.e.g.putting an essential game tool behind the player’s right shoulder even though they can’t see it)
  • Use World Space diegetic UI – place spatial UIs in the scene rather than on the screen.
  • Set up the model’s import scale factor properly so they are appropriately sized in the scene. Build everything according to your game engine’s scale.

Further Reading: Google’s Guide of Best Practices: Creating Art Assets for VR

Design Considerations for Integrating VR Audio

  • Create 3D sound sources in the environment(change spatial blend to 3D and use  linear volume rolloff).
  • Create moving audio using the animation timeline and animating the audio parameters to draw the player’s attention.
  • Use a spatializer plugin which allows you to spatialise the audio for a better VR experience in Edit > Project Settings > Audio, then check spatialize on the audio source.
  • Headphones: attach the audio listeners to the camera rig
  • Speakers: if the user has surround sound, attaching audio listeners to the head will cause the audio to seem to be moving when the user’s head moves even though the speakers are stationary. The better approach would be to place the audio listeners in the scene on an empty game object(which doesn’t update with the user’s head rotation).
  • Consider using Ambisonic Audio: a full sphere of audio capture, creating a fully immersive soundscape, particularly for use in 360 film. can be useful for capturing ambient environment sounds. Requires the use of special audio equipment and editing software.

Designing Immersive User Interactions

  • Engage with 360 degrees of movement which encourages user exploration in every direction.
  • Use visual and audio cues so the user understands where the things are
  • Making a comfortable space and comfortable movements for the player, where they can move around at their own pace and everything is positioned nicely inside the playable space(especially the things that they can pick up – they must be within reach).
  • Be aware of how tall the player should feel in the environment, make sure your experience accounts for both tall and short users. Everything should feel like a good natural height.
  • Using hand controls(if your device has hand tracking) – responsive feedback to the user and natural actions. Add hand collision detection to detect trigger collider and collectables. For example:void OnTriggerEnter(collider other){if(other.tag == “Hand”){transform.parent = other.transform; transform.position = other.transform.position; transform.rotation = other.transform.rotation; transform.Rotate(0, -90, -45);}}
  • Tracking the user’s gaze(primarily for mobile devices and hand free interactions) by adding a raycast from the camera to detect when the player is looking at something like an enemy. For example:if(Physics.Raycast(transform.position, transform.forward, out hit) && hit.collider.tag == “enemy”){Destroy(hit.collider.gameObject);}void OnDestroy(){Instantiate(Resources.Load(“pickupEffect”), transform.position, transform.rotation}