HoloLens Beginner’s Guide

Title Page

HoloLens Beginner’s Guide

Join the AR revolution with HoloLens

Jason M. Odom

BIRMINGHAM - MUMBAI

Copyright

HoloLens Beginner’s Guide

Copyright © 2017 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: April 2017

Production reference: 1260417

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham 

B3 2PB, UK.

ISBN 978-1-78646-472-9

www.packtpub.com

Credits

About the Author

Jason Odom is a game developer with nearly 20 years of experience working on titles for companies such as Activision and Take 2 Interactive, along with years of game development teaching and mentoring. He is the founder of Thought Experiment, a start-up mixed reality studio, and he is currently working on HoloQuest and CrossTrainMR properties for HoloLens. When he is not making software and applications, he is a loud and proud mixed reality evangelist with a highly active YouTube channel, and a prolific technology blogger for NextReality. You can connect to him on the following:

h t t p s ://w w w . y o u t u b e . c o m /p l a y l i s t ?l i s t =P L E j p 5C b k s q k 4x - 3r O _ 6G 8I - k 5_ v O E X 25W

h t t p s ://g i t h u b . c o m /S u b e r e 23?t a b =r e p o s i t o r i e s

h t t p ://w w w . t h o u g h t e x p e r i m e n t s t u d i o s . c o m /

http://next.reality.news/

I would like to thank my family and friends for being patient with me as I sat locked away from the world working on this project. I would also like to send out a big Thank you to James Ashley for his subtle yet meaningful hints at how to make it through your first book, as well as Jesse McCulloch and the HoloDevelopers – Slack Community for being a constant source of great HoloLens information.

About the Reviewer

Vangos Pterneas is helping innovative companies increase their revenue using motion technology and virtual reality. He is an expert in Kinect, HoloLens, Oculus Rift, and HTC Vive.

Microsoft has awarded him with the title of Most Valuable Professional for his technical contributions to the open source community. Vangos runs LightBuzz Inc, collaborating with clients from all over the world. He’s also the author of the book Getting Started with HTML5 WebSocket Programming, by Packt, and The Dark Art of Freelancing.

www.PacktPub.com

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at service@packtpub.com for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

h t t p s ://w w w . p a c k t p u b . c o m /m a p t

Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Customer Feedback

Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at Amazon Book URL.

If you'd like to join our team of regular reviewers, you can e-mail us at customerreviews@packtpub.com. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!

Customer Feedback

Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at h t t p s ://w w w . a m a z o n . c o m /H o l o L e n s - B e g i n n e r s - G u i d e - J a s o n - O d o m - e b o o k /d p /1786464721.

If you'd like to join our team of regular reviewers, you can e-mail us at customerreviews@packtpub.com. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!

Dedication

For Aaron,

"Next door the TV’s flashing blue frames on the wall.

It’s a comedy of errors you see.

It’s about taking a fall."

Table of Contents

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

Welcome to the New World

What exactly is the HoloLens?

What you need to develop for the HoloLens

Unity

Visual Studio community

HoloToolkit - Unity

HoloLens emulator

Visual Studio tools for Unity

Other useful tools

Installing the software

Installing Visual Studio

Unity HoloLens technical preview

Visual Studio 2015 tools for Unity

HoloLens emulator

HoloToolkit-Unity

Unity3d quick overview

The main view

Summary

HoloWorld

Hello World!, as its customary

The layout of a HoloLens project

Unity directory structure

Saving the scene

Objects and components

Objects

Components

First project

Our cubism period

Adding a little color

Material plane

Creating the text

Physics simulations

Building the project in Unity

Build settings

Player settings

Compiling our project

Setting Up Visual Studio

Using the HoloLens emulator

Compilation time

Summary

I am in Control

Prefabs

Let's save the HelloWorld

Parent/Child relationship

Creating prefabs

Installing the HoloToolkit

User Input overview

InputManager prefab

New Camera prefab updated HoloToolkit

Cursor

Gaze

Gesture

Voice

Animation

A quick history

Humble beginnings

Time to move!

It's easier than it seems

Mecanim

The Mecanim window

Make them interactive

Here is the Game Plan

Our first script

Materials, textures, and shaders

Materials

Search For It

Adding the details

Changing the color

OnFocusEvent component

Speech Input Source

Time to build and test

Summary

User-Friendly Interface

Reloading

Unity C# scripting

What is C#?

A quick example

Where to begin

The breakdown of a Unity C# class

Variables

Access modifiers

Class methods

Understanding scope

A lot to take in

User interface design

General user interface best practices

Something new

Microsoft HoloLens best practices

The Unity camera and HoloLens

Unity UI

Overview

Interactivity

HoloLens UI

A visible canvas

Now it's time to build again

Any object can be an interface object

Summary

Now That Is How It Should Sound

Audio overview

Here is what it is all about

Unity audio components

Audio Listener

Audio Source

Spatial sound and HoloLens

Spatial sound

Setting up Audio Sources

Building a soundscape

Finding sounds

Importing our sounds into Unity

Audio Source options

Using our sounds

Let's listen to our work

Summary

Not So Blank Spaces

Spatial Mapping

Let's get into this

Importing the essentials

Spatial Mapping in practice

Spatial Mapping Observer

Spatial Mapping Manager

Object Surface Observer

Our new Skeeball machine

Application management

Design patterns

There can be only one

Creating our application manager

The ApplicationManager code

Attaching our application manager and wiring it up

So let it be written... so let it be done...

The home stretch

Code matters

Let us enjoy our hard work

Summary

The Tools of the Trade

Persistence

Why are persistent holograms necessary?

Coordinate systems

World Anchors

World Anchor system

Some quick upkeep

World Anchor code

ApplicationManager changes

A few more quick changes

Seeking help

The in-game debug window

The prefab

Let's wire this sucker up

In-game menu

Creating the menu items

Setting up to code

Time to code

Summary

Share What You Have Got

Networking

What is a network?

Making the Skeeball game a shared experience

Quick setup

A very simple first step

Too much information

Spawning our Skeeball machine

It's not really backtracking if it was the plan

Application manager

Enums

Updating the Update() method

Filling in the blanks

The payoff

Summary

Putting It All Together

Our main Prefab

The organization system

The reorganization

Importing the RampColliders

When bounding boxes collide

Who wants to score?

Think globally act locally

Finishing the Prefab

Ball Spawn Point

SkeeSpawner

Speed indicator

GameBall Prefab

Tags and Layers

ThrowBall.cs

Finishing touches

Updating Placeskee ballMachine

ApplicationManager

Action makes sound

Some quick house cleaning

Replacing the main Prefab

Summary

Fixing Problems

The Application Manager

Seeing it all at once

What does our Application Manager do?

Debugging

Bring the black flag

Inspector

Unity Property Attributes

Unity Inspector Debug Mode

Debug.Log/Console window

Visual Studio Debugger

Profiling and fixing performance

HoloLens Developers Portal

HoloToolkit FPS prefab

Unity Profiler

Visual Studio performance profiler

Unity Frame Debugger

Visual Studio Graphics Debugger

Simplygon

Summary

Preface

Microsoft HoloLens is on par with a modern-day wonder. It is the world’s first untethered, head mounted computer capable of projecting holograms into the world around us. This new technology enables forms of interaction that we have not even been able to imagine yet. New ideas are coming to the surface every day.

As this new device and its conceptual successors are further iterated upon, becoming more powerful and smaller, we will see a shift away from the 2D displays that we have had in place for the last 90 years. Even the pocket computers we use daily now will likely fall away into shared experiences that are projected through glasses, and maybe even one day contact lenses.

This book is here to get those interested in being on the front end of the coming wave up to speed with the development process for Windows mixed reality (formerly Windows Holographic), the ecosystem that Microsoft has created in parallel with Windows 10 for their new class of devices.

What this book covers

Chapter 1, Welcome to the New World, will introduce Microsoft HoloLens and its capabilities, as well as introduce and install the tools we will use to develop software for the device.

Chapter 2, HoloWorld, outlines how to build our first basic application. It will include the entire process from start to running on HoloLens.

Chapter 3, I Am in Control, covers the pillars of HoloLens input, Gaze Gesture, and Voice. We will also get into some of the elements of Unity development, such as Prefabs.

Chapter 4, User-Friendly Interface, takes you through C# scripting at a high, and teaches you to use what you have learned to build some interactive user interface elements. We then get to see these elements at work.

Chapter 5, Now That Is How It Should Sound, teaches you to take the knowledge accrued in the earlier chapters and begin your book project. We then learn about sound: AudioSources, AudioListener, and Spatialized Audio.

Chapter 6, Not So Blank Spaces, covers one of the major elements that help HoloLens create the magic that it is capable of--spatial mapping. We then use the spatial map in the context of our project so that we can learn how to move objects around our spatial map.

Chapter 7, The Tools of the Trade, focuses on the World Anchor, World Anchor Manager, and World Anchor Store. These important elements of persistence are a must-know. We then create a simple in-game debug output to help us track down problems.

Chapter 8, Share What You Have Got, explains the Holographic Sharing service that Microsoft provides as part of the HoloToolkit. This service allows us to create multiuser experiences for HoloLens.

Chapter 9, Putting It All Together, takes some of everything we have learned and brings it together to help us round out our project and make a playable game.

Chapter 10, Fixing Problems, showcases what debugging and profiling options exist within the confines of our tools, Unity and Visual Studio, and there are many.

What you need for this book

You require either a Windows 10, compatible PC and a HoloLens, or a PC with Windows 10 Pro and the HoloLens Emulator.

Who this book is for

If you are a developer new to Windows Universal development platform and want to get started with HoloLens development, this is the book for you. No prior experience of C# programming or of the .NET framework is needed to get started with this book.

Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: Run vs_community. exe and complete the installation process.

A block of code is set as follows:

New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: In the Camera component, click on the Clear Flags and select Skybox drop-down menu

Warnings or important notes appear in a box like this.

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail feedback@packtpub.com, and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for this book from your account at h t t p ://w w w . p a c k t p u b . c o m. If you purchased this book elsewhere, you can visit h t t p ://w w w . p a c k t p u b . c o m /s u p p o r t and register to have the files e-mailed directly to you.

You can download the code files by following these steps:

Log in or register to our website using your e-mail address and password.

Hover the mouse pointer on the SUPPORT