SharpGIS

#GIS from a .NET developer's perspective

Adding a Gaze Cursor to your HoloLens App

Adding a gace cursor to your app is important to give the user feedback what you’re looking at and whether you can interact with it.

First make sure you installed the HoloToolkit into your project.

Add new empty GameObject and rename it to Managers

Select the “Managers” object and in the Inspector click “Add Component” and add the “Gaze Manager” script.

image

In the added component’s “Raycast Layer Mast” dropdown, unselect “TransparentFX”

image

From HoloToolkit\Prefabs\Input\ add “Cursor” object to Managers object.

image

Save the scene, build the app and deploy. You now have a cursor at the center of your view following holograms and the spatially mapped mesh.

Rendering the Spatial Mapping Mesh

A lot of the HoloLens apps will sometimes render the mesh it scans to show you which surfaces it has detected – it can give a really cool effect to understand the play space.

Also if you’re using the Emulator, you won’t actually be able to see the virtual room.you’re placing holograms in, so being able to render the spatially mapped mesh can be useful for building apps if you’re not among the lucky few who has an actual HoloLens yet.

So let’s use the spatial mapping mesh we get from the HoloLens sensors and render it inside the application.

First make sure you installed the HoloToolkit into your project.

From HoloToolkit\Prefabs\SpatialMapping drag the SpatialMapping prefab into the root of your Hierarchy.

image

Select the added “SpatialMapping” object, and ensure “Draw Visual Meshes” is checked on. The default material is the “Wireframe”. Feel free to experiment with other materials.

image

Build your app and deploy it. You should now see the mesh rendered on top of walls, floors etc.

Untitled

Using HoloLens’ Spatial Mapping to occlude objects

Spatial Mapping is probably one of the most important aspects of the HoloLens. It’s what it uses to know where it is in a room and how you can make holograms interact with the real world. It essentially scans your surroundings and builds a 3D model. Here’s an example of some of the mesh it has generated for a house:

image

Now we can use this to avoid being able to see our holograms “through walls”. If we bring up the sample we build in the first blogpost, here’s what happens when the holograms goes behind a wall and ruins the illusion:

Nonoccluded2

 

First enable SpatialPerception capability. In Unity go to Edit –> Project Settings –> Player, click the “Windows Store” tab, and check off the capability:

image

Note: If you already have generated a Visual Studio project, this checkbox doesn’t actually “work”. You can do two things: Either delete the generated app and regenerate it again, or go into Visual Studio, and manually edit the “Package.appxmanifest” in a text editor (the manifest designer doesn’t have this capability listed yet) by adding the following line to the manifest:

    <uap2:Capability Name="spatialPerception" />

image

Next we’ll add the SpatialMapping component from the HoloToolkit. Make sure you’ve followed the steps from the previous blogpost to add the HoloToolkit to your project.

Select the object or collection you want to have spatial mapping occlude. In this case I’ll select the HologramCollection, so all children of this collection will get occluded. Click “Add Component” in the inspector and add the “Spatial Mapping Renderer”.

 

 

dimage

image

That’s it! Now save the scene, build your project again, and redeploy from Visual Studio.

Occluded2

The clipping is a little bit off – this is because the mixed reality capture webcam is “off” compared to what you really see in the HoloLens and the mesh scan wasn’t too good on this corner.

Installing HoloLens HoloToolkit into your Unity Project

Make sure you first read “Creating your very first holographic app in Unity” for setting up your hololens project.

The hololens team has created a useful “HoloToolkit” for use with Unity. It provides stuff like Spatial Mapping, client/service for sharing holograms among multiple users, cursors, gesture handling, spatial sound etc.

It’s pretty simple to install in to your project, so here’s the simple step-by-step:

  1. Go to https://github.com/microsoft/HoloToolkit-Unity and click “Download Zip” to download the toolkit.
  2. Right-click the downloaded zip, select properties, Check the “Unblock” checkbox and click OK.
  3. Unzip the folder “HoloToolkit-Unity-master\Assets” into your Assets folder in your Unity project.

Done!

You should now see all the HoloToolkit in your Project view (Unity doesn’t even need to restart, but will auto-detect the new files and import them).

image

I’ll be blogging about using the Toolkit in upcoming blogposts as I figure out how to use the pieces.

Creating your very first holographic app in Unity

Most of the tutorials at the Holographic Academy starts out with a starter-project with a bunch of stuff already set up for you. If you’re new to Unity and/or holographic development, I found that a little bit “cheating” and wanted to know how to do things “from scratch”, to property understand it. I thought I would share my findings in a set of blogposts – they wiill serve as notes for myself, but figured it might be useful for others as well. If something is wrong or you know a better way, please comment in the comment section.

I have all the steps recording in a video at the bottom, but for those who like to read and understand the steps, I’ll go with that first. So lets get started.

First launch Unity and create a new project. Name it whatever you’d like.

After launch, you’ll see in the Hierarchy view a “Main Camera” and a “Directional Light” object.

First we’ll configure the camera for Unity. Keep the name “Main Camera”. From my understanding this is what automatically becomes the camera controlled by your HoloLens. But we have to configure it to be placed at the center of the world.

Select the camera in the hierarchy and In the inspector set the position and rotation to all zeros:

image

Next we need to set the camera to render “nothing” as black. By default it renders blue skies, but since “black” renders as transparent and we want to see the real world around our holograms, we set “Clear Flags” to “Solid Color” and “Background” to “Black”:

image

It is also recommended to set the Near Clipping Plane to 0.85 m. This prevents users from getting “too close” to holograms and get all cross eyed from it. It can be very uncomfortable for people, but feel free to set it to 0.1, to get really up close to your holograms.

 

Next we need to configure the app for Virtual Reality. Go to Edit –> Project Settings –> Player. Click the green “Store Logo” tag, expand “Options” and check off “Virtual Reality Supported”. You should then see “Windows Holographic” listed under the “Virtual Reality SDKs” list.

image

Lastly, we configure the app to run with the fastest rendering possible. To go Edit –> Project Settings –> Quality. Under the green “Store Logo” tag to the left of “Default” click the little black dropdown triangle (highlighted with the red arrow blow) and select “Fastest”. You should see “Fastest” now be green in the first line under the store logo.

image

 

Now at this point we’ve done all the steps for configuring your Holographic app. To recap:

  1. Place camera at 0,0,0, name it “Main Camera” and set the background to solid color black.
  2. Enable the app for Virtual Realtiy
  3. Set quality settings to “Fastest”

At this point we can save the app, and build a Windows Store Visual Studio project to deploy to the HoloLens, but since we haven’t added anything to the scene, it would be a boring app, so let’s do that first before creating the visual studio project.

Right-click inside the Hierarchy Panel, and select “Create Empty”. You should see a “GameObject” be created. Right-click it at click “Rename”, and name it something like “HologramCollection”. Double-check the transform settings and sure position is placed at 0,0,0.

Next select the HologramCollection object, and right-click it. Select “3D Object –. Cube”. A new cube should be created under the collection, and the Hierarchy panel should look like this:

image

Select the Cube, and in the position, set it to 0, –0.5, 2. This means “place it 0.5 meters below the camera, and two meters in front. When the app starts, this is where the hologram will be placed relative to the HoloLens. The cube is 1x1x1 meter. That’s a little bit large for a Hologram, so set the scale to 0.25 for all 3 values, to make it .25m on each side.

image

That’s it for setting up our scene. Next lets create and build it. First go to “File –> Save Scene as…” and give it a name like “Main Scene”.

Next go to “File –> Build settings…”. For platform select “Windows Store”. SDK to “Windows 10”, UWP Build Type to “D3D” and click “Build”.

Lastly click “Add open scenes” and ensure the scene you just saved got added to the list at the top.

image

You’ll be asked for a folder to create it in. Create a folder in your project called “App”, and select the folder. The the project is done being created (it takes a while the first time), go into the app folder and open the Visual Studio solution.

Next, set the build architecture to x86, and select either the holographic emulator, or if you have a device select the “Remote Machine” and enter the IP (Tip: From within the hololens open the start menu and ask Cortana “What is my IP”), or plug the device in with USB and select “Device”.

Hiit F5 and start your first holographic app!

All the steps are also shown in the video below. At the end of the video you’ll see the app running inside the HoloLens.

 

Enjoy!

The beginnings of an AllJoyn based Home Automation Controller

I’ve been working on building my own home automation controller to make my home smarter. I decided to build this around AllJoyn so I can avoid getting any type of device-protocol lock-in, but can abstract everything with AllJoyn.

I’m currently at a stage where I have several, lights, switches, temperature, humidity and door/window sensors, as well as a way to directly read my house’s SmartMeter to get real-time power consumption, all exposed via AllJoyn.

Since I want to build a controller; that can pick up any AllJoyn device at runtime, without the need to have a preconfigured list of supported device types, I needed a way to discover any device without any prior knowledge. Luckily there’s a great library with full source from Microsoft that does just this, and I wrapped it all up into a little NuGet package, and wrote an article on how to use it on Hackster here: https://www.hackster.io/dotMorten/discovering-and-interacting-with-any-alljoyn-device-0dbd86

 

I’m excited to be going to CES2016 for a few days this week, and will be meeting with the AllSeen Alliance who has a big presence there, and get some inspiration and hopefully get some questions answered, before moving forward with my controller.

Here’s a few photos of it all running on a Raspberry PI with a little 5” display.

WP_20151026_11_34_30_Pro

Home screen

 

WP_20151222_12_58_27_Rich_LI

Tracking power consumption over time.

 

image

Automation Rule Designer

 

Just some of the AllJoyn devices in my house…

First release of WindowsStateTriggers

With the release of the Windows 10 Universal SDK today I’ve updated the Github Repository for the WindowsStateTriggers for the final SDK. Go grab the source code today!

During the beta period I’ve received several great fixes and additions from the community. Thank you all who contributed!

The Nuget package is available here: https://www.nuget.org/packages/WindowsStateTriggers

Get full source here: https://github.com/dotMorten/WindowsStateTriggers

 

See the WIKI or Test App for usage some usage examples.

Create a Universal 3D App in a matter of minutes

We just released version 10.2.6 of 'ArcGIS Runtime for .NET SDK', which now supports 3D and KML files on both Windows Store 8.1, Windows Phone 8.1 and WPF. This means you can now quickly build 3D applications that supports all these platforms. We also released the package on NuGet for your convenience.

As a demonstration how quick it is to create a Windows Phone and Windows Store 3D app, here's a little video creating a universal 3D app from scratch in a couple of minutes:

Displaying a backbutton on your app title bar

Disclaimer: This article is written based on Windows 10 Tech Preview – Build 10122. Things might change completely in the future.

Some of the Windows 10 apps have a back button at the top of the app bar. Here’s an example in the Settings Control Panel:

image

We can add this to our own apps but using the SystemNavigationManager’s AppViewBackButtonVisibility.

Here’s how that will look like:

SystemNavigationManager.GetForCurrentView().AppViewBackButtonVisibility = AppViewBackButtonVisibility.Visible;

When you run this code you’ll now also see the exact same back button in your Windows UWP Desktop app! No need to waste precious screen real-estate when there’s room on the top bar.

Of course for Windows Phone this isn’t really needed (although nothing seems to happen if you call this API), call it anyway, just in case you're running on a device that can show the backbutton - it seems like it just doesn't show on devices with hardware:


   //Show UI back button - do it on each page navigation
   if (Frame.CanGoBack)
      SystemNavigationManager.GetForCurrentView().AppViewBackButtonVisibility = AppViewBackButtonVisibility.Visible;
   else
      SystemNavigationManager.GetForCurrentView().AppViewBackButtonVisibility = AppViewBackButtonVisibility.Collapsed;


   //Hook up back buttons - Do this just once - ie on app launched
   SystemNavigationManager.GetForCurrentView().BackRequested += (s, e) =>
   {
       if (Frame.CanGoBack)
           Frame.GoBack();
   };
   if (Windows.Foundation.Metadata.ApiInformation.IsTypePresent("Windows.Phone.UI.Input.HardwareButtons"))
   {
       //Also use hardware back button
       Windows.Phone.UI.Input.HardwareButtons.BackPressed += (s, e) =>
       {
           if (Frame.CanGoBack)
           {
               e.Handled = true;
               Frame.GoBack();
           }
       };
   }

Now you get back button support in both desktop and phone apps when you can navigate back!

Unfortunately the BackRequested and BackPressed event arguments are different, so you can’t reuse the same event handler for both. That’s quite a shame – I hope Microsoft will be cleaning that up soon.