The layout of an ExtensionSDK

by Morten 28. January 2014 22:37

Extension SDKs are a very powerful way distribute your control libraries for use in Windows Store and Windows Phone apps.
This article will go through the layout of the extension sdk, and later take that knowledge to build an extension sdk from an already released app.

An ExtensionSDK essentially consists of 3 parts:

  • Files to use during design time
  • Files to deploy as content
  • Assemblies to use for reference

In addition there's a metadata file 'SDKManifest.xml' that describes the content.

The root layout then looks like the following:
    \EXTENSIONNAME\VERSION\DesignTime\
    \EXTENSIONNAME\VERSION\Redist\
    \EXTENSIONNAME\VERSION\References\
    \EXTENSIONNAME\VERSION\SDKManifest.xml
…where 'EXTENSIONAME' is the name of your extension, and VERSION is version number in the format "1.2.3.4".
   
For each of these groups you can control what gets deployed in debug and release or both. If you don't want to control whether you use debug or release, you will below these folders use the folder 'CommonConfiguration'. For debug specific configuration use 'Debug', and for release configuration use 'Retail'. In most case you will be using 'CommonConfiguration' though.
This means our folder structure now looks like this:
    \EXTENSIONNAME\VERSION\DesignTime\CommonConfiguration\
    \EXTENSIONNAME\VERSION\Redist\CommonConfiguration\
    \EXTENSIONNAME\VERSION\References\CommonConfiguration\

Next level down in the folders describe if files are related to AnyCPU, x86, x64 or ARM builds (the latter is very useful for C++ projects). For AnyCPU use 'neutral', meaning it doesn't matter. So use this for .NET Assemblies compiled for AnyCPU, image resources, winmd files etc. You will want to use the architecture specific folder if you deploy binaries that are architecture specific.

So what goes in what folders:

  • DesignTime: This is where you will put .Design assemblies if you have specific design time binaries for your assemblies, as well as Generic.xaml. You only need to deploy 'neutral' and/or 'x86' architectures, since VS runs in a 32bit process.
  • Redist: Images, shaders, Generic.xbf, videos etc, AND C++ binaries.
  • References: .NET DLLs, C++ WinMDs, xml doc.

Here's an example layout of an extension sdk that consists of two libraries: One C++ WinRT component (NativeLib) and a Managed library:
    \MyControlLib\1.0.0.0\SDKManifest.xml
    \MyControlLib\1.0.0.0\DesignTime\CommonConfiguration\neutral\ManagedLib\Themes\Generic.xaml
    \MyControlLib\1.0.0.0\DesignTime\CommonConfiguration\x86\ManagedLib.Design.dll
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\neutral\ManagedLib.pri
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\neutral\NativeLib.pri
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\neutral\ManagedLib\Icon.png
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\neutral\ManagedLib\Themes\Generic.xbf
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\neutral\NativeLib\shaders\PixelShader.cso
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\neutral\NativeLib\shaders\VertexShader.cso
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\ARM\NativeLib.dll
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\x86\NativeLib.dll
    \MyControlLib\1.0.0.0\Redist\CommonConfiguration\x64\NativeLib.dll
    \MyControlLib\1.0.0.0\References\CommonConfiguration\neutral\NativeLib.winmd
    \MyControlLib\1.0.0.0\References\CommonConfiguration\ARM\ManagedLib.dll
    \MyControlLib\1.0.0.0\References\CommonConfiguration\ARM\ManagedLib.xml
    \MyControlLib\1.0.0.0\References\CommonConfiguration\x64\ManagedLib.dll
    \MyControlLib\1.0.0.0\References\CommonConfiguration\x64\ManagedLib.xml
    \MyControlLib\1.0.0.0\References\CommonConfiguration\x86\ManagedLib.dll
    \MyControlLib\1.0.0.0\References\CommonConfiguration\x86\ManagedLib.xml

The SDKManifest could look like the following:

    <?xml version="1.0" encoding="utf-8" ?>
    <FileList
      xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
      xsi:noNamespaceSchemaLocation="SDKManifest.xsd"
      DisplayName="My Super Duper Control Library"
      ProductFamilyName="MyControlLib"
      Description="My Control Library"
      MinVSVersion="12.0"
      Identity="MyControlLib, Version=1.0.0.0"
      MinToolsVersion="12.0"
      AppliesTo="WindowsAppContainer + ( Managed )"
      SupportedArchitectures="x86;x64;ARM"
      DependsOn="Microsoft.VCLibs, version=12.0"
      SupportsMultipleVersions="Error">
        <File Reference="NativeLib.winmd" Implementation="NativeLib.dll" />
        <File Reference="ManagedLib.dll"/>
    </FileList>

   
Note that if you don't have native dependencies, this would change quite a lot. The full set of properties are pretty poorly documented today, so generally I download and install a wealth of extension sdks and look at them and see if they do similar things to me and then copy from that.

Building an Extension SDK from an installed app

So now that we know the layout of an extension sdk, let us apply that to 'reverse-engineering' an already deployed app into an extension sdk and use that to build our own app on top. Because Windows Store apps aren't fully encrypted, this means you can often take parts of an app that's separated out into libraries and build a new app from these libraries. This is something to consider when you build your app - if you are really good are separating your stuff into sub-libraries, you also make it easier for others to reuse your stuff. As an example let's download the Bing Maps Preview app and reverse it into an SDK and build our own 3D Map App.

When you installed the app, you will be able to access a folder with a name similar to the following with administrator rights:
"c:\Program Files\WindowsApps\Microsoft.Maps3DPreview_2.1.2326.2333_x64__8wekyb3d8bbwe\"

In here we'll find a lot of logic for the app, but the main one we are interested in is the "Bing.Maps" folder and the Bing.Maps dll+winmd. The folder is essentially the content and is image resources and shaders. The Bing Maps.dll and Winmd are C++ WinRT components. Since the dll is C++, the architecture will either be ARM, x86 or x64 depending on what PC you downloaded it on. In my case it's x64 so I should be able to build an extension sdk that will support 64 bit PCs from this alone. If I want to support more, I will have to install the app on a x86 or ARM PC and copy the dll from there as well (the other files are neutral and will be the same).

So let's first create the following folder : "Bing.Maps\1.0.0.0\".

Next, let's copy the "Bing.Maps" folder that has all the images and shaders into
    \Bing.Maps\1.0.0.0\Redist\CommonConfiguration\neutral\Bing.Maps\
Next copy the Bing.Maps.dll into (x64 if that's what you have, change/add ARM/x86 if your binary isn't x64)
    \Bing.Maps\1.0.0.0\Redist\CommonConfiguration\x64\Bing.Maps.dll
Lastly, copy the Bing.Maps.winmd into:
    \Bing.Maps\1.0.0.0\References\CommonConfiguration\neutral\Bing.Maps.winmd

Lastly we need to create a new file in \Bing.Maps\1.0.0.0\SDKManifest.xml to describe the SDK:

    <?xml version="1.0" encoding="utf-8" ?>
    <FileList
      xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance"
      xsi:noNamespaceSchemaLocation="SDKManifest.xsd"
      DisplayName="Bing Maps"
      ProductFamilyName="Bing.Maps"
      MinVSVersion="12.0"
      Identity="Bing.Maps, Version=1.0.0.0"
      MinToolsVersion="12.0"
      AppliesTo="WindowsAppContainer"
      SupportedArchitectures="x86;x64;ARM"
      DependsOn="Microsoft.VCLibs, version=12.0"
      SupportsMultipleVersions="Error">
        <File Reference="Bing.Maps.winmd" Implementation="Bing.Maps.dll" />
    </FileList>

Voila! We now have an ExtensionSDK. There's several ways you can 'install' this into Visual Studio. The simplest way is to copy the folder into your user folder under %USERPROFILE%\AppData\Local\Microsoft SDKs\<target platform>\v<platform version number>\ExtensionSDKs.
In this case %USERPROFILE%\AppData\Local\Microsoft SDKs\Windows\v8.1\ExtensionSDKs\

If you're building an installer you can also install it into
    %Program Files%\Microsoft SDKs\Windows\v8.1\ExtensionSDKs
Or specify a link to the location of the folder in a registry key:
    HKLM\Software\Microsoft\Microsoft SDKs\Windows\v8.1\ExtensionSDKs\Bing.Maps\1.0.0.0\

Lastly you can do this in your project file    by using the 'SDKReferenceDirectoryRoot' tag. Add the following right before the <Target Name="BeforeBuild"/> tag at the very bottom.

  <PropertyGroup>
    <SDKReferenceDirectoryRoot>c:\myfolder\my_sdks\;$(SDKReferenceDirectoryRoot)</SDKReferenceDirectoryRoot>
  </PropertyGroup>

Note that for the latter, the folder should point to a root extension sdk folder, meaning the SDK above must be located in a certain tree under this folder. In this case:
    c:\myfolder\my_sdks\Windows\v8.1\ExtensionSDKs\Bing.Maps

When you've done any of these install options, you can now get started building an app. Go to "Add References" and the Bing Maps entry should show up.

image

Now add the following XAML to your page:

        <bing:Map x:Name="map" xmlns:bing="using:Bing.Maps">
            <bing:Map.MapProjection>
                <bing:ThreeDimensionalMapProjection />
            </bing:Map.MapProjection>
        </bing:Map>

And in code-behind after "InitializeComponents":

            map.BaseLayers = Bing.Maps.BaseLayers.CreatePhotoRealisticOverlay();

Run the app and you should see a 3D globe!

image

Note: This Bing Maps SDK is not based on anything officially released but on a un-finished app. This is by all means a giant hack and only meant as an exercise to build an Extension SDK. Use this at your own risk and don’t attempt to publish any apps using it.

References:

Tags:

Windows Runtime | Windows Phone | XAML | Virtual Earth

Hacking the Silverlight Unit Tests to support returning Task

by Morten 21. December 2012 15:05

The Silverlight/Windows Phone unit test framework has always supported running asynchronous tests – a feature that until recently wasn’t there in WPF without jumping some really ugly (and flaky) hoops. Basically you can write a silverlight and windows phone unit test like this:

[TestClass]
public class TestClass1 : SilverlightTest
{
    [TestMethod]
    [Asynchronous] 
public void Test1() { DispatcherTimer timer = new DispatcherTimer() { Interval = TimeSpan.FromSeconds(2) }; timer.Tick += (a, b) => { timer.Stop(); base.TestComplete(); }; timer.Start(); } }

The problem with this code though is that this is only for Silverlight and Windows Phone. If you are cross-compiling for multiple platforms and want to run on WPF this wouldn’t work. It’s also not pretty that you have to inherit from SilverlightTest, remember to decorate the class with [Asynchronous] as well as calling TestComplete. Even worse, if you forget to stop the timer, it would CRASH the entire test run. The unit test framework is a little flaky when it comes to a task accidentally completing twice (instead of reporting it as an error, it crashes the entire test run and you’ll never get your daily test report…).

With Visual Studio 2012 and .NET 4.5 we can now simply return an object of type ‘Task’ and we would be good to go. This is awesome for testing your new async/await based stuff that returns task. So in WPF you would simply return your task object. As an example, let’s say we have the following really advanced computing task:

public static Task<int> Compute(int input)
{
    TaskCompletionSource<int> tcs = new TaskCompletionSource<int>();
    DispatcherTimer timer = new DispatcherTimer() { Interval = TimeSpan.FromSeconds(2) };
    timer.Tick += (a, b) =>
    {
        timer.Stop();
        if (input <= 0)
            tcs.SetException(new ArgumentOutOfRangeException("Number must be greater than 0"));
        else 
            tcs.SetResult(input);
    };
    timer.Start();
    return tcs.Task;
}

Now to test this in .NET 4.5 (including Windows Store Apps) you can simply write the following unit test:

[TestClass] 
public class TestClass1
{
[TestMethod] public async Task Test42() { var result = await Utility.Compute(42); Assert.AreEqual(result, 42); }
}

Nice and simple. However in Silverlight and Windows Phone you would have to write the following instead (I highlighted the extra or changed code required):

[TestClass]
public class TestClass1 : SilverlightTest
{
    [TestMethod]
    [Asynchronous]
    public async void Test42()
    {
        var result = await Utility.Compute(42);
        Assert.AreEqual(result, 42);
        base.TestComplete();
    }
}

Wouldn’t it be nice if the unit test I just wrote for WPF would work as is in Silverlight and on Windows Phone? Of course you could create a SilverlightTest class that has an empty TestComplete method, define an AsynchronousAttribute just for fun, and sprinkle a compiler conditional around the void/Task return type, but that just feels messy to me.

Fortunately the unit test framework for Silverlight is open source, so it’s possible to hack it in there. There are two main places you will need to change, which I will go through here. Note this is based on changeset #80285.

In the file “\Microsoft.Silverlight.Testing\UnitTesting\UnitTestMethodContainer.cs” we add the highlighted code to the method that detects if the Asynchronous attribute is on a method:

private bool SupportsWorkItemQueue()
{
    if (_testMethod != null)
    {
        if (_testMethod.Method.ReturnType != null && 
            _testMethod.Method.ReturnType == typeof(System.Threading.Tasks.Task) ||
            _testMethod.Method.ReturnType.IsSubclassOf(typeof(System.Threading.Tasks.Task)))
            return true; //Task Support
        else
            return ReflectionUtility.HasAttribute(_testMethod, typeof(AsynchronousAttribute));
    }
    else if (MethodInfo != null)
    {
        return ReflectionUtility.HasAttribute(MethodInfo, typeof(AsynchronousAttribute));
    }
    else
    {
        return false;
    }
}

Next is modifying the Invoke method that executes your test, which is located in ‘Microsoft.Silverlight.Testing\Metadata\VisualStudio\TestMethod.cs’. This is where the main work is done to enable tasks to work:

public virtual void Invoke(object instance)
{
    _methodInfo.Invoke(instance, None);
}

This now changes to:

public virtual void Invoke(object instance, CompositeWorkItem workItem)
{
    var t = _methodInfo.Invoke(instance, None) as System.Threading.Tasks.Task;
    if (t != null)
    {
        if (t.IsFaulted)
        {
            throw t.Exception;
        }
        else if (!t.IsCompleted)
        {
            var context = System.Threading.SynchronizationContext.Current;
            t.ContinueWith(result =>
            {
                context.Post((d) =>
                {
                    if (result.IsFaulted)
                    {
                        Exception ex = result.Exception;
                        if (ex is AggregateException)
                            ex = ex.GetBaseException();
                        workItem.WorkItemException(ex);
                    }
                    else
                        workItem.WorkItemCompleteInternal();
                }, null);
            });
        }
    }
}

Basically it grabs the task that is returned and calls the code that TestComplete would have called or what a raised exception would have called in case the test raises an exception. Also note that we changed the signature of the method to give us the CompositeWorkItem we need to raise these events on. This change does affect quite a lot of other code, but it’s merely a matter of adding the same parameter there as well, and the only place that calls this method (which is the CompositeWorkItem) to set this parameter to ‘this’.

Now you can also write tests that tests for exceptions thrown. Often you don’t even need to await the result in those cases:

[TestMethod]
[ExpectedException(typeof(ArgumentOutOfRangeException))]
public Task TestOutOfRange()
{
    return Utility.Compute(0);  //no need to await
}

[TestMethod]
public Task TestOutOfRange_Failure() //This test will fail
{
    return Utility.Compute(0);
}

And here’s what that looks like for the entire test run:

image

To make it easy on you, you can download the modified unit test framework source here.

…But EVEN better: Go vote for this to be part of the official toolkit here:  http://silverlight.codeplex.com/workitem/11457

Tags:

WPF | WPF vs Silverlight | Windows Phone | Windows Runtime | Silverlight

The Windows Phone Quirks Mode And 3rd Party Libraries

by Morten 9. November 2012 08:10

With the new Windows Phone 8.0 OS a lot of things has changed with the new kernel and CLR, and in addition there are quite a few breaking changes to the SDK. However WP8 has a “quirks mode” that it uses to detect if an app was built for 7.1 (Mango), it will execute it as if it was running on a WP7.1 device. That means that if your app is using some of the features that has changed in WP8, it should continue to run with no problems (I have identified several compatibility issues though). This is great because the over 120,000 apps in the store today should (for the most part) work on your new WP8 device.

However if you upgrade your app and compile it for WP8, you will exit this quirks mode, and your app could potentially break if you are subject to any of the breaking changes. You can find a good list of the breaking changes here: http://msdn.microsoft.com/en-US/library/windowsphone/develop/jj206947(v=vs.105).aspx

Upgrading poses a big problem though: Most of you are probably relying on 3rd party libraries that haven’t been upgraded to or certified for WP8. The Quirks Mode is enabled for your entire app, and cannot run parts of your app in quirks mode, and other parts outside quirks mode. This means if your 3rd party library hits any of these breaking changes and you use it in a WP8 app, your app WILL break.

image

If you are a 3rd party library developer, you should test your library for compatibility. If you find any issues, you should probably release two versions, one for WP7 and one for WP8.

Of course if you don’t really need any of the new WP8 features, my recommendation is to stick to WP7.1; - at least until there’s a big enough user-base for WP8 and a small WP7.1 user-base.

Note: All this also applies to good old browser Silverlight which have had quirks mode for a long time, so this isn’t a new concept. This is actually the reason that the product I work on releases both Silverlight 4 and Silverlight 5 versions, because the Silverlight 4 assemblies causes problems when used in a Silverlight 5 app. We had a lot of customers still stuck on SL4 and others who wanted to use new SL5 features, so we chose to support both for some time. The same will be the case for WP7 which will be around for a while, and you might have to support both.

Tags:

Windows Phone

Why Windows Phone 8 Excites Me

by Morten 30. October 2012 11:00

I'm here at the build conference, where it was announced that we finally got access to the Windows Phone 8 SDK. I’ve been browsing the SDK reference, and found a bunch of stuff that I think is going to make this phone huge.
To me, it’s not about the new improved tile interface, the camera lenses or skype integration. It’s what’s under the hood that is really going to make a difference, allowing us to finally build amazing apps that sometimes were hard or flat out impossible.

So a lot of it is missing features that I felt has been holding back the WP7.5 platform (and will continue with 7.8 which is a sorry excuse of an update that’ll confuse consumers), but I can honestly say that I think the feature-set in the WP8 SDK is a home run and there’s nothing big missing from there that I need (even some stuff I didn’t think I need that I now do :-). I know some of these features are available on other platforms already (flamers move on), but the combination of all of these is what makes this all great.

Enterprise ready

We finally get a full enterprise story with your own “enterprise store”. Before WP8, the only option to deploy apps in the enterprise was developer unlock all the phones, or put them in the store as betas (that expires after 3 months) or make them publicly available but hidden behind a login. None of them was a very good solution. We now get a proper enterprise marketplace for easy deployment and update of enterprise apps.

Bluetooth devices

We now get access to bluetooth devices. This is huge and can spawn a new type of eco system. In my work area that means could mean super-high-precision GPS receivers and laser range finders just to start. But in other areas this could be for instance credit card readers and barcode scanners to do on-the-spot purchasing.

Background downloader

Allows you to queue up large downloads while your app is not running. Great for loading larger amounts of data that your app needs for a specific job or for being offline for extended time.

Hot-swappable storage

With hot-swappable storage you can quickly provision large amounts of data to bring with you in the field. Again this could be huge for the enterprise solution. Having to provision data over the air could be an issue even with the background downloader (think gigabytes of data). Imagine highly detailed maps for a specific region you’re about to enter – you bring the right SD card for that area, and someone else brings another SD card with data for their area.
And for those who just want to throw music on there and needs more space: It’s greatc for that too :-)

Direct3D

Direct3D support! We now get direct access to the GPU. This allows us to write high-performance rendering for games, maps, etc. Note: There’s no Direct2D support (but I’m sure someone will build that on top of D3D for us soon, since it really just an abstraction on top of the 3D libraries).

Native code support

We can now write/reuse/run native code on our smartphones. You might not want to write C++, but there are huge amounts of libraries today written in C++ that we can now use. Think of for instance the Sqlite Database. Where I work we have huge amounts of native libraries that has taken years to develop and would be near-impossible to port to .NET, not to mention they require the fastest possible processing of large amounts of data, and C++ is more or less unmatched for that purpose. This doesn’t mean you have to write your entire app in C++. It could just mean that you bring in a native library (like Sqlite) and code against it from C#. The integration is very similar to WinRT (in fact they call it WinPRT – ‘P’ for Phone, and shares a lot of libraries too). So you might never have to worry about native code, but you can still get the benefit of other 3rd parties’ hard work!

.NET

Having .NET (and C#/VB.NET) is not really a new feature, but having worked in the ISV space for many years, and delivering SDKs to smaller ISVs, I know how important it is to have a .NET SDK. Many smaller businesses often have a developer or two, and most of them I find to be .NET developers. The fact that they can write a quick app for their enterprise in an environment that are familiar to them will be huge. This is probably the biggest differentiator to all the other phone platforms.

 

 

New Windows Phone 8 SDKs (slide from Build Keynote)

Tags:

Windows Phone

Running a Storyboard as a Task

by Morten 5. September 2012 18:08

Sometimes you have some code that needs to run after a storyboard has completed. In my case I'm working on a little board game, and after each move (which I animate using a Storyboard), I need to figure out the next move, and either start a new play-animation or pass the turn to the other player.

Therefore I run in a loop until the turn is over. You can detect when a storyboard has finished when the "Completed" event triggers, but that makes for some recursive spaghetti code. It's much easier if I could just "await" the storyboard using a task. So I created the little extension method below that makes this possible. All you have to do to start and wait for the storyboard to finish is:

    await myStoryboard.BeginAsync();

Below is the little extension method (which also serves as a general example on how you turn an event-based class into an awaitable Task using the TaskCompletionSource):

using System;
using System.Threading.Tasks;
using Windows.UI.Xaml.Media.Animation;

namespace SharpGIS
{
    public static class StoryboardExtensions
    {
        public static Task BeginAsync(this Storyboard storyboard)
        {
            System.Threading.Tasks.TaskCompletionSource<bool> tcs = new TaskCompletionSource<bool>();
            if (storyboard == null)
                tcs.SetException(new ArgumentNullException());
            else
            {
                EventHandler<object> onComplete = null;
                onComplete = (s, e) => {
                    storyboard.Completed -= onComplete; 
                    tcs.SetResult(true); 
                };
                storyboard.Completed += onComplete;
                storyboard.Begin();
            }
            return tcs.Task;
        }
    }
}

Note: This code is written for WinRT. If you want to use this for Silverlight or WPF, just change ‘Eventhandler<object>’ to ‘EventHandler’.

Tags:

Windows Phone | WPF | Windows Runtime | Silverlight | XAML

Why Custom Controls are underrated

by Morten 6. February 2012 22:48

I’m a big fan of building controls. I love writing them, designing them, trying to make it work in as many scenarios I can while keeping them simple, extensible and most importantly reusable. In fact for the past 6 years, it’s all I’ve been doing full time (first ASP.NET and later XAML), and frequently in my spare time as well.

If you dabble in XAML, you have most likely already been building some controls, by going “File -> New -> User Control” in Visual Studio. You probably do this because you want to create a new page in your app, or you just want to encapsulate some of the UI in a separate section. Or perhaps it’s because you realize that this little tidbit can be used over and over again in your application. Or maybe you have even considered it can be used again and again across many DIFFERENT applications. If you have tried those two last categories (or if you will one day), this blog post is for you! It will apply to any of the XAML techs there is: WPF, Silverlight, Windows Phone and the future Windows 8 Runtime.

Despite the title in this blog post, User Controls are awesome. They are quick to throw together and reuse over and over - and there’s a lot of value in that. But what if I told you there’s another control type that has even MORE power, better performance, and can be way more flexible and reusable than a user control, and where the clean code will make most developers fall in love?

The thing is you already know this because you’ve been using them all the time: Button, ListBox, ItemsControls, Grid, StackPanel etc. are all controls harnessing the same power you can! And you have probably seen XAML styles that completely changed the look and feel of a control, without touching any of its code. To give you an idea of how powerful this is, look at this Silverlight Sample below. On the left you will see a ListBox binding to a list of planets. You have probably already done something like this. On the right, you see a solar system. But in fact this is ALSO a ListBox. And there is NO extra code involved here. It’s done entirely by restyling the template. Notice how selection and up/down keys work just like it does with the “normal” ListBox. So I got to reuse all the code that has this, and all I had to do was restyle the ListBox a bit. Something I could have done entirely in Blend without ever touching code.

Let me repeat that: I didn’t add any code to the ListBox do this. In fact the code behind for this page is completely empty. If you don’t believe me, here’s the source code. You can also see more about this technique in this presentation from Mix’08, or read David Ansons blogpost on it.

So at this point hopefully I have won you over to learning more about Custom Controls (if not I’m amazed you have read this far :-).

The Anatomy of A User Control

To start, let’s first look at the anatomy of a typical UserControl and try and fully understand how that works first. Below here we have the XAML portion of our control that defines the layout. We’ll keep it simple and have a Grid with a Button inside it:

<UserControl x:Class="MyApp.SilverlightControl1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
    
    <Grid x:Name="LayoutRoot" Background="White">
        <Button Content="Click Me" Click="Button_Click" Opacity=".5" />
    </Grid>
</UserControl>

And we have the code-behind that loads up the control, handles user interaction etc.

using System.Windows;
using System.Windows.Controls;
using System.Windows.Media;

namespace SolarSystemRetemplate
{
    public partial class SilverlightControl1 : UserControl
    {
        public SilverlightControl1()
        {
            InitializeComponent();
        }

        private void Button_Click(object sender, RoutedEventArgs e)
        {
            LayoutRoot.Background = new SolidColorBrush(Colors.Red);
        }
    }
}

The things worth noting here are two things: “LayoutRoot” is defined in the XAML using x:Name, and we automatically get a variable by that name in code behind. Also the event handler hooked to the Button’s Click event is magically linked to the code-behind. All this is really handled by the compiler and the “InitializeComponent” call - a method that interestingly doesn’t exist here. The reason this works is really because this is a partial class as indicated, and Visual Studio creates a little ‘secret’ file under the covers for you. You can get to if if you right-click the method and select “Go To Definition”. Here’s what the contents of that file looks like:

namespace MyApp {    
    
    public partial class SilverlightControl1 : System.Windows.Controls.UserControl {
        
        internal System.Windows.Controls.Grid LayoutRoot;
        
        private bool _contentLoaded;
        
        /// <summary>
        /// InitializeComponent
        /// </summary>
        [System.Diagnostics.DebuggerNonUserCodeAttribute()]
        public void InitializeComponent() {
            if (_contentLoaded)
                return;
            _contentLoaded = true;
            System.Windows.Application.LoadComponent(this, 
new System.Uri("/MyApp;component/SilverlightControl1.xaml",
System.UriKind.Relative)); this.LayoutRoot = ((System.Windows.Controls.Grid)(this.FindName("LayoutRoot"))); } } }

You’ll notice that the LayoutRoot is defined here as internal, and it’s assigned using the “FindName” method.

This is one of the nice things about UserControls: A lot of the work is automatically done for you, but with Custom Controls you will have to do this yourself! (but this isn’t so bad considering the power you get!). And here’s the kicker: A UserControl is just another custom control!

The Anatomy of A Custom Control

A custom control doesn’t have a XAML and a code-behind component in the same way UserControl does. Instead it’s ALL code along with a default XAML template. You can consider the template the equivalent of the XAML in the User Control, but the important part to remember here is that this template can be changed by ANYONE, which is what I did to the ListBox in the solar system sample. Another thing to note is that since the template doesn’t have a corresponding code-behind where Visual Studio generates a partial class for you, any event handlers cannot be defined in the template. So how do we go about recreating the user control above as a custom control?

For Silverlight this is easy. Right-click your project and select “File -> Add New -> Silverlight Templated Control”. WPF and Windows Phone doesn’t come with this template so you’ll have to do it manually there, by creating a class and a generic template file. After you do this, you’ll notice two new files: First a simple C# class, and second a new file in \Themes\Generic.xaml. The second file is where you place all templates for all your controls in that assembly. It HAS to have this name and live in this folder for the custom control to pick up the template.

Below is what this template looks like. I’ve added the grid and the button inside the suggested border that was created for me.

<ResourceDictionary
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="clr-namespace:MyApp">

    <Style TargetType="local:TemplatedControl1">
        <Setter Property="Template">
            <Setter.Value>
                <ControlTemplate TargetType="local:TemplatedControl1">
                    <Border Background="{TemplateBinding Background}"
                            BorderBrush="{TemplateBinding BorderBrush}"
                            BorderThickness="{TemplateBinding BorderThickness}">
                        <Grid x:Name="LayoutRoot">
                            <Button x:Name="ClickButton" Content="Click me!" Opacity=".5" />
                        </Grid>
                    </Border>
                </ControlTemplate>
            </Setter.Value>
        </Setter>
    </Style>
</ResourceDictionary>

First notice the TemplateBinding statements on the border. This is an important feature of controls. You can bind straight to a dependency property defined in your control code. Since a custom control inherits from “Control”, you automatically get Background, BorderBrush, BorderThickness and many other general properties from the inheritance. The great thing is that you can just write <my:TemplatedControl Border=”Red” /> and the border will automatically be bound into this template (and anywhere else where you have a TemplateBinding to that property). This beats UserControl, where to accomplish this in Silverlight you will have resolve to a hack by setting the DataContext of the control to itself, breaking the DataContext flow.

Second, notice that I didn’t add a click-handler to Button. If I did, this template would fail to load. We’ll hook the click handler up later.

Next let’s look at the code for the control:

using System.Windows;
using System.Windows.Controls;
using System.Windows.Controls.Primitives;
using System.Windows.Media;

namespace MyApp
{
    [TemplatePart(Name="LayoutRoot", Type=typeof(Control))]
    [TemplatePart(Name = "ClickButton", Type = typeof(ButtonBase))]
    public class TemplatedControl1 : Control
    {
        Control layoutRoot;
        ButtonBase button;
        public TemplatedControl1()
        {
            this.DefaultStyleKey = typeof(TemplatedControl1);
        }
        public override void OnApplyTemplate()
        {
            if (button != null) //unhook from previous template part
            {
                button.Click -= new RoutedEventHandler(button_Click);
            }    
            button = GetTemplateChild("ClickButton") as ButtonBase;
            if (button != null)
            {
                button.Click += new RoutedEventHandler(button_Click);
            }
            layoutRoot = GetTemplateChild("LayoutRoot") as Panel;
            base.OnApplyTemplate();
        }

        private void button_Click(object sender, RoutedEventArgs e)
        {
            layoutRoot.Background = new SolidColorBrush(Colors.Red);
        }
    }
}

First I declare the “TemplatePart” attributes on the control. They tell what parts (ie controls) are expected to be in my template. In this case LayoutRoot of type Panel (Grid is a Control), and ClickButton of type ButtonBase. These are not strictly required, but they help Expression Blend understand the template requirements when you later customize the control. I always declare the lowest needed type in the control hierarchy to make the template more flexible. For instance I use ButtonBase and not Button, because I only rely on the Click event which is declared on the ButtonBase base class. That way I don’t lock a user of the control into using “Button” but they can place ANY control that inherits from ButtonBase here. Same thing applies for the LayoutRoot, where I just need the Background property.

Next the control inherits from “Control”. Custom controls must inherit from this.

In the constructor I define the “DefaultStyleKey”. This tells the framework that I have a default template defined in Themes\Generic.xaml. If I didn’t the user would always have to explicitly defined a control template for the control.

Lastly, the most important part is “OnApplyTemplate”. This method is called when the control has loaded the template. This is our earliest opportunity to grab references to controls in the template, ie. the TemplateParts. In this case I grab a reference to the ButtonBase defined in the template. If it’s found, I’ll add a click handler to it. Also if a new template gets applied, I must remember to unhook from the previous instance (this is a rare scenario though, and you could probably get away with skipping that bit). It’s also important to note that Template Parts are always optional! So always do the null check anywhere you rely on a reference to a template part.

And that’s really it! I kept the sample simple, so it is easier to go through the individual parts of a control, therefore the differences between a custom control and a user control doesn’t really stand out. If this was all you needed to do, a custom control is probably overkill. But think of scenarios where you have a lot of code-behind that you want to reuse, but you don’t want to lock the design in. The major next parts you will want to add to this now is more Dependency Properties you can bind into the template, as well as VisualStates - ie. storyboards that triggers on certain events. The great thing about Visual States is that the code-behind doesn’t define the storyboard or what it does - only when it starts. This gives the user even more flexibility to customize the behavior.

Adding Visual States to the control

Let’s add some mouse over states to our control, and have the control animate when that happens. In the code-behind where we defined the TemplateParts let’s add two TemplateVisualState attributes:

[TemplateVisualState(GroupName = "HoverStates", Name = "MouseOver")]
[TemplateVisualState(GroupName = "HoverStates", Name = "Normal")]

Again these are optional, but great for Blend integration.

Next add the code that triggers the visual state to the control:

bool isMouseOver;
protected override void OnMouseEnter(System.Windows.Input.MouseEventArgs e)
{
    isMouseOver = true;
    ChangeVisualState(true);
    base.OnMouseEnter(e);
}
protected override void OnMouseLeave(System.Windows.Input.MouseEventArgs e)
{
    isMouseOver = false;
    ChangeVisualState(true);
    base.OnMouseLeave(e);
}

private void ChangeVisualState(bool useTransitions)
{
    if (isMouseOver)
    {
        GoToState(useTransitions, "MouseOver");
    }
    else
    {
        GoToState(useTransitions, "Normal");
    }
}

private bool GoToState(bool useTransitions, string stateName)
{
    return VisualStateManager.GoToState(this, stateName, useTransitions);
}

This is really all the code we need. It’s pretty simple. If the mouse is over, trigger the MouseOver state, else trigger the Normal state. Note how we don’t really define what “MouseOver” looks like. That’s the job of the template. Let’s define that (you might already be very familiar with this when overriding templates - it’s exactly the same thing, except we get to define the default state):

<ControlTemplate TargetType="local:TemplatedControl1">
    <Border Background="{TemplateBinding Background}"
            BorderBrush="{TemplateBinding BorderBrush}"
            BorderThickness="{TemplateBinding BorderThickness}">
        <VisualStateManager.VisualStateGroups>
            <VisualStateGroup x:Name="HoverStates">
                <VisualState x:Name="MouseOver">
                    <Storyboard>
                        <ColorAnimation
                            Storyboard.TargetName="BackgroundElement"
                            Storyboard.TargetProperty="(Rectangle.Fill).(SolidColorBrush.Color)"
                            To="Yellow" Duration="0:0:.5" />
                    </Storyboard>
                </VisualState>
                <VisualState x:Name="Normal">
                    <Storyboard>
                        <ColorAnimation
                            Storyboard.TargetName="BackgroundElement"
                            Storyboard.TargetProperty="(Rectangle.Fill).(SolidColorBrush.Color)"
                            To="Transparent" Duration="0:0:.5" />
                    </Storyboard>
                </VisualState>
            </VisualStateGroup>
        </VisualStateManager.VisualStateGroups>
        <Grid x:Name="LayoutRoot">
            <Rectangle x:Name="BackgroundElement" Fill="Transparent" />
            <Button x:Name="ClickButton" 
                    Content="Click me!" Opacity=".5" />
        </Grid>
    </Border>
</ControlTemplate>

So the changes here is adding a rectangle in the background that we animate into yellow when the mouse hovers over.

You now have a control that sets a background on some Panel when some ButtonBase is clicked, as well as running an animation on MouseEnter/Leave. This could serve the purpose for quite a lot of controls, without you having to rewrite the code!

Here’s a few resources you will want to read if you want to learn more about this:

A couple of other controls I’ve built over the time and described on this blog:

If you want to go even more hardcore, wrap your head around the ArrangeOverride and MeasureOverride methods. This is where you can get some really amazing control over how the contents are laid out, but this is outside the scope of this article, but I urge you to read into it. Here’s one article to get your started on that: http://www.switchonthecode.com/tutorials/wpf-tutorial-creating-a-custom-panel-control

Tags:

XAML | WPF | Windows Runtime | Silverlight | Windows Phone

Overwriting the default WebRequest used by WebClient

by Morten 23. January 2012 21:18

In Silverlight/WPF and WPF there’s today a WebClient class for making webrequests. Under the covers it creates HttpWebRequest for sending the request and HttpWebResponse for handling the response coming back from the server. In an older blogpost, I created a custom WebClient for WP7 that added support for GZIP compression this way.

However the default WebClient doesn’t actually create an “HttpWebRequest/Response” object, since this is an abstract class. What it really does is create internal subclasses that is then used. This is handled by the WebRequestCreator static class which really creates these classes for you. However a less-known feature is that you can actually tell your application that you want to use a specific client for handling the web request and responses for specific domains. You might actually already have fiddled with this in Silverlight if you wanted to explicitly let Silverlight handle the web requests instead of handing it off to the hosting browser control using the following statement:

WebRequest.RegisterPrefix("http://www.mydomain.com/", WebRequestCreator.ClientHttp);

This call makes all requests to www.mydomain.com go through the ClientHttp. As mentioned it’s possible to make your own requestor and completely intercept ANY requests made by WebClient!

If you have your own HttpWebRequest implementation ‘MyHttpWebRequest’, this would look something like this:

public static class MyWebRequestCreator
{
    private static IWebRequestCreate myCreator;

    public static IWebRequestCreate MyHttp
    {
        get
        {
            if (myCreator == null)
            {
                myCreator = new MyHttpRequestCreator();
            }
            return myCreator;
        }
    }

    private class MyHttpRequestCreator : IWebRequestCreate
    {
        public WebRequest Create(Uri uri)
        {
            return new MyHttpWebRequest(uri);
        }
    }
}

So now you can just call:

WebRequest.RegisterPrefix("http://", MyWebRequestCreator.MyHttp);
WebRequest.RegisterPrefix("https://", MyWebRequestCreator.MyHttp);

…and ALL web requests to http or https websites, will be routed through your MyHttpWebRequest class (which you granted still have to build). This class could just wrap the built-in ClientHttpWebRequest and you can modify/add/remove/decode stuff to your pleasing. This is awesome for test framework mock ups, but it’s also really neat for the GZIP WebClient control I mentioned earlier. Why? Well to use my GZipWebClient, you would have to go through all your code and change:

WebClient client = new WebClient();

to

WebClient client = new SharpGIS.WebClient();

This can be a lot of work, and even if you did that, what about all the 3rd party libraries you use, where you can’t go in and change that? Well enter RegisterPrefix! Do you want GZIP support in Hammock? Done! Do you want GZIP support in RestSharp? Done!* Do you want GZIP support in [insert favorite API here]? Done!

*Update: Actually this might not be the case. Some libraries uses some features that either is not possible to implement with the custom HttpWebRequest (like UserAgent), and I didn’t add support for Cookies as well. Both of these are for instance used by RestSharp.

So I went ahead and updated my Nuget Package for the GZipWebClient, so go grab v1.1, and you will only have to add 1-2 LINES OF CODE, and just execute it ONCE (for instance at startup), and EVERYTHING WebClient will be enhanced with GZIP compression. You can also grab the updated source on the original blogpost here.

I really have to give some credit to my co-worker Anargyros for this one, who mentioned that it might be possible to do it this way. Go follow him on twitter and give him some twitter love @tomahawk1187

Tags:

Windows Phone | WPF | Silverlight

Building A Multi-Touch Photo Viewer Control

by Morten 17. January 2012 22:02

If you want to view an image on your Windows Phone app, or on your Windows 8 tablet, most people would probably expect to be able to use their fingers to pinch zoom and drag using the touch screen.

Since this is a common scenario, I want to create a simple reusable control that allows me to do this using very little xaml, along the lines of this:

<my:ImageViewer
    Thumbnail="http://url.com/to/my/thumbnail.jpg"
    Image="http://url.com/to/my/MyImage.jpg"  />

…where Thumbnail is a low resolution image that loads fast, while the full resolution Image is being downloaded.

If you just want to use this control and don’t want to learn how to create a custom control, skip to the bottom to download the source for both Windows Phone and Windows 8 Runtime.

First off, we’ll create a new Windows Phone Class Library project and name it “SharpGIS.Controls”. (or whatever you like)

image

Create a new folder “Themes”, add a new XAML resource file and name it “Generic.xaml”. Make sure the “build action” for this file is set to “ApplicationDefinition”.

We will want to define the default template in this file for our control (if you are used to making User Controls, this is essentially where the XAML for custom controls go instead).

In the xaml we will want two things: An image for displaying a fast-loading thumbnail at first, and a second image for displaying the high resolution image. Also we will use a Grid around them to group them together. Generic.xaml should look something like this:

<ResourceDictionary
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="clr-namespace:SharpGIS.Controls">
    <Style TargetType="local:ImageViewer">
        <Setter Property="Template">
            <Setter.Value>
                <ControlTemplate TargetType="local:ImageViewer">
                    <Grid x:Name="Root" Background="Transparent">
                        <Image x:Name="Thumbnail" Source="{TemplateBinding Thumbnail}" CacheMode="BitmapCache" />
                        <Image x:Name="Image" Source="{TemplateBinding Image}" CacheMode="BitmapCache" />
                    </Grid>
                </ControlTemplate>
            </Setter.Value>
        </Setter>
    </Style>
</ResourceDictionary>

Notice the “CacheMode” property. This is important to set, to get proper performance when zooming and panning the image. If you have a Windows Phone try the official Facebook app, open an image and you will see what I mean (if you worked on that app PLEASE fix this).

Next up is the actual code. Add a new class, and name it “ImageViewer”. Make it inherit from ‘Control’. Add a constructor and use the “DefaultStyleKey” to tell Silverlight that you have a template defined in Generic.xaml that it should use.

public class ImageViewer : Control
{
    public ImageViewer()
    {
        DefaultStyleKey = typeof(ImageViewer);
    }
}

Next we define the dependency properties for the two images that the template was binding to:

public ImageSource Image
{
    get { return (ImageSource)GetValue(ImageProperty); }
    set { SetValue(ImageProperty, value); }
}

public static readonly DependencyProperty ImageProperty =
    DependencyProperty.Register("Image", typeof(ImageSource), typeof(ImageViewer), null);

public ImageSource Thumbnail
{
    get { return (ImageSource)GetValue(ThumbnailProperty); }
    set { SetValue(ThumbnailProperty, value); }
}

public static readonly DependencyProperty ThumbnailProperty =
    DependencyProperty.Register("Thumbnail", typeof(ImageSource), typeof(ImageViewer), null);

We should now be able to use this control in a Windows Phone project. Add a new Windows Phone Appplication project to your solution, right-click the ‘references’ and select “add reference’. Pick the SharpGIS.Controls project.

You should now be able to use some XAML like this to display an image:

<my:ImageViewer
    xmlns:my="clr-namespace:SharpGIS.Controls;assembly=SharpGIS.Controls"
    Thumbnail="http://url.com/to/my/thumbnail.jpg"
    Image="http://url.com/to/my/image.jpg" />

That’s all fine, but you still can’t use any touch to zoom the image.

Go back to the code and override OnApplyTemplate(). This code executes when the template from Themes\Generic.xaml has been loaded, and it’s your chance to grab any reference to the elements in there and ‘do something’ with them. In user controls you would often set the event handlers directly in the xaml. With templates on custom controls, you will have to hook these up in code-behind during OnApplyTemplate().

Here we will hook up for the manipulation events, as well as assign a transform we will apply to the element when these events trigger.

private Grid Root;
        
public override void OnApplyTemplate()
{
    Root = GetTemplateChild("Root") as Grid;
    if (Root != null)
    {
        Root.ManipulationDelta += Root_ManipulationDelta;
        Root.ManipulationStarted += Root_ManipulationStarted;
        Root.RenderTransform = new CompositeTransform();
    }
    base.OnApplyTemplate();
}

The ManipulationDelta event triggers as you move. It will give you information about how much the user dragged, and how much he/she pinches, as well as the center of the pinch. Unfortunately the pinch scale amount is shown as separate X and Y directions, and no ‘Uniform Scale’ is shown. This makes it hard to get a good average of the scale, and you would have to pick just one of them (one could be pinching and the other stretching).

I’ve found that defining the amount you’re scaling is the change of the length of the diagonal of the boundingbox of all the touch points works well. Errrrrrr, that might have sounded confusing. Let’s use a picture instead. The orange circles below are touchpoints, and the rectangle is the box that encompasses all of them (2 or more points). The blue line is the diagonal length of this box. So the amount of scaling = length_Before / length_After.

image

We don’t get the actual touch points in the manipulation events. So instead I start with a simple box that I define as 1x1 and track the scaling of it. The diagonal length of that box at the beginning is the square root of 2, which we will define in the ManipulationStarted event. We also add a property for tracking the center of the box.

private Point? lastOrigin;
private double lastUniformScale;

private void Root_ManipulationStarted(object sender, ManipulationStartedEventArgs e)
{
    lastUniformScale = Math.Sqrt(2);
    lastOrigin = null;
}

So all that’s left is listen to the ManipulationDelta event, and update the transform on the grid. This consist of comparing previous origin to the new, as well as calculating the scale factor based on the box diagonal. Also note that when you apply scale, this is means you’re scaling out and away from the upper left corner. To offset this, you will need to add some additional translation to the image, as shown below:

private void Root_ManipulationDelta(object sender, ManipulationDeltaEventArgs e)
{
    var transform = Root.RenderTransform as CompositeTransform;
    if (transform != null)
    {
        var origin = e.ManipulationContainer.TransformToVisual(this).Transform(e.ManipulationOrigin);

        if (!lastOrigin.HasValue)
            lastOrigin = origin;

        //Calculate uniform scale factor
        double uniformScale = Math.Sqrt(Math.Pow(e.CumulativeManipulation.Scale.X, 2) + 
                                        Math.Pow(e.CumulativeManipulation.Scale.Y, 2));
        if (uniformScale == 0)
            uniformScale = lastUniformScale;

        //Current scale factor
        double scale = uniformScale / lastUniformScale;

        if (scale > 0 && scale != 1)
        {
            //Apply scaling
            transform.ScaleY = transform.ScaleX *= scale;
            //Update the offset caused by this scaling
            var ul = Root.TransformToVisual(this).Transform(new Point());
            transform.TranslateX = origin.X - (origin.X - ul.X) * scale;
            transform.TranslateY = origin.Y - (origin.Y - ul.Y) * scale;
        }
        //Apply translate caused by drag
        transform.TranslateX += (origin.X - lastOrigin.Value.X);
        transform.TranslateY += (origin.Y - lastOrigin.Value.Y);

        //Cache values for next time
        lastOrigin = origin;
        lastUniformScale = uniformScale;
    }
}

And that’s it!

Now what’s left is to turn off the thumbnail when the image has loaded, as well as raise loaded events for the high resolution image, so that you can display a progress bar while you wait for the image to load. I won’t go into details on this, but in OnApplyTemplate, simply grab the Image template child, and listen for the ImageLoaded event.

I’ve packaged it all up in one complete control, as well as a sample showing how this would be used in an application where you would navigate to a page with this control on it.

You can download the source and a sample app here.

Here’s a preview off what that app looks like:

Tags:

Windows Phone | XAML

Building an Augmented Reality XAML control

by Morten 7. December 2011 22:17

Augmented Reality on Windows Phone and Windows 8 Metro style apps

I’ve lately been playing a lot with augmented reality on my Windows Phone and my Windows 8 Tablet. Both of them feature an accelerometer, compass and gyro as well as a back facing camera. This makes it possible to overlay various XAML elements on top of the camera feed, and have them move with the camera as you rotate and orient the device. As an example of this, you can try the AR view in my GuidePost app for Windows Phone (and while you’re at it, make sure you throw some good reviews in there ;-). The app uses the exact approach described here.

To do augmented reality in XAML, I built a custom panel control that automatically handles the placement of all it’s child elements based on the device’s orientation. I used an attached property to define what ‘direction’ an element is supposed to be placed in. You already know this from Grid and Canvas, where for instance Grid.Row=”1” places it in the second row of the grid, or Canvas.Top=”100” places the element 100 pixels from the top of the canvas. In my case I use ARPanel.Direction to define a Point where the first value is the azimuth (ie up/down) and second value is the compass heading.

Note: Since you probably know the Grid panel very well, to help explain how this custom panel works, you’ll sometimes see a “How Grid uses this” section, to explain how the grid control would do something similar.

Here’s what the four cardinal directions and up and down would look like in such a custom panel that we’ll be building:

<ar:ARPanel>
    <TextBlock Text="North" ar:ARPanel.Direction="0,0" />
    <TextBlock Text="East"  ar:ARPanel.Direction="0,90" />
    <TextBlock Text="South" ar:ARPanel.Direction="0,180" />
    <TextBlock Text="West"  ar:ARPanel.Direction="0,270" />
    <TextBlock Text="Up"    ar:ARPanel.Direction="90,0" />
    <TextBlock Text="Down"  ar:ARPanel.Direction="-90,0" />
</ar:ARPanel>

Neat huh? This makes it VERY simple to spit out a bunch of different AR-type of apps. So how did I build this?

The custom panel

When building a custom panel, there’s really just 3 things you need to do:

  1. Inherit from Panel
  2. Override MeasureOverride
  3. Override ArrangeOverride

Or to put that in code:

public class ARPanel : Panel
{
    protected override Size MeasureOverride(Size availableSize)
    {
        //TODO
    }
    protected override Size ArrangeOverride(Size finalSize)
    {
        //TODO
    }
}

The two methods here have the following purposes:

MeasureOverride is called first, and it should go through all the child elements and tell them “Hey I got this much space available - how much of that do you need?”.

How Grid uses this: If a grid is 150x100 and has two rows and two columns, it would look at the row/column of the element then say “I got this much 75x50 space in this row/column - how much do you want”. If all the row/column sizes are auto, it would send the entire size of the grid itself to the child.

ArrangeOverride is called second. This goes through all the child elements and tells them “I heard you want this much space (as indicated by child.DesiredSize), but I can only give you this much, and this is the area where you get to place yourself”.

How Grid uses this: After going through the measure step, Grid would now determine what the size of each row and column needs to be based on the child desired elements (unless the cell sizes are fixed), and then calculate the rectangle for each cell distributed based on the row/column definitions. These rectangles are then used to place the child elements in the row/column they are set to be placed in.

Now first let’s implement MeasureOverride, since this is very simple in this scenario. Each child element can render itself anywhere around you in a 360 degree space, so we don’t need to limit the child elements to any size other than the size of the panel itself. So we’ll ask each child “Given all of my space, how much would you want to use?”. This is simply done by going through each child element and calling Measure on them with the full size available (after each measure call, you will notice that child.DesiredSize will now be assigned). We end the call by returning the size that the panel needs to it’s parent - in this case we want to use the entire space available to us:

protected override Size MeasureOverride(Size availableSize)
{
    foreach (var child in Children)
        child.Measure(availableSize);
    return availableSize;
}

MeasureOverride is called whenever the panel’s size is changing or anything else that would affect the size of the child elements. You can trigger this yourself by calling panel.InvalidateMeasure(). In our specific case we don’t ever need to call this, and the measure will only get invalidated when the panel’s size changes - for instance when page orientation changes, and luckily this is all done for us. (Grid would call InvalidateMeasure if the row/column definitions change)

Next up is ArrangeOverride. Now that we measured all the elements, we need to determine where to place them on the screen, and how much space we need to give each of them. We’ll basically go through each element and call Arrange(rect) on them, where the ‘rect’ is the rectangle it gets to render itself in. Ie. the top-left corner is where the area starts, and the width/height is the size it gets. It’s then up to the child to make use of that area. In our ARPanel case, the top/left corner would constantly change as the orientation of the device changes, but we’ll keep the width/height to the desired size of the element. So the method will look something like this:

protected override Size ArrangeOverride(Size finalSize)
{
    foreach(var child in Children)
    {
        double centerX = TODO;
double centerY = TODO;
double left = centerX - child.DesiredSize.Width * .5; double top = centerY - child.DesiredSize.Height * .5; Rect rect = new Rect(new Point(left, top), child.DesiredSize); child.Arrange(rect); } }

I subtract half the width/height of the size to center the element on top of ‘centerX’ and ‘centerY’, which we will still have to determine;

So this is basically our panel implementation. The tricky part is figuring out the “TODO” part of the above code based on the motion sensor. So basically, given a device orientation, what is the screen location of a direction?

Warning: I spent quite a lot of time on getting this part right. It’s a lot of vector math, and some of it I came to using pure trial and error :-). Therefore I might not even completely get all of it myself, but it works, and it works great. I’m pretty sure the math is right, but if I don’t go too much into detail in some of the math, just think of it as magic that works and use the code as is :-) For those of you who are used to XNA 3D Game programming, I’m sure this will all make sense to you though, since this is very similar to how you control a camera in a 3D environment.

The motion sensor

The Motion Sensor is what will be driving the placement of the child elements. Motion is a combination of Compass, Accelerometer and optionally Gyro. The Gyro is not required, but it makes the result A LOT better - especially when you rotate the device fast. Future certified Windows 8 tablets will require all 3 sensors, but for Windows Phone, only some of the newer Mango phones has a gyro, and I believe the Dell Venue Pro compass sensor doesn’t work. Future cheaper Windows Phone devices might not come with any sensors available, so beware that your app might not work on all devices, and you should check for the capability and notify the user if it’s missing.

On Windows Phone you can check whether the motion sensor is supported using ‘Microsoft.Devices.Sensors.Motion.IsSupported’. I didn’t find an equivalent property in Windows Runtime though. The following code starts reading from the sensor for both WinPhone and Windows Runtime:

#if WINDOWS_PHONE
    if (Microsoft.Devices.Sensors.Motion.IsSupported)
    {
        motion = new Microsoft.Devices.Sensors.Motion();
        motion.CurrentValueChanged += motion_CurrentValueChanged;
        motion.Start();
#elif WINRT
    motion = Windows.Devices.Sensors.OrientationSensor.GetDefault();
    if (motion != null)
    {
        motion.ReadingChanged += motion_CurrentValueChanged;
#endif
    }
    else
    {
        throw new InvalidOperationException("Motion sensor not supported on this device");
    }
#endif

When we get a new reading, all we have to do is tell the panel, that the current child placements are invalid. We don’t want to do too much work here, because the motion sensor can be triggering much more frequent than the layout cycle, so all we do here is flag the panel for arrange and clear any parameters that could be affected by the orientation change (like the attitude which I will get back to):

    private Matrix? _attitude;

    private void motion_CurrentValueChanged(object sender, EventArgs e)
    {
        _attitude = null;
 #if WINDOWS_PHONE
        Dispatcher.BeginInvoke(() => InvalidateArrange());
#elif WINRT
        Dispatcher.Invoke(Windows.UI.Core.CoreDispatcherPriority.Normal, 
                          (a, b) => InvalidateArrange(), this, null);
#endif
    }

Handling sensor and screen coordinate systems

The Motion sensor gives us a Matrix that defines the rotation of the device relative to up and north directions. The screen has a different coordinate system that is relative to the upper left corner of the screen. When the screen changes between landscape and portrait mode, this coordinate system changes relative to the motion sensors coordinate system, and it’s important to take this into account. Lastly we need to define a field of view for the screen. If you are overlaying elements on top of the camera, the field of view must match the field of view of the camera. Think of it as how ‘narrow’ the view is. A lens that is zoom in far has a small field of view, and only few elements will be visible on the screen, whereas a wide angle lens will have more child elements on the same screen. I found that on my phones a FOV of roughly 35 degrees seems appropriate. So to sum up we need 3 things: The orientation of the sensor, the orientation of the view (screen), and the parameters for the camera (the projection).

First the view, which is slightly different between phone and runtime (probably because the ‘natural’ orientation on a phone is portrait mode, whereas on a PC it’s landscape mode, so the sensors are mounted different).

    Matrix view;
if (orientation == LandscapeLeft) { view = Microsoft.Xna.Framework.Matrix.CreateLookAt( new Microsoft.Xna.Framework.Vector3(0, 0, 1), Vector3.Zero, #if WINDOWS_PHONE Vector3.Right); #elif WINRT Vector3.Up); #endif } else if (orientation == LandscapeRight) { view = Microsoft.Xna.Framework.Matrix.CreateLookAt( new Vector3(0, 0, 1), Vector3.Zero, #if WINDOWS_PHONE Vector3.Left); #elif WINRT Vector3.Down); #endif } else //portrait mode { view = Microsoft.Xna.Framework.Matrix.CreateLookAt( new Vector3(0, 0, 1), Vector3.Zero, #if WINDOWS_PHONE Vector3.Up); #elif WINRT Vector3.Left); #endif

Next we define a viewport based on this view. Note that this depends on the size of the screen, so on size changed we need to remember to reset this value.

private Microsoft.Xna.Framework.Graphics.Viewport? _viewport;
private Microsoft.Xna.Framework.Graphics.Viewport Viewport
{
    get
    {
        if (!_viewport.HasValue)
        {
            _viewport = new Microsoft.Xna.Framework.Graphics.Viewport(0, 0, (int)ActualWidth, (int)ActualHeight);
            _cameraProjection = null; //camera projection depends on viewport - force a reset
        }
        return _viewport.Value;
    }
}

And from this we can now define the projection of the camera / field of view.

private Matrix CameraProjection
{
    get
    {
        if (!_cameraProjection.HasValue)
        {
           _cameraProjection = Matrix.CreatePerspectiveFieldOfView(
MathHelper.ToRadians((float)FieldOfView), Viewport.AspectRatio, 1f, 12f); } return _cameraProjection.Value; } }

Now we need the attitude of the device based on the current sensor reading (this was the value we reset in reading changed event). We rotate this value so it matches the rotation of the XNA coordinate system as well.

    private Matrix Attitude
    {
        get
        {
            if (!_attitude.HasValue)
            {
                if (motion != null 
#if WINDOWS_PHONE
                    && motion.IsDataValid
#endif
                    )
                {
                    _attitude = Matrix.CreateRotationX(MathHelper.PiOver2) * CurrentReading;
                }
                else
                    return Matrix.Identity;
            }
            return _attitude.Value;
        }
    }
#if WINDOWS_PHONE
    private Matrix CurrentReading
#elif WINRT
    private SensorRotationMatrix CurrentReading
#endif
    {
        get
        {
            return
#if WINDOWS_PHONE
                motion.CurrentValue.Attitude.RotationMatrix;
#elif WINRT
                motion.GetCurrentReading().RotationMatrix;
#endif
        }
    }

Ok that’s a lot of funky stuff, but now we are pretty set to project from vectors to screen coordinates using this simple method:

Matrix world = Matrix.CreateWorld(Vector3.Zero, new Vector3(0, 0, -1), new Vector3(0, 1, 0));
// Projects the point from 3D space into screen coordinates.
private Vector3 Project(Vector3 vector)
{
    return Viewport.Project(vector, CameraProjection, view, world * Attitude);
}

Only thing is we have a direction ray and not a vector. A little extra method for converting to a vector is needed too:

private static Vector3 PolarToVector(double px, double py, double radius)
{
    var O = (py - 90) * PI_OVER_180; // / 180d * Math.PI;
    var W = (90 - px) * PI_OVER_180; // / 180d * Math.PI;
    var x = (float)((Math.Cos(O) * Math.Sin(W)) * radius);
    var y = (float)((Math.Cos(W)) * radius);
    var z = (float)((Math.Sin(O) * Math.Sin(W)) * radius);
    return new Vector3(x, y, z);
}

We don’t have any radius, but I found always using a value of ‘10’ (which is 10 units down the ray) works pretty well. So… we are now pretty set to implement the ArrangeOverride method.

Arranging the child elements

When arranging the children, we need to first check if it’s inside the screen view, and next where on the screen that is. I’m using the BoundingFrustum to do this. Think of the bounding frustum as the space the camera can see, and is a cone the expands out from the camera We can set this up using:

    BoundingFrustum viewFrustum = new BoundingFrustum(Attitude * view * CameraProjection);

If we now build a BoundingSphere around each element, we can use the .Contains method to see if that element is within the frustum. So we first grab the Point object from the element, and build a bounding sphere. If it is inside, all we have to do is call the Project method above to get the screen location, and lastly call Arrange on the element.

So our entire ArrangeOverride method ends up looking like this:

    protected override Size ArrangeOverride(Size finalSize)
    {
        if (ActualWidth > 0 && ActualHeight > 0 && motion != null
#if WINDOWS_PHONE
            && motion.IsDataValid
#endif
            )
        {
            BoundingFrustum viewFrustum = new BoundingFrustum(Attitude * view * CameraProjection);
            foreach (var child in Children)
            {
                object posObj = child.GetValue(DirectionProperty);
                if (posObj is Point && !double.IsNaN(((Point)posObj).X))
                {
                    Point p = (Point)posObj;
                    Vector3 direction = PolarToVector(p.X, p.Y, 10);
                    var size = child.DesiredSize;
                    //Create a bounding sphere around the element for hittesting against the current frustum
                    //This size is not entirely right... size we have is screen size but we use the world size. 
                    //*.008 seems to roughly fit as conversion factor for now
                    var box = new BoundingSphere(direction, (float)Math.Max(size.Width, size.Height) * .008f);
                    if (viewFrustum.Contains(box) != ContainmentType.Disjoint) //partially or fully inside camera frustum
                    {
                        Vector3 projected = Project(direction);
                        if (!float.IsNaN(projected.X) && !float.IsNaN(projected.Y))
                        {
                            //Arrange element centered on projected coordinate
                            double x = projected.X - size.Width * .5;
                            double y = projected.Y - size.Height * .5;
                            child.Arrange(new Rect(x, y, size.Width, size.Height));
                            continue;
                        }
                    }
                }
                //if we fall through to here, it's because the element is outside the view,
                //or placement can't be calculated
                child.Arrange(new Rect(0, 0, 0, 0));
            }
            return finalSize;
        }
        else
            return base.ArrangeOverride(finalSize);
    }

Congrats on making it this far - because we are actually pretty much done! All there’s left to do is put it all together, reset the used orientation parameters when needed and define the attached property for use on the child elements. And I already did all that for you!

You can download the control library together with a sample app here. There’s both a Windows Phone and Windows 8 Metro version included. You can use the code as you like, but if you improve on it, I would LOVE to hear about it. I didn’t include a camera background in the samples, so I’ll let that bit be up to you to add (you can use a VideoBrush as the background for that).

Note that for Windows Runtime, I have only tested this on the Samsung Build Tablet. I have no idea how this behaves on a device without a motion sensor (probably crashes, but let me know in the comments below). Also the Build tablets have a buggy fusion sensor firmware, which means that readings are fired VERY infrequent, making it almost useless for anything but rough testing.

Also note that the compass sensor can get very confused when you are inside - especially if the building has a metal frame construction. If directions seem off, try going outside away from anything that confuses the magnetic field.

Tags:

Windows Phone | Windows Runtime

WinRT vs. Silverlight - Part 8 - What other people are blogging

by Morten 6. December 2011 18:27

See intro blogpost here.

Over the last few months several other people have been writing blog posts covering the transition from WPF/Silverlight/WP7 to WinRT. Below are some of the ones I’ve stumbled upon.

Colin Eberhardt - XAMLFinance - A Cross-platform WPF, Silverlight and WP7 Application
An app that reuses code across 3 different XAML platforms and compiles for all of them. A great example that this can be accomplished. Also make sure to check out Colin’s blog for more WinRT goodness.

Jeffrey Richter - Core .NET Type usable from a Metro Styl Application
A list of the “standard” .NET types that are available in WinRT as well.

Andy’s blog - Physics Games: Multi-targeting Windows 8 and Windows Phone 7
Andy goes through building a physics-based game for both Windows 8 and Windows Phone 7. Also check out his Physics Helper library for WinRT.

Tim Greenfield - “WinRT Genome Project”
Visual comparison of how much overlay Silverlight 5 and WinRT has.

Tim Greenfield - Silverlight 5 vs. WinRT comparison
A follow-up to the link above with a comparison of namespaces, members, types and differences between SL5 and WinRT. This is an amazing list if you want to get into the details when reusing code between the two frameworks.

Pontus Wittenmarks’s - 10 tips about porting Silverlight apps to WinRT/Metro style apps (Part 1)
A quick list of of tips when porting from Silverlight to WinRT.

Petzold Book Blog - Windows 8 Dependency Property Strangeness
Talks about some of the issues with Dependency Properties in Windows 8. I already briefly touched on this, but this goes a lot more in-depth.

If I find more, I’ll add them here, or feel free to mention any other good resources in the comments below.

Tags:

Windows Runtime | WPF vs Silverlight | Windows Phone | WPF | Silverlight

About the author

Morten Nielsen

Silverlight MVP

Morten Nielsen
<--That's me
E-mail me Send mail

Twitter @dotMorten 

Disclaimer

The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.

© Copyright 2005-2014

Month List

Recent Comments

Comment RSS