Building A Powerful Platform for Windows Store Mapping apps

When Microsoft announced WinRT at the Build conference in Anaheim, I instantly started researching and prototyping what this new platform could mean for the company I’m working for. The promise of integrating legacy native code with XAML and .NET seemed to finally be the exactly what I’ve been looking for. Also the tight integration between XAML and DirectX, something which is really complex to get working in WPF was intriguing, since we were often hitting the limit of what XAML could do.

We have a huge amount of native code that does a lot of advanced spatial analysis, advanced map rendering, spatial databases, etc. Even better was that most of it is written in a cross-platform way using C++ and was already running on Windows Classic, Linux, iOS, Android and several other platforms.

In hindsight I’m really amazed how quickly this work can go. Granted a lot of time was spent on researching, prototyping, ‘socializing the idea’ etc, but after we had the bases right, we were actually able to move very fast, and in less than 3 months create a whole new SDK geared for the Windows Store (albeit beta).

The things that made this go so fast was:

  1. We had lots of C++ code that was already written to be compiled cross-platform, so most of the development time was spent on exposing this via the WinRT interface and C++/CX.
  2. We chose to build a hybrid SDK based on both C++ and C#. This enabled us to port large amount of our existing C# code from our WPF and Silverlight SDKs. It also allowed us to not be limited by the inheritance limitations that WinRT has by simply creating .NET class libraries rather than WinRT class libraries, which in turn greatly simplifies the API for other developers.

Things that set us back:

  1. Our rendering engine only supported DirectX 9 and OpenGL. Windows Store apps require DirectX 11, which is quite different from v9, so a lot of work had to be done there, because we wanted to do it in a way that wasn’t being hurt by the least common denominator (ie. if DX11 supports a feature that DX9 or OpenGL doesn’t, it shouldn’t hold us back from using it). In the end, our map rendering engine became better because of it for all the platforms.
  2. The documentation on SurfaceImageSource (the glue behind DirectX and XAML) was very limited.
  3. Some time was spent on making sure the native code passes certification, although not too bad.

Several people both within the company, from Microsoft, MVPs etc has been extremely helpful getting us through those setbacks. Thank you! You know who you are (and yes I owe you beer :-)

So enough about that. Instead, I would really encourage you to go download our SDK. It’s actually free! Just go to our developers siteand hit the download button. You’ll be required to register/sign in – don’t worry – as far as I know we don’t spam :-)

Grab the download, install it, and create a new C# or VB.NET Windows Store app. Add a reference to the ArcGIS Runtime SDKs, set the build target to x86, x64 or ARM (AnyCPU won’t work since this has cool C++ code in its guts).

image

And now code away. There’s a few samples on how to get started with the API as well as a library reference on the developers site.  We know the documentation is a little slim at this point – we’re hard at work improving that. However we do have a getting started tutorial here: http://developers.arcgis.com/en/windows-store/guide/add-a-map-to-your-app.htm

In addition we have a set of samples in the form of a sample browser available today on GitHub: https://github.com/Esri/arcgis-samples-winstore

There’s also a toolkit with some controls here that you are encouraged to fork and contribute to: https://github.com/Esri/arcgis-toolkit-winstore

I did a short introduction to the API at our plenary about 4 mins into this video:

You can download the source code for the sample I built on stage on my github account here: https://github.com/dotMorten/WinStore-Routing-Sample

Go download and code away today, and ask questions and give us feedback in the forum.

Also please go read the official SDK release announcement here: http://blogs.esri.com/esri/arcgis/2013/03/25/arcgis-runtime-sdk-for-windows-store-apps-beta-is-available/

Submitted my first WinPhone 7 Application

imageI’ve just submitted my first application to the Windows Phone Marketplace. It’s a very simple app that creates a “Guide Post” with signs on a pole showing distances and direction to any point in the world. You have probably seen these at various sites all over the world, or in the TV Show “M*A*S*H”.

It’s all built in Silverlight, and uses PlaneProjection to give give the signs a 3D effect. You use your finger to slider over the screen to rotate it.

In addition to that there’s a Map where you can view your current location and the great circle line (ie. shortest path) between you and the points of interest. This is of course using the ESRI ArcGIS API for Windows Phone that we just released.

Below is a few screenshots and a video clip of the app in action.

screen_12-1-2010_8.51.30.996screen_12-1-2010_8.50.48.336
screenshot_12-2-2010_12.26.46.864screenshot_12-2-2010_12.27.59.830

GuidePost Screen Capture

It was a great fun little app to build and only took a few evenings to get done. You can download the app to your phone from this link: http://bit.ly/WP7GP

ESRI ArcGIS Silverlight/WPF API released

Last week we finally released v1.0 RTW of our Silverlight/WPF API. Today at our User Conference in San Diego, Jack Dangermond announced, that it even will be free for non-commercial use. The API will run on both v2 and v3 of Silverlight.

Furthermore today we released the entire source code for the toolkit assembly on CodePlex: http://esrisilverlight.codeplex.com

Also today at the Windows Partner Conference, we announced MapIt, a mapping solution running on a combination of SQL Server 2008, IIS and Silverlight/WPF and also with integration into Sharepoint. You can download a free 60-day evaluation of MapIt today. I encourage you to go see some of the demonstration videos – especially if you are new to GIS, this video could be an eye-opener to what information is hidden in your existing business intelligence.

If you are at the ESRI User Conference this week, come by the Silverlight Island in the showcase area and talk some Silverlight. I’ll be there most of the time Tuesday through Thursday. I’ll also be joining Rex Hansen and Art Haddad at the .NET SIG Tuesday evening, and the Silverlight intro sessions Wednesday and Thursday. Also watch Rich’s Demo Theater session which will demonstrate the ESRI Map Web Part for SharePoint and MapIt (the title on this session is incorrect).

Spherical/Web Mercator: EPSG code 3785

I just received an update from the EPSG mailing list:

New to Version 6.15 are (among other things): Added spherical Mercator coordinate operation method and associated CRS as seen in popular web mapping and visualisation applications.

It looks like they FINALLY added the spherical Mercator / Web Mercator projection used in Virtual Earth and Google Maps.

This is a big surprise. EPSG’s earlier statement whether to include it was this:

"We have reviewed the coordinate reference system used by Microsoft, Google, etc. and believe that it is technically flawed. We will not devalue the EPSG dataset by including such inappropriate geodesy and cartography.

Guess they changed their mind, or did they just devalue their dataset? Then again, judging from the remarks EPSG put in there, their arrogance still shines through. There´s absolute nothing wrong with using a sphere instead of a flattened sphere. Sure it's not as accurate as for instance WGS84, but then again WGS84 is not accurate either - no ellipsoid is. But we know the exact differences between the two, and as always you will need to take these things into account so I don´t see the real issue. Viisually the distortion is far less than what you would notice, and when doing area, distance and bearing calculations you would first of all never use the mercator units without taking the projection distortion into account, and if you do your calculationg in long/lat it's more or less just as easy to use WGS84 as a base for your calculations (since no datum transform is really needed).

Anyway, finally we get an official code for "Web Mercator": EPSG:3785 

Here are the details of the entry:

Full WKT with authorities (untested!):
PROJCS["Popular Visualisation CRS / Mercator", GEOGCS["Popular Visualisation CRS", DATUM["Popular Visualisation Datum", SPHEROID["Popular Visualisation Sphere", 6378137, 0, AUTHORITY["EPSG",7059]], TOWGS84[0, 0, 0, 0, 0, 0, 0], AUTHORITY["EPSG",6055]], PRIMEM["Greenwich", 0, AUTHORITY["EPSG", "8901"]], UNIT["degree", 0.0174532925199433, AUTHORITY["EPSG", "9102"]], AXIS["E", EAST], AXIS["N", NORTH], AUTHORITY["EPSG",4055]], PROJECTION["Mercator"], PARAMETER["False_Easting", 0], PARAMETER["False_Northing", 0], PARAMETER["Central_Meridian", 0], PARAMETER["Latitude_of_origin", 0], UNIT["metre", 1, AUTHORITY["EPSG", "9001"]], AXIS["East", EAST], AXIS["North", NORTH], AUTHORITY["EPSG",3785]]
  

Projected CRS
COORD_REF_SYS_CODE: 3785
COORD_REF_SYS_NAME: Popular Visualisation CRS / Mercator   
AREA_OF_USE_CODE: 3544   
COORD_REF_SYS_KIND: projected   
COORD_SYS_CODE: 4499       
DATUM_CODE:
SOURCE_GEOGCRS_CODE: 4055 (see below)
PROJECTION_CONV_CODE: 19847   
CMPD_HORIZCRS_CODE:
CMPD_VERTCRS_CODE:
CRS_SCOPE: Certain Web mapping and visualisation applications.   
REMARKS: Uses spherical development. Relative to an ellipsoidal development errors of up to 800 metres in position and 0.7% in scale may arise. Some applications call this WGS 84. It is not a recognised geodetic system: see WGS 84 / World Mercator (CRS code 3395)   
INFORMATION_SOURCE: Microsoft.
DATA_SOURCE: OGP
REVISION_DATE: 3/14/2008
CHANGE_ID:
SHOW_CRS: TRUE
DEPRECATED: FALSE

Geographic CRS:
COORD_REF_SYS_CODE: 4055
COORD_REF_SYS_NAME: Popular Visualisation CRS
AREA_OF_USE_CODE: 31262
COORD_REF_SYS_KIND: geographic 2D
COORD_SYS_CODE: 6422
DATUM_CODE: 6055 (see below)
SOURCE_GEOGCRS_CODE:
PROJECTION_CONV_CODE:
CMPD_HORIZCRS_CODE:
CMPD_VERTCRS_CODE:
CRS_SCOPE: Certain Web mapping and visualisation applications.
REMARKS: Some applications erroneously call this WGS 84. It uses a sphere with a radius having the same value as the semi-major axis of the WGS 84 ellipsoid. There is no geodetic recognition of this system.
INFORMATION_SOURCE: Microsoft.
DATA_SOURCE: OGP
REVISION_DATE: 3/13/2008
CHANGE_ID:
SHOW_CRS: TRUE
DATUM_CODE: FALSE


Datum:
Code: 6055       
Datum Name: Popular Visualisation
Datum Type: geodetic
Origin Description: Not specified in the classical sense of defining a geodetic datum.
Datum Epoch:
Ellipsoid Code: 7059 (see below)
Prime Meridian Code: 8901
Area Code: 1262
Datum Scope    : Used by certain popular Web mapping and visualisation applications.
Remarks: Not recognised by geodetic authorities.
Information Source: Microsoft.
Data Source: OGP   
Revision Date: 13-Mar-08
Change ID:
Deprecated: No


Ellipsoid:
Code: 7059
Ellipsoid Name: Popular Visualisation Sphere   
Semi-major axis (a): 6378137   
Axes units code: 9001       
Inverse flattening (1/f):
Semi-minor axis (b): 6378137   
Ellipsoid?: No   
Remarks: Sphere with radius equal to the semi-major axis of the GRS80 and WGS 84 ellipsoids. Used only for Web approximate mapping and visualisation. Not recognised by geodetic authorities.   
Information Source: Microsoft.
Data Source: OGP
Revision Date: 14-Mar-08
Change ID:   
Deprecated?: No

New SQL Server Spatial blog

It looks like Ed Katibah (Spatial Program Manager of SQL Server) finally started blogging.

His first blogpost covers some interesting changes in the latest February CTP.

There are some new methods added to the list:

  • Geometry::Filter
  • Geography::Filter
  • Geography::EnvelopeCenter
  • Geography::EnvelopeAngle
  • Geography::STDistance is no longer limited to having one of the inputs of type Point.
  • Geography::Reduce

Unfortunately I can't try these new features today, because a bug prevents you from using SQL Server 2008 on a Leap Day and the doc on MSDN doesn't cover these new methods yet, so there's not telling how these methods really work.

Ed's and Isaac Kunens blogs are a must-read if you plan on doing some serious SQL Server Spatial work.

Creating OGC conformance test map in SQL Server 2008

The Simple Features Specification v1.1.0 has a conformance test based on "Joe's Blue Lake vicinity map", which roughly looks like this:

To create the map in Microsoft SQL Server 2008, first create the tables:

-- Lakes
CREATE TABLE lakes (fid INTEGER NOT NULL PRIMARY KEY,name VARCHAR(64),shore Geometry);
-- Road Segments
CREATE TABLE road_segments (fid INTEGER NOT NULL PRIMARY KEY,name VARCHAR(64),aliases VARCHAR(64),num_lanes INTEGER,centerline Geometry);
-- Divided Routes
CREATE TABLE divided_routes (fid INTEGER NOT NULL PRIMARY KEY,name VARCHAR(64),num_lanes INTEGER,centerlines Geometry);
-- Forests
CREATE TABLE forests (fid INTEGER NOT NULL PRIMARY KEY,name VARCHAR(64),boundary Geometry);
-- Bridges
CREATE TABLE bridges (fid INTEGER NOT NULL PRIMARY KEY,name VARCHAR(64),position Geometry);
-- Streams
CREATE TABLE streams (fid INTEGER NOT NULL PRIMARY KEY,name VARCHAR(64),centerline Geometry);
-- Buildings
CREATE TABLE buildings (fid INTEGER NOT NULL PRIMARY KEY,address VARCHAR(64),position Geometry,footprint Geometry);
-- Ponds
CREATE TABLE ponds (fid INTEGER NOT NULL PRIMARY KEY,name VARCHAR(64),type VARCHAR(64),shores Geometry);
-- Named Places
CREATE TABLE named_places (fid INTEGER NOT NULL PRIMARY KEY,name VARCHAR(64),boundary Geometry);
-- Map Neatline
CREATE TABLE map_neatlines (fid INTEGER NOT NULL PRIMARY KEY,neatline Geometry);

Next step is to insert the rows:

-- Lakes
INSERT INTO lakes VALUES (101, 'BLUE LAKE',Geometry::STPolyFromText('POLYGON((52 18,66 23,73 9,48 6,52 18),(59 18,67 18,67 13,59 13,59 18))', 101));
-- Road segments
INSERT INTO road_segments VALUES(102, 'Route 5', NULL, 2,Geometry::STLineFromText('LINESTRING( 0 18, 10 21, 16 23, 28 26, 44 31 )' ,101));
INSERT INTO road_segments VALUES(103, 'Route 5', 'Main Street', 4,Geometry::STLineFromText('LINESTRING( 44 31, 56 34, 70 38 )' ,101));
INSERT INTO road_segments VALUES(104, 'Route 5', NULL, 2,Geometry::STLineFromText('LINESTRING( 70 38, 72 48 )' ,101));
INSERT INTO road_segments VALUES(105, 'Main Street', NULL, 4,Geometry::STLineFromText('LINESTRING( 70 38, 84 42 )' ,101));
INSERT INTO road_segments VALUES(106, 'Dirt Road by Green Forest', NULL, 1,Geometry::STLineFromText('LINESTRING( 28 26, 28 0 )',101));
-- DividedRoutes
INSERT INTO divided_routes VALUES(119, 'Route 75', 4,Geometry::STMLineFromText('MULTILINESTRING((10 48,10 21,10 0),(16 0,16 23,16 48))', 101));
-- Forests
INSERT INTO forests VALUES(109, 'Green Forest',Geometry::STMPolyFromText('MULTIPOLYGON(((28 26,28 0,84 0,84 42,28 26),(52 18,66 23,73 9,48 6,52 18)),((59 18,67 18,67 13,59 13,59 18)))', 101));
-- Bridges
INSERT INTO bridges VALUES(110, 'Cam Bridge', Geometry::STPointFromText('POINT( 44 31 )', 101));
-- Streams
INSERT INTO streams VALUES(111, 'Cam Stream',Geometry::STLineFromText('LINESTRING( 38 48, 44 41, 41 36, 44 31, 52 18 )', 101));
INSERT INTO streams VALUES(112, NULL,Geometry::STLineFromText('LINESTRING( 76 0, 78 4, 73 9 )', 101));
-- Buildings
INSERT INTO buildings VALUES(113, '123 Main Street',Geometry::STPointFromText('POINT( 52 30 )', 101),Geometry::STPolyFromText('POLYGON( ( 50 31, 54 31, 54 29, 50 29, 50 31) )', 101));
INSERT INTO buildings VALUES(114, '215 Main Street',Geometry::STPointFromText('POINT( 64 33 )', 101),Geometry::STPolyFromText('POLYGON( ( 66 34, 62 34, 62 32, 66 32, 66 34) )', 101));
-- Ponds
INSERT INTO ponds VALUES(120, NULL, 'Stock Pond',Geometry::STMPolyFromText('MULTIPOLYGON( ( ( 24 44, 22 42, 24 40, 24 44) ),( ( 26 44, 26 40, 28 42, 26 44) ) )', 101));
-- Named Places
INSERT INTO named_places VALUES(117, 'Ashton',Geometry::STPolyFromText('POLYGON( ( 62 48, 84 48, 84 30, 56 30, 56 34, 62 48) )', 101));
INSERT INTO named_places VALUES(118, 'Goose Island',Geometry::STPolyFromText('POLYGON( ( 67 13, 67 18, 59 18, 59 13, 67 13) )', 101));
-- Map Neatlines
INSERT INTO map_neatlines VALUES(115,Geometry::STPolyFromText('POLYGON( ( 0 0, 0 48, 84 48, 84 0, 0 0 ) )', 101));
 

And voilá... you can now for instance run all the conformance tests, but the map is actually also a good base-map for learning the various spatial operations.

If you use my SQL Spatial Query Tool, here's a query you can use to visualize the map:

SELECT neatline, 'Background' as name, 'map_neatlines' as layer, 'White' as FillColor, 'Transparent' as LineColor, 1 as LineThickness FROM map_neatlines UNION ALL
SELECT shores, name, 'ponds' as layer, 'LightBlue' as FillColor, 'Blue' as LineColor, 1 as LineThickness FROM ponds UNION ALL
SELECT boundary, name, 'forests' as layer, 'Green' as FillColor, '#ff005500' as LineColor, 4 as LineThickness FROM Forests UNION ALL
SELECT shore, name, 'lakes' as layer, 'Blue' as FillColor, 'Transparent' as LineColor, 2 as LineThickness FROM lakes UNION ALL
SELECT boundary, name, 'named_places' as layer, '#00ffff99' as FillColor, 'Brown' as LineColor, 2 as LineThickness FROM named_places UNION ALL
SELECT centerline, name, 'streams_outline' as layer, 'Blue' as FillColor, 'Blue' as LineColor, 5 as LineThickness FROM streams UNION ALL
SELECT centerline, name, 'streams_fill' as layer, 'LightBlue' as FillColor, 'LightBlue' as LineColor, 3 as LineThickness FROM streams UNION ALL
SELECT centerline, name, 'road_segments' as layer, 'LightGreen' as FillColor, 'Black' as LineColor, num_lanes*2 as LineThickness FROM road_segments UNION ALL
SELECT centerlines, name, 'divided_routes' as layer, 'LightGreen' as FillColor, '#aaff0000' as LineColor, num_lanes*2 as LineThickness FROM divided_routes UNION ALL
SELECT footprint, address, 'buildings_footprint' as layer, 'Black' as FillColor, 'Transparent' as LineColor, 1 as LineThickness FROM buildings UNION ALL
SELECT position.STBuffer(0.5), address, 'buildings_position' as layer, 'Red' as FillColor, 'Transparent' as LineColor, 1 as LineThickness FROM buildings UNION ALL
SELECT position.STBuffer(1), name, 'bridges' as layer, '#550000ff' as FillColor, 'Black' as LineColor, 1 as LineThickness FROM bridges

Which will generate a map like this: 

Download SQL queries (1.71 kb)

The community works

It looks like Microsoft listened to the community and plans on switching latitude and longitude ordering for WKT and WKB:
http://blogs.msdn.com/isaac/archive/2007/12/27/latitude-longitude-ordering.aspx

This change will only be for WKT and WKB. That does make some sense, since the ordering is related to the SRID which is not included in the WKT and WKB, thus making it impossible to decipher without any prior knowledge. It would still be nice if the two geometry models shared a more common interface for anything that is remotely similar, but this is a good start.

I’m actually starting to like working with the geography type (to begin with I hated the idea of having two different although similar geometry models). It’s fun to do buffers on points and see that they don’t look circular on a projected map, because they are, well… projected. Showing the same circles on a 3D globe reveals that they are in fact perfectly circular. It makes distance queries so much easier than for instance using this approach.

400km buffer around Copenhagen

One thing that still bugs me about the geography datatype though… why is the linear unit type that is used for calculating area, length, buffers etc. declared as part of the SRID (which in this case is always geographic) ? This requires one to always first have to look up the linear unit in the spatial_ref_sys table for each returned row (since homogeneous SRIDs aren’t enforced on the table) and divide by the factor returned to ensure that we know the unit we get back. In my mind a linear unit should never be related to a geographic coordinate system. After all, that’s why we only define linear units in projected and geocentric coordinate systems where they makes sense.

It would be nice if the methods either took a unit type as input (although that would break with the common interface I just wished for) or simply just always outputs the results in SI units (then it would just be a matter of multiplying with a constant well-known conversion factor later if you want to display it in for instance miles, thus separating code from presentation). Luckily most of Katmai’s SR definitions use meters as default, but you never know if someone chose to change the conversion factors in the system table so you will always have to check first.

Shapefile to SqlServer 2008 to WPF

I've been playing with the spatial support of SQL Server 2008 today, but quickly hit two obstacles:

  1. How do I get data into the database.
  2. How do I visualize a query (apart from the Well-Known Text output).

Surprisingly there aren't really any good (free) tools out there yet to upload for instance shapefiles to Katmai. I pulled out some old code I had lying around and threw a small tool together that can upload a shapefile to the database either as geometry or geography. Katmai is pretty picky when it comes to the geography datatype, so make sure your shapefiles contain valid geometry and has vertices in the valid domain, or use the geometry format. You can download the tool below.

Next obstacle was visualizing the data. Playing with the spatial queries and not beeing able to see the results on a map quickly got boring. I haven't had a chance to play much with WPF, so I thought this was a good occation to play with it. Based on this MSDN article I managed to fairly quickly make a little tool that takes a SQL query as input and outputs the results in a WPF control. At the moment it only works with polylines and polygons - Points are simply ignored (tip: use STBuffer() on points to see them). It will look for the first column that contains a SqlGeometry or SqlGeography datatype and put that on the map. The remaining columns are shown in the box to the right when you hover over the shapes with the mouse.

The two tools are not perfect in any way (and there's no bug tracking support here :-)), but they provide a quick way to play with the spatial queries. Both tools requires Windows Authentification to access the sql server and runs on .NET 3.5. Furthermore the query tool uses the Microsoft.SqlServer.Types.dll that comes with installing Katmai (to parse the results), so for now you'll need to run it from a computer that has Katmai installed.

Read more and download SqlSpatialTools

New Virtual Earth release

I just visited http://maps.live.com and it looks like they finally put the new release out there. A revamped design and Birds Eye View imagery inside the 3D viewer. You can now also create animations using your collections.

Nyhavn, Copenhagen - A place I missed a lot this summer

Another new feature is the new "Create model" button in the collection editor when you are in 3D mode. Using it will prompt you to download "3dvia" enabling you to model your own buildings and insert them into VE. It looks somewhat similar to SketchUp used for the same thing in Google Earth. A neat feature is that you get the map as a background to assist you in your modelling, and comes with a bunch of default textures as well: