“Wow, this is some Minority Report level stuff” said one marketer who tried the Microsoft PixelSense table at Code Crew’s development center. I didn’t know it at the time, but in fact director Steven Spielberg acknowledged that the concept for Minority Report display technology came from consultation with Microsoft during the making of the 2002 sci-fi classic.

The marketer’s reaction is typical of what happens when people see PixelSense, and it’s precisely that quality that has me excited about PixelSense as a social marketing device. It attracts attention, provokes engagement, and gets people talking, all without really trying.

Code Crew recently had the opportunity to develop a branded app for Microsoft PixelSense (known as the Surface until June 2012). Six developers worked to produce the app in under a month. And while our NDA prevents us from providing project and client particulars, the platform impressed me enough that I’ve decided to write an introductory blog on the topic.

PixelSense is an interactive tabletop device with platform software provided by Microsoft and hardware, called the SUR40, provided by Samsung. Here is what one form of the device looks like:

Fig 1. Microsoft PixelSense 2.0 running on the Samsung SUR40. It’s a lot more fun than this picture suggests.

It might look like an oversized tablet computer on legs, but it’s more than that for the following reasons:

• It’s social. It responds to as many as 50 touch points at once, so multiple users can interact with the same device, and each other, at the same time. So it’s social in the old fashioned sense because it encourages people to interact with each other as well as the device. A bit like playing a board game with your friends or family.
• It can see. Its surface is light sensitive so it can perceive not only touch, but all manner of physical objects placed on it.

PixelSense apps are custom coded and run on a 64-bit version of Windows 7 Professional for Embedded Systems. Applications for PixelSense are written in Windows Presentation Foundation (WPF) or XNA using Microsoft Surface 2.0 Software Development Kit (SDK). While the development techniques for PixelSense are similar to other Microsoft products such as Windows Phone or X-Box, the special SDK has its unique requirements so previous experience with PixelSense development will make for a much smoother project. We were lucky enough to have one developer familiar with the SDK who could advise other members of the team.

The hardware part of the PixelSense equation is the Samsung SUR40. It is a table that features a 40-inch wide LED back-lit interactive screen. It is only 4 inches thick, which allows it to be used as a tabletop with the supplied legs, embedded in some other piece of furniture, or as a wall mounted display (like a whiteboard).

The screen can provide HD video at 1080 lines of vertical resolution and progressive scan. It has an aspect ratio of 16:9 and a 1920 x 1080 pixel resolution. It is covered with Gorilla Glass to protect it in public spaces.

The SUR40 hardware runs on an AMD Athlon II X2 Dual-Core CPU at 2.9 GHz, AMD’s Radeon HD 6700M Series GPU, and a 320 GB hard drive. That’s enough power to run PixelSense apps.

PixelSense allows the device to see what users are doing without using an actual camera. It sees via infrared sensors that come out of each and every pixel on the screen. Unlike the touch screens we use in tablets and phones, this device can register information from hand gestures and physical objects on up to 50 points on the screen at the same time. And it can be programmed to respond to what it sees, whether that’s a fingertip, a hand, or a beer bottle.

Fig 2. PixelSense UI

This provides plenty of room to build highly interactive and informative content that can be accessed and absorbed in a tactile way by multiple users. The highly interactive nature of the interface, and the ability to intuitively move, throw, and manipulate virtual objects, is just plain fun. And the fun is infectious which draws people to the device.

PixelSense can also be used to enhance discussions by providing information, discussion points, and tools to visualize and “tactilize” conversations. For example, PixelSense was used by MSNBC on air during the 2008 US Presidential Election to display election maps and illustrate trends.

PixelSense is a lunge forward from the now familiar graphical user interface (GUI) to the Natural User Interface (NUI) The idea is to remove the artificial access points to digital content, like the mouse and keyboard. Instead, touch interaction in a social context makes the digital environment feel less constrained. PixelSense is designed to make the control side of the user experience virtually invisible.

What table can tell you what’s on it? Besides the rich graphics, video, and touch interaction, PixelSense provides an added new feature. Since it has as many “eyes” on its screen, it can recognize and be programmed to respond to different types of objects you put on top of it. This starts from our hands, and an almost endless range of hand-gestures and positions. But objects can also be seen.

Fig 3. PixelSense can see physical objects and printed patterns, and be programmed to respond.

A credit card could be put on it to run an automatic transaction. Place your drink on it and if the shape of the glass that contains it is unique, or contains a printed image called a “blob,” PixelSense could figure out what you’re having and show you some interesting facts connected to that drink. That could be great to discuss with the people you’re sharing the table with.

I see three main limitations for PixelSense:

1. It’s light and heat sensitive. The ability of PixelSense to “see” is also the source of its biggest limitation. Light shining directly on it, especially warm light with a high infrared component, confuses the system and will cause windows to open and virtual objects to move. Having a bright light shining directly in our eyes is disorienting for humans and is also disorienting for PixelSense. The solution to this problem is indirect, soft lighting. Samsung publishes the “SUR40 for Microsoft Surface Venue Readiness Guide” which documents how to set up the SUR40 to avoid this problem.
2. It’s design paradigm requires some adjustment. PixelSense offers a social user experience with direct manipulation of virtual objects. Most designers have experience ether with a single user looking at a smallish screen and manipulating objects with a keyboard and mouse, or, in the case of mobile, a single user directly manipulating virtual objects on a small screen. A responsive tactile and broad visual environment like the one provided by PixelSense is in danger of being dumbed down to a plain, unexciting solution because it is unfamiliar. Microsoft publishes the “Surface 2.0 Design and Interaction Guide” to help designers exploit the potential of PixelSense.
3. It’s not free. Although costs vary by market, the current retail price of the SUR40 is $8,400 per unit in the US. Add to that the cost of app development and, while the price is not exorbitant, deploying multiple units is expensive enough that the opportunity to generate buzz, attract customers, and leave them with a positive customer experience needs to be likely enough to justify the investment. In other words, don’t just do it because it’s cool, do it because it addresses your marketing objectives.

From a marketer’s perspective, I see three main advantages that PixelSense offers:

1. It’s a digital product that engages people in the physical world. You can put it in a retail environment, an office, or public space of any type and customers and prospective customers can and will engage with it.
2. It’s fun. The highly intuitive, visually attractive, and tactually engaging nature of the interface makes it fun to use. It generates consistently positive customer experiences and some of the magic rubs off on the sponsoring brand.
3. It generates buzz. When people hear about it they want to see it. When people see it, they want to try it. When people try it, they can’t help talking about. It’s a magnet for attention, customer engagement, and word of mouth buzz

PixelSense is worth considering as part of any marketing campaign that needs to reach out to customers in the physical world.

Samsung SUR40 for Microsoft Surface Venue Readiness Guide

Microsoft Surface 2.0 Design and Interaction Guide

Microsoft PixelSense Web Site