“Wow, this is some Minority Report level stuff” said one marketer who tried the Microsoft PixelSense table at Code Crew’s development center. I didn’t know it at the time, but in fact director Steven Spielberg acknowledged that the concept for Minority Report display technology came from consultation with Microsoft during the making of the 2002 sci-fi classic.

The marketer’s reaction is typical of what happens when people see PixelSense, and it’s precisely that quality that has me excited about PixelSense as a social marketing device. It attracts attention, provokes engagement, and gets people talking, all without really trying.

Code Crew recently had the opportunity to develop a branded app for Microsoft PixelSense (known as the Surface until June 2012). Six developers worked to produce the app in under a month. And while our NDA prevents us from providing project and client particulars, the platform impressed me enough that I’ve decided to write an introductory blog on the topic.

PixelSense is an interactive tabletop device with platform software provided by Microsoft and hardware, called the SUR40, provided by Samsung. Here is what one form of the device looks like:

Fig 1. Microsoft PixelSense 2.0 running on the Samsung SUR40. It’s a lot more fun than this picture suggests.

It might look like an oversized tablet computer on legs, but it’s more than that for the following reasons:

• It’s social. It responds to as many as 50 touch points at once, so multiple users can interact with the same device, and each other, at the same time. So it’s social in the old fashioned sense because it encourages people to interact with each other as well as the device. A bit like playing a board game with your friends or family.
• It can see. Its surface is light sensitive so it can perceive not only touch, but all manner of physical objects placed on it.

[pixelsenseblogsad]

PixelSense apps are custom coded and run on a 64-bit version of Windows 7 Professional for Embedded Systems. Applications for PixelSense are written in Windows Presentation Foundation (WPF) or XNA using Microsoft Surface 2.0 Software Development Kit (SDK). While the development techniques for PixelSense are similar to other Microsoft products such as Windows Phone or X-Box, the special SDK has its unique requirements so previous experience with PixelSense development will make for a much smoother project. We were lucky enough to have one developer familiar with the SDK who could advise other members of the team.

The hardware part of the PixelSense equation is the Samsung SUR40. It is a table that features a 40-inch wide LED back-lit interactive screen. It is only 4 inches thick, which allows it to be used as a tabletop with the supplied legs, embedded in some other piece of furniture, or as a wall mounted display (like a whiteboard).

The screen can provide HD video at 1080 lines of vertical resolution and progressive scan. It has an aspect ratio of 16:9 and a 1920 x 1080 pixel resolution. It is covered with Gorilla Glass to protect it in public spaces.

The SUR40 hardware runs on an AMD Athlon II X2 Dual-Core CPU at 2.9 GHz, AMD's Radeon HD 6700M Series GPU, and a 320 GB hard drive. That’s enough power to run PixelSense apps.

PixelSense allows the device to see what users are doing without using an actual camera. It sees via infrared sensors that come out of each and every pixel on the screen. Unlike the touch screens we use in tablets and phones, this device can register information from hand gestures and physical objects on up to 50 points on the screen at the same time. And it can be programmed to respond to what it sees, whether that’s a fingertip, a hand, or a beer bottle.

Fig 2. PixelSense UI

This provides plenty of room to build highly interactive and informative content that can be accessed and absorbed in a tactile way by multiple users. The highly interactive nature of the interface, and the ability to intuitively move, throw, and manipulate virtual objects, is just plain fun. And the fun is infectious which draws people to the device.

PixelSense can also be used to enhance discussions by providing information, discussion points, and tools to visualize and “tactilize” conversations. For example, PixelSense was used by MSNBC on air during the 2008 US Presidential Election to display election maps and illustrate trends.

PixelSense is a lunge forward from the now familiar graphical user interface (GUI) to the Natural User Interface (NUI) The idea is to remove the artificial access points to digital content, like the mouse and keyboard. Instead, touch interaction in a social context makes the digital environment feel less constrained. PixelSense is designed to make the control side of the user experience virtually invisible.

What table can tell you what’s on it? Besides the rich graphics, video, and touch interaction, PixelSense provides an added new feature. Since it has as many “eyes” on its screen, it can recognize and be programmed to respond to different types of objects you put on top of it. This starts from our hands, and an almost endless range of hand-gestures and positions. But objects can also be seen.

Fig 3. PixelSense can see physical objects and printed patterns, and be programmed to respond.

A credit card could be put on it to run an automatic transaction. Place your drink on it and if the shape of the glass that contains it is unique, or contains a printed image called a “blob,” PixelSense could figure out what you’re having and show you some interesting facts connected to that drink. That could be great to discuss with the people you’re sharing the table with.

I see three main limitations for PixelSense:

1. It’s light and heat sensitive. The ability of PixelSense to “see” is also the source of its biggest limitation. Light shining directly on it, especially warm light with a high infrared component, confuses the system and will cause windows to open and virtual objects to move. Having a bright light shining directly in our eyes is disorienting for humans and is also disorienting for PixelSense. The solution to this problem is indirect, soft lighting. Samsung publishes the “SUR40 for Microsoft Surface Venue Readiness Guide” which documents how to set up the SUR40 to avoid this problem.
2. It’s design paradigm requires some adjustment. PixelSense offers a social user experience with direct manipulation of virtual objects. Most designers have experience ether with a single user looking at a smallish screen and manipulating objects with a keyboard and mouse, or, in the case of mobile, a single user directly manipulating virtual objects on a small screen. A responsive tactile and broad visual environment like the one provided by PixelSense is in danger of being dumbed down to a plain, unexciting solution because it is unfamiliar. Microsoft publishes the “Surface 2.0 Design and Interaction Guide” to help designers exploit the potential of PixelSense.
3. It’s not free. Although costs vary by market, the current retail price of the SUR40 is $8,400 per unit in the US. Add to that the cost of app development and, while the price is not exorbitant, deploying multiple units is expensive enough that the opportunity to generate buzz, attract customers, and leave them with a positive customer experience needs to be likely enough to justify the investment. In other words, don’t just do it because it’s cool, do it because it addresses your marketing objectives.

From a marketer’s perspective, I see three main advantages that PixelSense offers:

1. It’s a digital product that engages people in the physical world. You can put it in a retail environment, an office, or public space of any type and customers and prospective customers can and will engage with it.
2. It’s fun. The highly intuitive, visually attractive, and tactually engaging nature of the interface makes it fun to use. It generates consistently positive customer experiences and some of the magic rubs off on the sponsoring brand.
3. It generates buzz. When people hear about it they want to see it. When people see it, they want to try it. When people try it, they can’t help talking about. It’s a magnet for attention, customer engagement, and word of mouth buzz

PixelSense is worth considering as part of any marketing campaign that needs to reach out to customers in the physical world.

Samsung SUR40 for Microsoft Surface Venue Readiness Guide

Microsoft Surface 2.0 Design and Interaction Guide

Microsoft PixelSense Web Site

I was catching up with a creative director friend of mine yesterday in her favorite French bistro, when she paused to reflect on the last few years and said “I never would have imagined you’d be doing what you’re doing now when I met you four years ago.” She knew me as a technology analyst for Info-Tech Research Group, covering mobile/web design, development, and outsourcing. We were discussing a significant change in my career that I want to share with you, since it may also provide an opportunity for some mutual benefit.

As an industry analyst, I’d noticed the following marketing trends:

• Marketing budgets have been steadily drifting towards digital approaches and now also towards mobile.
• It is sometimes difficult for agencies, whether digital boutiques or full-service shops, to get the technical talent they need, when they need it, at the price that keeps their pitches competitive.
• Some agencies responded to this challenge by outsourcing digital and mobile development. In some cases it’s been very successful. In other cases it’s been problematic, with problems often related to communication difficulties or cultural misunderstandings.

In the same time period I covered app/web development outsourcing to offshore locations like Latin America, India, and the Philippines. During field research in the Philippines I was surprised by the high caliber of technical talent, strong English communication skills, and cultural affinity with the West. It occurred to me that these resources could be used to provide agencies and media teams the talent they are looking for. So after consultation with ad industry leaders, and drawing on extensive research and my hands-on experience in software development, I launched Code Crew.

Code Crew is a mobile and digital development shop offering a white label service to ad agencies, media teams, and creative boutiques. Our development center is in Manila and we use local production analysts to help define requirements and manage projects. Services include the production of mobile and digital marketing materials such as branded mobile apps, mobile and conventional web sites, HTML5 ads, kiosk apps, PixelSense apps, etc. We already have a substantial portfolio and a solid team with deep experience (see Code Crew).

Moving from Canada to Manila to launch Code Crew was certainly not something I’d imagined doing back then. But it’s been a fascinating experience and I’d love to talk to you about it, and the benefits it could bring to your agency or media team, either online or face-to-face during our roadshow.

We’ve organized a roadshow to meet clients, colleagues, and prospective clients. It’s a chance for me to learn about new ways people are using mobile and digital marketing technology, share the market intelligence I’ve collected, and find opportunities that are mutually rewarding for Code Crew and our agency partners.

Below is the schedule and I will also attend Advertising Week in New York, so that’s another opportunity to connect:

• London: Sept 3–6
• Toronto: Sept 24–28
• New York: Oct 1–5 (at Advertising Week), Oct 8–12 (office meetings)
• San Francisco: Oct 15–17

If that sounds like something you’d like to do, please contact my Communications Director, Anton Estrada with a few times and dates that are convenient for you in the time frames above and he’ll organize a meeting.

Anton Estrada, Communications Director

1 800 334 4904 ex 800