Perhaps the most important ingredient in the success of a software development team is the raw talent of its team members. This is supported by the so called 10x rule, which states that the strongest developer is about 10 times as productive as the weakest. This huge variation in productivity has long been observed among development managers and is also backed by a significant body of research [1].

The 10x rule supports the view that no other decision is as important as who you hire. While you can nurture talent and help the good become great, transforming poor performers into coding rock stars is completely impractical in a development team with deadlines and deliverables. And all development teams have deadlines and deliverables.

The 10x rule also supports the view that you have to filter out many of those on the low end of the 10x spectrum to find the gems, or even the 5x supporting players. How can this be done? One of the best ways to know if someone has mastered a skill is to have them perform it and evaluate the results. If you need a musician, there’s the audition. If you need a chief, there’s t he taste test. If you a need a software developer, there’s the coding test.

Coding tests are nothing new and have many variations: simple paper exercises, whiteboard algorithm problems, or mini development projects that can be completed in an hour. More recently, startups like HackerRank, TestDome, and Codility offer cloud-based testing solutions that have the potential to make pre-interview testing faster, easier, and more effective.

This post reviews Codility from the CTO or software development manager’s standpoint. It’s based on my experience using Codility to evaluate the performance of a software engineering team I’d been brought in to manage, improve, and expand while CTO of a large digital marketing company. I began by testing existing team members and then tested prospective new hires.

Codility is a cloud-based developer evaluation tool that:

• Allows development managers to create online coding tests in most popular languages by combining carefully designed standardized programming tasks.
• Provides developers with an online test environment to read the task, code a solution, and run that solution much as they would in a standard code editor.
• Automatically evaluates test session results for correctness and performance against multiple test cases.
• Compares results with those of thousands of other test-takers that have completed the same test.
• Reports test session results to the development manager and optionally provides feedback to the developer.

Ten years ago, Greg (Grzegorz) Jakacki was facing a problem that many development managers struggle with: only a small proportion of applicants for programming jobs could actually program to a level even close to what he required. Using principles inspired by coding competitions like the International Olympiad in Informatics [2], where he had won a bronze medal, Greg and his colleagues developed automated testing software that filtered out 90% of applicants, allowing Greg to focus his interviews on the better skilled developers who passed the test.

Greg then pitched the idea of a web based version of his software at the 2009 Seedcamp competition in London. He won, and Codility was born. Today, Codility has offices in London, Warsaw, and San Francisco that support 200 corporate customers in over 120 countries. Since 2009, Codility has run over 2 million tests, which provides a significant dataset with which to compare new test takers.

Codility is primarily used as a pre-interview test for software developers. It can also be used to evaluate performance of an existing software development team in order to facilitate professional development or restructuring.

For Developer Recruitment. Codility is not intended to be a one stop evaluation tool. Rather, it filters out poorly performing developers so the hiring manager can focus on strong candidates. Additional interviews and evaluations are required. However, Codility can greatly reduce the number of interviews you must perform to build your team, it’s not uncommon to filter out 80% to 90% of applicants in the initial testing. And because you can compare a candidate’s performance to thousands of developers worldwide, test results are more meaningful than if you can only compare to the developers within your organization.

For Development Team Restructuring. Imagine that you’ve been brought in to fix a poorly performing software team. You find issues with product definition, development process, team tooling, etc. The team may also have some underperforming members who are reducing overall productivity and impacting moral. But, given the less than optimal work conditions, it’s hard to be sure if they simply don’t have the support they need to thrive, or if they don’t have the raw talent. Testing all team members can help you understand the team’s talent landscape and take action to improve it. It also helps you understand how test scores translate into actual developer quality and productivity, providing a meaningful metric for new candidates.

Criticisms. The primary criticism of Codility and controlled testing of software developers in general is that the tests are too academic and do not measure what’s needed to succeed as a real life developer. In real life there is teamwork, Internet resources, and time to mull things over. In real life you might do it differently than what you learned in school, as long as it works and works on time.

While all this is true, it does not diminish the value of testing. Remember that the purpose of testing is simply to create a short list of candidates to interview and further evaluate, not to rank all candidates. While a candidate with a score of 85% may or may not be a better choice than one with a score of 75%, ones that do not pass at all are almost certainly a bad choice. They are unlikely to contribute more to the team then they require from it, so even if they are nice people, they will not be good team members.

I’ll walk you through the basics steps in using Codility, and later give you some pointers of how to make it work for you.

1. Name your test and select the language or languages you want in which developers will code. Choose any combination of C, C++, C#, Go, Java, JavaScript, Lua, Objective-C, Pascal, PHP, Perl, Python, Ruby, Scala, and VB.NET. If you choose more than one language, the test takers can select whichever they prefer.
   

   
   

2. Select a few tasks for the test. Three is the most common number for a test, since it allows you to vary the type of tasks without exhausting the test taker. Tests are categorized by difficulty (Easy, Medium, Hard) and type (coding, algorithm, bug fixing, and SQL). Each task typically requires the developer to write or modify a short program (usually 10–20 lines).
   
3. Administer tests sessions. These can be done onsite or emailed to the test taker for offsite completion. The test-taker reads the problem and initial code, then works in the provided online editor (or optionally in their preferred editor, pasting it in when complete). They run the code and if it compiles and provides the expected result for the example test case, the system says “OK.” However, in order to get a 100% score, the code must run correctly and efficiently against all possible inputs. Test takers can run their solution as many times as required and submit the task when they are happy with their code.
   
4. Review task results. You’ll be notified when the test is complete and can then review the score for each task and see the actual code submitted as well as the total time spent working on it. You can run a screen-cast video of the code as it was typed to get an idea of how the developer arrived at the solution.
   

   You can also see the pass/fail results of each correctness and performance test:
   

5. Review test results. Each task is graded for correctness, and some for code performance. These task scores are averaged to produce a composite test score.



6. Compare results to all test takers. What does the test result mean? You can compare individual task scores with the scores of all test takers, usually thousands of developers. The results are displayed when you select a task. Some tasks are very easy, for good developers they are more of a warmup round. For example, 73% of test takers got 50% or more on the SqlDepartments task, while 71% got 100%.
   

   Other tests are much more difficult with almost everyone failing:
   

Here are some pointers to get the most from Codility:

1. Try the free trial to help you understand how Codility works and what the tests are like. The trial allows you to test up to 50 candidates in a 14-day period, but unfortunately you can only draw from a pool of six tasks to develop your tests. I found this to be insufficient for testing existing team members and new recruits. Additionally, since anyone can sign up for the trial without paying, developers could practise the six trial tasks before you test them. Therefore, I suggest using the free trial only for an initial walk through and for a complete evaluation pay for a month's subscription.
2. Test your existing team to better understand its strengths and weaknesses and to provide a metric against which to measure new candidates. You'll want new recruits to meet or exceed test scores of your top developers.
3. Select relevant test tasks. Test tasks should be similar to the kind of work your team does. This ensures the testing is relevant and not simply an academic exercise. A typical test will include two to four tasks and last an hour or two maximum.
4. Blend test task difficulty. Test task results on the easiest or hardest end of the spectrum are somewhat binary, with almost everyone passing the easiest tasks and almost everyone failing the hardest ones. If you choose only these, most test-takers will have the same score which will not help you choose between them. Instead, use published results to select tasks at varying levels of difficulty which can discriminate between varying talent levels. Getting the right combinations of tasks may take a few iterations of testing with your team.
5. Encourage test-takers to try a demo test. Codility publishes lessons, challenges, and demo tests for developers. Taking a demo test and reviewing the developer FAQs is a great way for developers to get comfortable with the test environment and process. This makes for more accurate test results since test anxiety will result in scores that do not reflect the test taker’s full potential.
6. Review the results. Compare task scores with Codility’s published results and against your team’s scores (if you're testing new hires). For a more detailed understanding review the analysis section to see which test cases passed or failed and to review the code. You can even play a screen video of the code as it was typed to understand how the developer arrived at the solution.
7. Passing the test is important, the exact score is not. The exact passing score you choose will vary with the position, the candidate pool, and the tasks selected. Run the same test with your team and come up with a passing threshold which you can adjust up or down if required, depending on candidate performance. For borderline test results you can review the code to make a final decision. While a minimum level of competency is vital, slight variations above the threshold may or may not be meaningful. The best developer may not be the one with the highest score, so move to other evaluation methods like the interview to determine suitability before making job offers.
8. Provide feedback, or not. Codility can optionally share complete test results with test takers. If you’re testing your team, I recommend sharing the results with individual team members and perhaps discussing them in a one-on-one. However, I don’t like sharing more than the test score with the external candidates since the detailed feedback could be shared with future test takers and give them an unfair advantage.
9. Test onsite if possible. I prefer to test onsite to ensure test takers don’t get help from a more experienced or competent friend. I’ll make an exception if a candidate is out of town, in which case a simple phone conversation about the coding choices they made during the test will reveal if the code submitted was theirs. Allowing online references can be OK, since time limits will prevent a test taker from using them for more than a supporting resource.
10. Answer questions about the tasks. I’ll make a senior developer available to answer questions during the test. Obviously, questions like “how do you solve this problem” will not be answered, but those that help the test-taker understand what is being asked of them are OK. This is especially important when testing developers who are not native English speakers.

• Free Trial: Up to 50 tests during 14-days created from 6 tasks.
• Startup Subscription: 199 USD monthly for up to 30 tests created from 86 tasks.
• Enterprise Subscription: Custom pricing for unlimited tests created from 136 tasks
• Per Test Package: Starts at 17.50 USD per test with discounts for volume.
• Discounts: One time 50% discount for startups that purchase a Per Test Package.

Used as the first step in a comprehensive strategy, Codility can increase both efficiency and effectiveness of your organization’s software developer talent evaluation. Given the importance of talent acquisition in a software team’s success, using it to find those 10x developers is a smart choice.

1. Productivity Variations Among Software Developers and Teams: The Origin of 10x. Steve McConnell, March 27, 2008
2. Application of Olympiad-Style Code Assessment to Pre-Hire Screening of Programmers, Olympiads in Informatics, Grzegorz Jakack, Marcin Kubica, & Tomasz Walen, 2011, Vol. 5, 32–43<

I recently attended the Society of Pharmaceutical and Biotech Trainers (SPBT) Annual Conference in Orlando, which featured many interesting presentations, including one by Dr. B. Price Kerfoot. Dr. Kerfoot led ground breaking research in spaced education and has recently explored game mechanics in learning. I had the pleasure of chatting with Dr. Kerfoot, Harvard Medical School professor, edtech researcher, and Qstream board member. A rough cut of our interview follows and a feature article entitled "Spaced-out learning" appears in the March 2014 issue of Training Journal.

Dr. Kerfoot’s research began by exploring two well documented effects and their influence on learning:

1. The spacing effect has been studied as far back as the 19th century by Ebbinghouse. The spacing effect shows that if you present information and then reinforce it over spaced intervals of time, it increases the acquisition of knowledge and encodes it so it’s better retained over the long term.
2. The testing effect, which shows that the act of testing increases retention. Testing in not just for evaluation, the retrieval required during testing can itself dramatically improve long-term retention.

In this video we discuss the limitations of traditional “bolus” elearning [1] and how Dr. Kerfoot’s research demonstrates that the spacing and testing effects can improve online learners’:

1. Information retention, knowledge transfer to related topics, and learning efficiency [2].
2. On the job performance [3].
3. Identification and remediation of underperforming learners [4, 5].

Video 1 of 2. B. Price Kerfoot on Spaced Education

[spaced-ed-ad]

More recently, Dr. Kerfoot studied the combination of spaced education with certain game mechanics (rules intended to make games engaging). In this video we discuss how game-like spaced education increases knowledge retention and test performance while being associated with improved course completion rates and motivation to participate in future spaced education programs [5, 6].

This video covers three main game elements:

1. Rewards. Gabe Zickerman SAPS model: Status, Access, Power, and Stuff as well as balancing intrinsic and extrinsic motivation.
2. Teams. Rewarding teams can be more effective than rewarding individuals [7].
3. Leaderboards. Designing leaderboards for create a shame-free, safe environment that motivates and engages learners.

Video 2 of 2. B. Price Kerfoot on Game Mechanics in Learning

Dr. Kerfoot’s research is part of the body of knowledge demonstrating that, while intensive training can produce a short-term spike in learning, the learning degrades over time. This forgetting curve is so consistent and predictable that it likely has a biological basis. Learning designers need to begin with the assumption that learning is forgotten over time and therefore performance based on that learning will also decline.

The spacing effect and the testing effect are powerful mechanisms to offset the forgetting curve. These principles can be designed into any learning program either by spreading out the initial learning/testing over time or by supplementing intensive training with spaced education. Game mechanics can also be integrated to enhance learner engagement and motivation.

The spacing and testing effects blended with game mechanics are well suited for mobile learning. Spaced education works best when small nuggets of information are offered over time. Small nuggets are also preferred in an mlearning context, where learners may not have more than a few minutes to study. Game mechanics will help motivate learners to engage in their mlearning, rather than having their attention captured by other temptations on the mobile device, whether Angry Birds, Facebook, or Twitter.

1. A Multi-Institutional Randomized Controlled Trial of Adjuvant Web-Based Teaching to Medical Students, Academic Medicine, B. Price Kerfoot and Harley Baker et al, March 2006.
2. Online Spaced Education Generates Transfer and Improves Long-Term Retention of Diagnostic Skills: A Randomized Controlled Trial, American College of Surgeons, B. Price Kerfoot and Yineng Fu et al, September 2010.
3. Durable Improvements in Prostate Cancer Screening from Online Spaced Education, American Journal of Preventive Medicine, B. Price Kerfoot and Elizabeth Lawler et al, November 2010.
4. Online ‘Spaced Education Progress-Testing’ of Students to Confront Two Upcoming Challenges to Medical Schools, Academic Medicine, B. Price Kerfoot and Kitt Shaffer et al, March 2011.
5. An Online Spaced-Education Game to Teach and Assess Medical Students: A Multi-Institutional Prospective Trial, Academic Medicine, B. Price Kerfoot and Harley Baker et al, October 2012.
6. An Online Spaced-Education for Global Continuing Medical Education, Annals of Surgery, B. Price Kerfoot and Harley Baker, July 2012.
7. Individual- versus Group-Based Financial Incentives for Weight Loss: a Randomized Controlled Trial, Annals of Internal Medicine, Jeffrey Kullgren and Andrea Troxel et al, April 2013.

The Experience API (xAPI) specification, also known as the Tin Can API, will move from version 0.95 to 1.0 on April 26, 2013. Once this version is released, expect support from LMS and elearning tool vendors to extend beyond the current group of early adopters.

The xAPI defines the next generation of SCORM which was originally released in January 2000 with support from the US government’s Advanced Distributed Learning (ADL) Initiative. The xAPI is built on standard Web Services and allows much more flexible tracking of a learner’s experience than SCORM. Support for mlearning is greatly enhanced and software development simplified. Most importantly, the xAPI can provide a granular view of a learner’s actions that will enable instructional designers to improve course quality based on real-world data.

Here are some excellent resources to get you up to speed on the xAPI:

• For a copy of the latest specification, see the Experience API on GitHub.
• For interviews covering both technical and pedagogical elements, see Project Tin Can.
• For an overview of its impact on instructional design, see Should Instructional Designers care about the Tin Can API?
• For inspiration about how learning can be enhanced with more data, see What we're learning from online education (TED Talk).

Stanford Research Institute’s Roy Amara had it right when he said "We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run." In line with his caution, I imagine the immediate impact of the xAPI will be minor but the long term impact will be significant. Most organizations have significant investments in the existing standard and will take time to adopt a new one, despite its immediate potential. However, as mobile devices reach near ubiquitous market penetration, corporate Cloud computing gains greater acceptance, and training professionals push for a better understanding of their learner’s experience, the xAPI will become widely implemented. When this happens learning professionals will have a richer understanding of the learner’s experience that will make training more effective at enhancing performance.

When considering a mobile marketing strategy, there are advantages and disadvantages to both mobile Web and mobile app. Review the pros and cons in this post to learn which is right for your clients or your brand.

We are in the middle of a mobile revolution. US smartphone subscribers hit 121M in October 2012 [2]. The smartphone, with its growing processing power, sophisticated camera, location awareness, mobile browser, and growing list of app functionality is changing the way we communicate and manage both our personal and business data. This has created an explosion of users leveraging the mobile Internet, with the average smartphone user spending about four hours on their phone each day [3] Add to this the tablet: a device category with almost no market share before the iPad’s introduction two years ago. Now the worldwide tablet market is expected to hit $122M US by the end of 2012 [4].

Smart marketers are looking for ways to exploit the mobile opportunity and an important question to consider is whether the best way to do it is with a mobile friendly web site, a branded native mobile app, or both.

Along with this mobile explosion is a steady increase in mobile app use. Smartphone users worldwide will download more than 45 billion apps this year, nearly twice the number of apps that were downloaded in 2011 [5]. October 2012 saw iPhone global daily app downloads surge to 5.4M, 33% higher than the previous month [6]. The week between Christmas and New Year is huge for app downloads as phones given for Christmas presents are unwrapped and loaded with the latest apps. In 2011 this period accounted for 1.2B apps downloaded globally, which is a 60% increase compared to the same period in the previous year [7]. All this downloading is having an impact on the economy. Aggregate data points valued last year’s app economy at $3.5B US, a figure expected to grow 90% by the end of 2012 [8].

How does mobile app usage compare to mobile Web usage? In the US, 47.6% of smartphone owners downloaded and used apps, compared 47.5% that surfed the Web [9]. So mobile app/web usage is roughly even. But heavy app users are also heavy Web users: 60% of US consumers who download apps monthly also accessed the mobile Web daily [10].

Another question to consider when planning mobile Web investments is the proportion of Web sites accessed from a mobile device compared to desktop or laptop computers. Web traffic from mobile devices is growing rapidly, more than doubling in the US if we compare August 2011’s 6.9% with 13.3% in August 2012 [11]. And the proportion of mobile Web is even greater in the developing world: for example: 58% for Zimbabwe, 48% for India, and 42% for Uzbekistan [12].

With the popularity of mobile apps and mobile Web roughly equal, let’s review some technical basics and consider technical and marketing merits of each.

Native mobile apps are built with hard-core programming languages like Objective C for iOS, Java for Android, and C# for Windows Phone, then compiled and distributed as a binary application and app store. However, a Web site’s front end is built with three main technologies: HTML, CSS, and JavaScript. In the early days of smartphone adoption, Web sites looked pretty much the same on a mobile device as on a desktop, but these three Web technologies have since evolved to give developers the ability create mobile friendly Web sites that make them look and feel a lot like mobile apps. If well designed they will fit a smaller screen, respond to swipe gestures, access some the phones features such as geolocation, and can store data on the phone.

In a recent mobility study I led for Info-Tech Research Group entitled “Developer Challenges and Cross-Platform Solutions” we interviewed about 50 mobile marketing executives and developers from North America, Europe, and Asia. One of the questions we asked was “What do you see as the advantages and disadvantages of mobile Web sites compared to native apps?” Figure 1 below graphs what they described as the advantages.

Fig 1. Advantages of mobile web sites vs mobile apps.

The data can be looked at from three main perspectives: that of the developer, the end user, and the marketer. From a developer’s perspective, mobile web provides cost effective cross platform access using web technologies that are already familiar, and once deployed are also easier to manage. From an end-user's perspective mobile Web provides easy access because there is no need to download an app, you just visit a Web site. The marketer’s perspective distills the main advantages of developer and end-user: Web is a less expensive and less time consuming way to get my message into mobile, with fewer barriers for the customer.

With mobile Web cheaper and easier than mobile apps, it might look like an easy decision to weave a marketing strategy based on mobile web, but we must also weigh these benefits against those available in native mobile apps.

Figure 2 below summarizes what the same Info-Tech respondents had to say about the disadvantages of mobile Web sites compared to native apps:

Fig 2. Disadvantages of mobile web sites vs mobile apps.

We can summarize the key points graphed above as follows, mobile Web sites cannot always (1) access all a phone’s features (e.g. camera), (2) be useful if there is no Internet connection, (3) provide highly satisfying speed and user experience. In addition, they do not live on a consumer’s mobile home screen, and therefore do not provide a constant reminder of the associated brand.

We can therefore conclude that although mobile Web sites are cheaper and easier, from a user or marketer’s perspective they are not usually better. It’s a case of not getting what you don’t pay for.

Packaged apps offer a compelling combination of multi-platform reach with native app performance. They are built with cross-platform development tools like Adobe PhoneGap and Sencha Touch that package up Web code in way that can interact with many phone features and be distributed through an app store like Apple’s App Store or Google’s Play.

Packaged apps consist of Web code wrapped in a native shell. Although they leverage browser functionality, traditional browser navigation and user interface elements (chrome) are optional and can be hidden. This allows packaged apps to appear as native apps do, yet stay interoperable across multiple platforms. Like native apps, they can be downloaded from app stores then installed and launched from the mobile device’s home screen. They can also work without an Internet connection.

A full 61% of developers who have adopted packaged apps have done so primarily to target another platform and to do so using existing skill (43%) [13].

But packaged apps are not perfect; some developers have abandoned them due to poor app performance (29%), restricted user interface capabilities (22%), and being a step behind native apps (29%) [13].

After complaints from Facebook users, CEO Mark Zuckerberg made a public announcement about the difficulties with packing HTML5 into its app:

The biggest mistake that we made as a company is betting too much on HTML5 as opposed to native because it just wasn’t there [14].

For many people, the mobile web versus mobile app question becomes a philosophical debate. Do you believe in open standards? Which platform or ecosystem do I like or think is best? I prefer to take a pragmatic approach and consider a brand’s marketing objectives and what sort of technical platform best supports them. Considering the following questions and answers to better understand which direction is right for your brand and your campaign:

• Does your Web site have or will soon have significant mobile traffic? If so, you need a mobile friendly Web site. Considering the traffic statistics cited above, the answer to this is almost always yes. If you’re not sure, you can check your Web analytics package to see the amount of traffic originating from mobile devices.
• Are you planning a quick-and-dirty campaign? Short campaigns focused on a niche market are easier to cost-justify with a mobile micro site rather than a full branded app.
• Do you want the brand reinforcement that comes with having your app on a customer’s home screen? Almost everyone wants this, so the more important question is "Does your app idea have enough appeal to make this possible?"
• Does your campaign idea require offline access? If so, you need an app, either packaged or native.
• Does your campaign idea require a fully immersive, highly responsive user experience? For example, a highly gameified campaign will require a native app.
• Does your campaign idea require access to features only available via native APIs? If so, you’ll need a native app. Since OS makers are constantly changing their application programming interfaces (APIs), and packaged web tool makers are constantly racing to catch up, it’s not possible to provide a definitive list. However, it’s worth checking the following technology: camera, gyroscope, accelerometer, microphone, compass, GPS, address book, voice recording, microphone, voice calling, and email messaging.

Today, 13% to 58% of all Web site traffic is consumed with a mobile device, depending on geography, and this traffic continues to rapidly increase. If your Web site is not mobile friendly, you’ll frustrate many of your customers and drive away your prospects. So a mobile friendly site is a must have.

A mobile app represents a novel marketing opportunity for many brands. However, if the idea it’s based on is poorly conceived or executed, its impact may not justify the cost of development. But given the interest in apps from an audience whose size is increasing daily, a good idea that’s well executed can have a huge impact.

1. A full version of the main report on which this post is based is available as a free download from Code Crew: Developer Spotlight Developer Challenges and Cross-Platform Solutions, Info-Tech Research Group, June 2012.

2. U.S. Mobile Subscriber Market Share, comScore, October 2012
3. Apps Further Bolster Smartphone Usage, Linda Barrabee, NPD’s Connected Intelligence SmartMeter, October 2012
4. Worldwide Quarterly Tablet Tracker, IDC, September 2012.
5. Market Trends: Mobile App Stores, Worldwide, 2012, Gartner Market Analysis and Statistics, August 2012
6. Post-iPhone 5 Launch Heavily Influences October Indexes, Viki Zabala, November 2012
7. Holiday 2011: Breaking the One Billion App Download Barrier, Peter Farago, January 2012
8. CHART OF THE DAY: The App Economy Is $3.5 Billion, Alex Cocotas, June 2012
9. The Mobile Content War Continues: Apps edging out mobile browser, Social Fresh, Jason Kieth, August 2012
10. Why The “Web Versus Application” Debate Is Irrelevant, Thomas Husson, May 2011
11. Mobile Phones and Tablets Now Account for 1 in 8 U.S. Internet Page Views, comScore Data Mine, October 2012
12. Mobile share of web traffic in Asia has tripled since 2010, Pingdom, May 2012
13. Cross-Platform Developer Tools 2012, Vision Mobile, February 2012
14. Facebook: “Betting on HTML5 Was a Mistake” – Technical Reasons and Reactions, Abel Avram, September 2012

After completing in-depth interviews with about 50 creative and digital directors, brand managers, and mobile development executives during a study I did for Info-Tech Research Group, two qualities emerged as important correlates of a branded app’s marketing effectiveness:

• Usefulness, the quality of an app that enables users to complete a practical task in a way that is significantly easier than their conventional approach to task completion. For example, a store locator feature is useful because with a single touch you can find the nearest store of interest, while on the Web you’d need to type in your location or skim over a list of store locations.
• Socialness, the quality of an app that enables users to interact with others in a way that is emotionally rewarding or useful; for example, sharing family photos or asking for help from a peer group.

Of course other qualities such as entertainment value and usability are also important, but this post focuses on usefulness and socialness as they apply to the branded mobile apps developed by pharmaceutical companies for the baby care and baby development market, a global market worth over $44B US annually [1].

In a study that measured physiological changes during the use of branded mobile apps [2], Dr. Potter concluded that a useful app "increases the general interest in the product category that you're trying to sell, and the app also improves the attitude that you have toward the sponsoring brand ... and the purchase intention that you have towards the product". With some carefully controlled research suggesting that useful apps increase marketing effectiveness, it’s safe to conclude that baby care apps will be more effective if they are more useful.

It doesn’t take much persuasion to convince a brand manager that social access is important. The hype level on the subject is close to deafening. But while a Facebook or Twitter share button provide a superficial social component, deep socialness require that social features are directly integrated into an app’s functionality. For example, if an app allows you to post data it has collected and a related question to an interactive Web forum or Facebook wall, it will facilitate meaningful conversations and raise an app’s social value. If the conversation takes place within the app, it will also drive app adoption because the users have to download the app to participate in the discussion.

This year’s Advertising Week in New York was full of presentations on mobile and social marketing. I attended one of these by Laura Simpson, Global Director of McCann Truth Central in which she shared some fascinating research results from their online quantitative study of 6,800 moms in developed and developing countries [3]. The research suggest moms leverage their social network to solve the practical "mom" problems they face daily and paints a picture of the "mom economy" as an ecosystem in which switched-on moms exchange information and support and are socially rewarded for sharing.

In her day-to-day activities, each mom accumulates a wide range of information, from food for their family, cleaning methods, party entertainment, child education and care, as well as shopping and accounting. The job, in other words, is a serious multi-tasking challenge. Not everyone can be good at everything, so most moms specialize in a few areas. Figure 1 below shows the percentage of moms surveyed that regarded themselves as an expert in a particular area.

Fig 1. Moms see themselves as experts in some but not all required areas.

Moms bridge the expertise gap by getting together with other moms for advice, ideas, favors, and empathy. This creates a culture of community and sharing. Moms love to share: 88% of respondents said they want to share any interesting piece of information they come across and 37% said they would like to share it with as many people as possible.

How are moms communicating to their community and conducting their tasks? Technology is increasingly being used as the medium for communication and data gathering: 84% of moms said that technology simplifies their life. In fact, so strong is their attachment to technology that 49% of married mothers preferred to save their communication tool, whether mobile phone or computer, over their engagement ring!

Given that socialness will enhance the marketing effectiveness of any app, and the mom economy makes socialness even more important, designing deep socialness into an app should be a design priority.

Apps that focus on baby care and baby development are rapidly becoming a popular category in mobile app stores. They are usually designed to address one or several daily baby concerns. Let’s look at the usefulness of the feature set and socialness of the design of five popular branded baby apps.

This iPhone and iTouch app was built by the makers of Similac baby formula. It primarily focuses on tracking baby feeding, sleeping, diaper changes, and growth. Tips and advice are thrown in as well.

[soliloquy id="946"]

Fig 2. Similac Baby Journal: Tracking, displaying, and sharing baby care data.

The app helps moms track their baby’s sleep patterns, breast feeding, bottle feeding, and diaper changes and displays the data graphically, which allows moms to improve their baby care routines. Moms can also email the information to their pediatricians, family, or friends.

• Usefulness: Moms inclined to track baby care data will find the app useful for data capture and data review.
• Socialness: The email sharing function facilitates social sharing. Socialness could be improved by allowing data to be shared app to app. For example, a Web or tablet-based app for pediatricians could receive data from the Baby Journal app, and further manipulate and display it in ways that would stimulate doctor-mom dialogue.

Similac Baby Journal on Apple’s App Store

[pharma-ad]

AmazingBaby is an iOS and Android app built by pediatric nutrition product maker Enfamil. The app focuses on child development and provides ways to monitor and structure play time and bath time. The app tracks motor, cognitive, communication, and social milestones.

[soliloquy id="951"]

Fig 3. AmazingBaby Enfamil: Activity monitoring, capturing, and sharing child development app.

The app features are organized by "moment," defined as play time, fresh air, bath time, and quiet time. In all, 17 pre-programmed activities are designed to be fun and beneficial for a baby’s development. There are descriptions for each activity and the skills associated. You can snap pictures as you play and share them with friends and family over email and Facebook. There is also a "memories" timeline, organized by date, where you can see all activities played, milestones reached, and moments that could be interesting to re-visit when enough time passes.

• Usefulness: Moms inclined to track baby development data will find the app useful for data capture and review. Moms looking for new ideas of how they can interact with their baby to simulate development may also find the pre-programmed activities of value.
• Socialness: The social aspect of this app focuses on communications to family and friends over email and Facebook, mostly focused on photo sharing. The app adds some socialness value because these photos can be tied to a specific aspect of the baby’s development or developmental milestone, rather than generic photo sharing.

AmazingBaby on Apple’s App Store | AmazingBaby on Google Play

Built by Pfizer, I-Nanny is an iPhone app designed to provide expert baby care advice. The app covers warm care (growth tips and tracking), baby care information, and expert Q & A. Figure 4 below shows some screens for the Chinese market.

[soliloquy id="956"]

Fig 4. Illuma I-Nanny: Chinese informational and advice app.

Illuma I-Nanny is primarily a reference app even if it provides some baby care tools. One very useful feature built into this app is that it provides opportunities for moms to interact directly with qualified medical and child care experts, but this is mostly a one-to-one communication. Moms can ask a doctor advice on different health issues relating to their baby.

• Usefulness: Having reference material close at hand is useful, but only marginally better than a reference book or computer. The build in advisory function is useful but presumably expensive to deliver.
• Socialness: The user-to-expert communication is a type of social interaction, but this app's socialness could be enhanced if it also provided tools to access mom’s peer group.

Illuma I-Nanny on Apple’s App Store

SMA Baby Know How is an English-language iPhone app built by Pfizer that focuses on baby care. It has features that center on practical advice, as well as pregnancy and baby care tracking tools. For pregnant women the app provides tools to countdown due dates, week by week guide to your baby’s development, a kick counter to monitor baby movement, and a timer to record length and intervals of contractions.

[soliloquy id="969"]

Fig 5. SMA Baby Know How: Baby care and video sharing app.

For newly delivered mothers the app provides a baby age counter, week-by-week guide to baby growth and development, a daily feeds and needs tool (including breast or bottle feeds, nappy changes and sleep), and a Baby First milestone tool, that track moments like first smiles, first steps, and sleep. The app also features practical advice and videos.

Sharing again is limited to emails to mom’s friends and family, but this is a good way to make a baby’s development more social.

• Usefulness: The app is packed with features, many of which moms will likely find useful.
• Socialness: The email sharing function facilitates social interaction. Socialness could be improved by allowing data to be shared app to app.

SMA Baby Know How on Apple’s App Store

Bedtime Baby Sleep is an iPhone app built by Johnson & Johnson as a tool to help the 25-30% of babies that have trouble sleeping. I’m sure their sleep deprived parents are also grateful. There is a personal touch from the start, requiring parents to register online and create an account and baby profile. The app integrates the company’s three-step nighttime routine that helps babies fall asleep easier and sleep through the night better.

[soliloquy id="961"]

Fig 6. Johnsons Bedtime: Sleep logging, habit, forum, and lullaby functions.

The app provides parents access to the advice of sleep analysts, including Dr. Jodi Mindell, who are ready to provide expert advice on sleep-related questions. Johnson & Johnson claims to have a clinically proven method for babies to fall asleep easily and sleep through the night. App features include a sleep log, a three-step nighttime routine with instructive videos, and step-by-step advice, access to sleep analysts, lullabies and ambient sounds to help soothe babies to sleep. The app links to a forum for moms.

• Usefulness: The app has a well-integrated set of features with a core focus on sleep, which moms will likely find useful.
• Socialness: Social aspects of this app are more diverse than the others surveyed here. In addition to one-to-one contact with a specialist, it has an integrated forum, which leverages peer-to-peer communication.

Johnson's Bedtime Baby Sleep on Apple’s App Store

Baby care and development apps are a great branding opportunity for pharmaceutical companies and other brands interested in accessing the mom economy. They can enhance a brand’s relationship with mom, the primary decision maker, gatekeeper, and brand ambassador to the market. Most of the apps outlined above accomplish this with high usefulness. They are packed with what are likely useful features.

Unfortunately, most of the apps I looked at only scratch the socialness surface with basic features like email and Facebook sharing. Their marketing effectiveness could be greatly enhanced if app features facilitated sharing of rich information and easy dialogue in the robust mom economy, whether over the Web, Facebook, and Twitter, or within an app itself. Sharing over existing social networks like Facebook leverages their existing user base and infrastructure. In-app sharing drives app downloads and keeps customers fully focused on the brand message within the app. Given the importance of the sharing to the mom economy, this kind of deep socialness can make the difference between a somewhat effective branded app and a highly effective branded app.

1. Baby Care Products Market - Global Industry Analysis, Size, Share, Growth and Forecast 2007 – 2017, Transparency Market Research, April 2012
2. The Effectiveness of Branded Mobile Phone Apps, Journal of Interactive Marketing, Steven Bellman and Robert F. Potter et al, November 2011
3. The Truth About Moms, McCann’s Thought Leadership Unit, July 2012