MobileHCI 2014

Invited Interactive Tutorial

Mobile-based Tangible Interaction Design for Shared Displays

MobileHCI 2014, Toronto, Ontario, Canada

Abstract—Multi-touch has become the dominant interaction technique on shared displays, such as interactive tabletop surfaces. Alternative techniques include in-air gestures, interactive pens, and conventional pointing devices such as a mouse. A theoretically appealing but less explored approach is tangible interaction. Tangibles are physical objects that can act as both control and representation for the underlying system, allowing users to create, access, and manipulate digital information. Tangibles can offer a comparatively richer interaction experience by providing additional sensory information, such as pressure and friction, and by extending the interaction and display space, for instance through off-screen content control or feedback. One limitation, which often discourages researchers and designers from using such techniques, is the need for additional hardware or devices (as tangibles). Since touchscreen smartphones are gradually becoming ubiquitous, the possibility of using such devices as tangibles may encourage researchers to explore the matter further. In this hands-on tutorial, we will discuss and explore how touchscreen-based smartphones can be used as tangibles to interact with shared displays, and participants will be guided through the process of designing and prototyping their own mobile-based tangible interactions on an interactive tabletop surface.

Ali Mazalek, Ahmed Sabbir Arif. 2014. Mobile-based tangible interaction techniques for shared displays. In Proceedings of the 16th international Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '14). ACM, New York, NY, USA, 561-562. DOI:

Date: Tuesday, September 23, 2014
Time: 9:00-12:00

Tutorials MobileHCI 2014


Audience & Background

This tutorial is intended for mobile HCI enthusiasts who are interested in exploring mobile-based tangible interactions. Participants must have experience developing with HTML5 and JavaScript. In addition, a basic understanding of gesture recognition and touch-based interaction approaches is useful. Participants are expected to bring their own laptops for development.


The tutorial begins with an overview of the concept of tangible interaction based on recommended reading materials, provided by the organizers ahead of time. Open discussion addresses design possibilities and practices for mobile-based tangible interactions. Participants then divide into groups of four, and each group is provided with two Android mobile devices, a development toolkit containing a gesture recognizer, JavaScript libraries, and sample source code. They are also provided with several example scenarios, where the use of mobile devices as active tangibles may benefit users. Refer to the tutorial paper for exmaples.


Reading Materials

Prepare Your Laptop for the Tutorial

Sample Source Code