Tangible user interface

Short description: Physically interactive user interface

Not to be confused with Text-based user interface.

Reactable, an electronic musical instrument example of tangible user interface SandScape device installed in the Children's Creativity Museum in San Francisco

A tangible user interface (TUI) is a user interface in which a person interacts with digital information through the physical environment. The initial name was Graspable User Interface, which is no longer used. The purpose of TUI development is to empower collaboration, learning, and design by giving physical forms to digital information, thus taking advantage of the human ability to grasp and manipulate physical objects and materials.[1]

This was first conceived by Radia Perlman as a new programming language that would teach much younger children similar to Logo, but using special "keyboards" and input devices. Another pioneer in tangible user interfaces is Hiroshi Ishii, a professor at the MIT who heads the Tangible Media Group at the MIT Media Lab. His particular vision for tangible UIs, called Tangible Bits, is to give physical form to digital information, making bits directly manipulable and perceptible. Tangible bits pursues the seamless coupling between physical objects and virtual data.

## Contents

* 1 Characteristics
* 2 Comparison with graphical user interfaces
* 3 Examples
* 4 State of the art
* 5 Physical icon
* 5.1 History
* 6 See also
* 7 References
* 8 External links

## Characteristics

There are several frameworks describing the key characteristics of tangible user interfaces. Brygg Ullmer and Hiroshi Ishii describe six characteristics concerning representation and control:[2]

1. Physical representations are computationally coupled to underlying digital information.
2. Physical representations embody mechanisms for interactive control.
3. Physical representations are perceptually coupled to actively mediated digital representations.
4. Physical state of tangibles embodies key aspects of the digital state of a system

Eva Hornecker and Jacob Buur describe a structured framework with four themes:[3]

1. Tangible manipulation: material representations with distinct tactile qualities, which are typically physically manipulated. A typical example is haptic direct manipulation: can user grab, feel and move important elements in the interface?
2. Spatial interaction: tangible interaction is embedded in real space; interaction occurs as movement in this space. An example is full body interaction: can the user make use of their whole body?
3. Embodied facilitation: the configuration of material objects and space affects how multiple users interact jointly with the tangible user interface. Examples include multiple access points: can all users in the space see what is going on and interact with central elements of the interface?
4. Expressive representation: expressiveness and legibility of material and digital representations employed by tangible interaction systems. An example is representational significance: do physical and digital representations have the same strength and salience?

According to Mi Jeong Kim and Mary Lou Maher, the five basic defining properties of tangible user interfaces are as follows:[4]

1. Space-multiplex both input and output.
2. Concurrent access and manipulation of interface components.
3. Strong specific devices.
4. Spatially aware computational devices.
5. Spatial re-configurability of devices.

## Comparison with graphical user interfaces

A tangible user interface is usually built for one specific target group, because of the low range of possible application areas. Therefore, the design of the interface must be developed together with the target group to ensure a good user experience.[5]

In comparison with a TUI, a GUI has a wide range of usages in one interface. Because of that it targets a large group of possible users.[5]

One advantage of the TUI is the user experience, because it occurs a physical interaction between the user and the interface itself (E.g.: SandScape: Building your own landscape with sand). Another advantage is usability, because the user knows intuitively how to use the interface by knowing the function of the physical object. So, the user does not need to learn the functionality. That is why the Tangible User interface is often used to make technology more accessible for elderly people.[6]

Interface type/attributes | Tangible user interface | Graphical user interface

|
|

Range of possible application areas | Build for one specific application area | Build for many kinds of application areas
How the system is driven | Physical objects, such as a mouse or a keyboard | Based on graphical bits, such as pixels on the screen
Coupling between cognitive bits and the physical output | Unmediated connection | Indirect connection
How user experience is driven | The user already knows the function of the interface by knowing how the physical objects function | The user explores the functionality of the interface
User behavior when approaching the system | Intuition | Recognition

[5]

A tangible user interface (TUI) may be distinguished from a graphical user interface (GUI) based on the way that users interact with digital information. Whereas a GUI mediates interaction based primarily on digital representations such as windows, icons, menus and pointers, a TUI splits digital information from its physical artifacts and enables users to take embodied actions and manipulate the system using spatial actions. For example, a screen is used to present digital information, while a mouse or touchscreen is used to interact with the digital information. In contrast, TUIs make digital data directly graspable via physical objects that help establish a tangible link between the digital and the physical worlds [7].

TUI are often designed for domain-specific tasks by third parties, such as collaborative design, education or simulation. Unlike a claim made that TUIs have a "low range of possible application areas," research would lead us to believe that the applicability of a TUI to an application depends on the context, intended task goals, and requirements of the user, rather than on inherent limitations. Designing a TUI typically requires close work with target users in order to ensure usability and effectiveness, especially for specialized or complex tasks [8].

In comparison, GUIs are usually more flexible, and can support a greater diversity of applications in a single interface, which makes them capable of targeting a greater and broader diversity of users [9]. However, TUI's and GUIs are not mutually exclusive and many hybrid systems integrating tangible and graphical elements exist [10][11]. Interactive tabletops, for instance, combine touch-based GUI widgets with tangible objects, the former multimodal interactions and support for multi-modal interactions as well as collaborative activities [12].

TUIs offer possible advantages in terms of embodied interaction and cooperative interaction [13][14]. The physical nature of the interface can be used to develop spatial thinking, promote a common understanding within group work, and provide some sensory information that compliments visuals [15][12]. While some studies acknowledge that physical manipulation may be seen as "intuitive", usability is affected by the quality of a design, the context of the task and the user's familiarity, and not an intrinsic property of tangibility [16]. TUIs have been used to make interfaces more interesting and accessible for various kinds of users, including those inexperienced with conventional digital systems [17][18].

## Examples

A simple example of tangible UI is the computer mouse: Dragging the mouse over a flat surface moves a pointer on the screen accordingly. There is a very clear relationship about the behaviors shown by a system with the movements of a mouse. Other examples include:

* Marble Answering Machine by Durrell Bishop (1992).[19] A marble represents a single message left on the answering machine. Dropping a marble into a dish plays back the associated message or calls back the caller.
* The Topobo system. The blocks in Topobo are like LEGO blocks which can be snapped together, but can also move by themselves using motorized components. A person can push, pull, and twist these blocks, and the blocks can memorize these movements and replay them.[20]
* Implementations which allow the user to sketch a picture on the system's table top with a real tangible pen. Using hand gestures, the user can clone the image and stretch it in the X and Y axes just as one would in a paint program. This system would integrate a video camera with a gesture recognition system.
* jive. The implementation of a TUI helped make this product more accessible to elderly users of the product. The 'friend' passes can also be used to activate different interactions with the product.[21]
* a projection augmented model.
* SandScape: Designing landscape with TUI. This interface lets the user form a landscape out of sand on a table. The sand model represents the terrain, which is projected on the surface. In real-time the model projects the deformations of the sand.[6]

Several approaches have been made to establish a generic middleware for TUIs. They target toward the independence of application domains as well as flexibility in terms of the deployed sensor technology. For example, Siftables provides an application platform in which small gesture sensitive displays act together to form a human-computer interface.

For collaboration support, TUIs have to allow the spatial distribution, asynchronous activities, and the dynamic modification of the TUI infrastructure, to name the most prominent ones. This approach presents a framework based on the LINDA tuple space concept to meet these requirements. The implemented TUIpist framework deploys arbitrary sensor technology for any type of application and actuators in distributed environments.[22]

## State of the art

Interest in tangible user interfaces (TUIs) has grown constantly since the 1990s, and with every year, more tangible systems are showing up. A 2017 white paper outlines the evolution of TUIs for touch table experiences and raises new possibilities for experimentation and development.[23]

In 1999, Gary Zalewski patented a system of moveable children's blocks containing sensors and displays for teaching spelling and sentence composition.[24]

Tangible Engine is a proprietary authoring application used to build object-recognition interfaces for projected-capacitive touch tables. The Tangible Engine Media Creator allows users with little or no coding experience to quickly create TUI-based experiences.

The MIT Tangible Media Group, headed by Hiroshi Ishi is continuously developing and experimenting with TUIs including many tabletop applications.[25]

The Urp[26] system and the more advanced Augmented Urban Planning Workbench[27] allow digital simulations of air flow, shadows, reflections, and other data based on the positions and orientations of physical models of buildings, on the table surface.

Newer developments go even one step further and incorporate the third dimension by allowing a user to form landscapes with clay (Illuminating Clay[28]) or sand (Sand Scape[29]). Again different simulations allow the analysis of shadows, height maps, slopes and other characteristics of the interactively formable landmasses.

InfrActables is a back projection collaborative table that allows interaction by using TUIs that incorporate state recognition. Adding different buttons to the TUIs enables additional functions associated to the TUIs. Newer versions of the technology can even be integrated into LC-displays[30] by using infrared sensors behind the LC matrix.

The Tangible Disaster[31] allows the user to analyze disaster measures and simulate different kinds of disasters (fire, flood, tsunami,.) and evacuation scenarios during collaborative planning sessions. Physical objects allow positioning disasters by placing them on the interactive map and additionally tuning parameters (i.e. scale) using dials attached to them.

The commercial potential of TUIs has been identified recently. The repeatedly awarded Reactable,[32] an interactive tangible tabletop instrument, is now distributed commercially by Reactable Systems, a spinoff company of the Pompeu Fabra University, where it was developed. With the Reactable users can set up their own instrument interactively, by physically placing different objects (representing oscillators, filters, modulators...) and parametrise them by rotating and using touch-input.

Microsoft is distributing its novel Windows-based platform Microsoft Surface[33] (now Microsoft PixelSense) since 2009. Beside multi-touch tracking of fingers, the platform supports the recognition of physical objects by their footprints. Several applications, mainly for the use in commercial space, have been presented. Examples range from designing an own individual graphical layout for a snowboard or skateboard to studying the details of a wine in a restaurant by placing it on the table and navigating through menus via touch input. Interactions such as the collaborative browsing of photographs from a handycam or cell phone that connects seamlessly once placed on the table are also supported.

Another notable interactive installation is instant city[34] that combines gaming, music, architecture and collaborative aspects. It allows the user to build three-dimensional structures and set up a city with rectangular building blocks, which simultaneously results in the interactive assembly of musical fragments of different composers.

The development of the Reactable and the subsequent release of its tracking technology reacTIVision[35] under the GNU/GPL as well as the open specifications of the TUIO protocol have triggered an enormous amount of developments based on this technology.

It is difficult to keep track and overlook the rapidly growing number of all these systems and tools, but while many of them seem only to utilize the available technologies and are limited to initial experiments and tests with some basic ideas or just reproduce existing systems, a few of them open out into novel interfaces and interactions and are deployed in public space or embedded in art installations.[36]

The Tangible Factory Planning[37] is a tangible table based on reacTIVision[35] that allows to collaboratively plan and visualize production processes in combination with plans of new factory buildings and was developed within a diploma thesis.

Another example of the many reacTIVision-based tabletops is ImpulsBauhaus-Interactive Table[38] and was on exhibition at the Bauhaus-University in Weimar marking the 90th anniversary of the establishment of Bauhaus. Visitors could browse and explore the biographies, complex relations and social networks between members of the movement.

Using principles derived from embodied cognition, cognitive load theory, and embodied design TUIs have been shown to increase learning performance by offering multimodal feedback.[39] However, these benefits for learning require forms of interaction design that leave as much cognitive capacity as possible for learning.

## Physical icon

A physical icon, or phicon, is the tangible computing equivalent of an icon in a traditional graphical user interface, or GUI. Phicons hold a reference to some digital object and thereby convey meaning.[40][41][42]

### History

Physical icons were first used as tangible interfaces in the metaDesk project built in 1997 by Professor Hiroshi Ishii's tangible bits research group at MIT.[43][44] The metaDesk consisted of a table whose surface showed a rear-projected video image. Placing a phicon on the table triggered sensors that altered the video projection.[45]

## See also

* Hardware interface
* Kinetic user interface
* Natural user interface
* Organic user interface
* Unconventional computing

## References

1. ↑ Ishii, Hiroshi (2008). "Tangible bits". Proceedings of the 2nd international conference on Tangible and embedded interaction - TEI '08. pp. xv. doi:10.1145/1347390.1347392. ISBN 978-1-60558-004-3.
2. ↑ Ullmer, Brygg; Ishii, Hiroshi (2000). "Emerging Frameworks for Tangible User Interfaces". IBM Systems Journal 39 (3–4): 915–931. doi:10.1147/sj.393.0915. https://rauterberg.employee.id.tue.nl/lecturenotes/JFS-seminar-ID/FW-TUI-2000.pdf. Retrieved 2024-02-17.
3. ↑ Hornecker, Eva; Buur, Jacob (2006). "Getting a grip on tangible interaction: A framework on physical space and social interaction". Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI. pp. 437–446. doi:10.1145/1124772.1124838. ISBN 1-59593-372-7. http://www.ehornecker.de/Papers/FrameworkCHI.pdf. Retrieved 2024-02-17.
4. ↑ Kim, Mi Jeong; Maher, Mary Lou (30 May 2008). "The Impact of Tangible User Interfaces on Designers' Spatial Cognition". Human–Computer Interaction 23 (2): 101–137. doi:10.1080/07370020802016415.
5. ↑ 5.0 5.1 5.2 Campbell, John; Carandang, Xharmagne (29 July 2012). "Comparing Graphical and Tangible User Interfaces for a Tower Defense Game". AMCIS 2012 Proceedings. https://aisel.aisnet.org/amcis2012/proceedings/HCIStudies/11/.
6. ↑ 6.0 6.1 Ishii, Hiroshi (2007). "Tangible User Interfaces". The Human-Computer Interaction Handbook. pp. 495–514. doi:10.1201/9781410615862-35. ISBN 978-0-429-16397-5.
7. ↑ Keefe, Daniel F.; Gupta, Ankit; Feldman, Daniel; Carlis, John V.; Krehbiel Keefe, Susi; Griffin, Timothy J. (2012). "Scaling up multi-touch selection and querying: Interfaces and applications for combining mobile multi-touch input with large-scale visualization displays". International Journal of Human-Computer Studies. Special issue on Developing, Evaluating and Deploying Multi-touch Systems 70 (10): 703–713. doi:10.1016/j.ijhcs.2012.05.004. ISSN 1071-5819. https://www.sciencedirect.com/science/article/pii/S1071581912000936.
8. ↑ Shaer, Orit (2009). "Tangible User Interfaces: Past, Present, and Future Directions" (in en). Foundations and Trends® in Human–Computer Interaction 3 (1–2): 1–137. doi:10.1561/1100000026. ISSN 1551-3955. http://www.nowpublishers.com/article/Details/HCI-026.
9. ↑ Dix, Alan, ed (2004). Human-computer interaction (3rd ed.). Harlow, England ; New York: Pearson/Prentice-Hall. ISBN 978-0-13-046109-4.
10. ↑ Ishii, Hiroshi; Ullmer, Brygg (1997). "Tangible bits: Towards seamless interfaces between people, bits and atoms". Proceedings of the ACM SIGCHI Conference on Human factors in computing systems. New York, NY, USA: ACM. pp. 234–241. doi:10.1145/258549.258715. ISBN 0-89791-802-9. https://doi.org/10.1145/258549.258715.
11. ↑ Shaer, Orit (2009). "Tangible User Interfaces: Past, Present, and Future Directions" (in en). Foundations and Trends® in Human–Computer Interaction 3 (1–2): 1–137. doi:10.1561/1100000026. ISSN 1551-3955. http://www.nowpublishers.com/article/Details/HCI-026.
12. ↑ 12.0 12.1 Hornecker, Eva; Buur, Jacob (2006). "Getting a grip on tangible interaction: A framework on physical space and social interaction". Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY, USA: ACM. pp. 437–446. doi:10.1145/1124772.1124838. ISBN 1-59593-372-7. https://doi.org/10.1145/1124772.1124838.
13. ↑ Dourish, Paul (2001). Where the action is: the foundations of embodied interaction. Cambridge, Mass: MIT Press. ISBN 978-0-262-04196-6.
14. ↑ Hornecker, Eva (2012). "Beyond affordance: Tangibles' hybrid nature". Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction. New York, NY, USA: ACM. pp. 175–182. doi:10.1145/2148131.2148168. ISBN 978-1-4503-1174-8. https://doi.org/10.1145/2148131.2148168.
15. ↑ Ullmer, B.; Ishii, H. (2000). "Emerging frameworks for tangible user interfaces". IBM Systems Journal 39 (3.4): 915–931. doi:10.1147/sj.393.0915. ISSN 0018-8670. https://ieeexplore.ieee.org/document/5387042.
16. ↑ Rogers, Yvonne; Sharp, Helen; Preece, Jenny (2011). Interaction design: beyond human-computer interaction (3rd ed.). Chichester, West Sussex, U.K: Wiley. ISBN 978-0-470-66576-3.
17. ↑ Fitzmaurice, George W.; Ishii, Hiroshi; Buxton, William A. S. (1995). "Bricks: Laying the foundations for graspable user interfaces". Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '95. New York, New York, USA: ACM Press. pp. 442–449. doi:10.1145/223904.223964. ISBN 0-201-84705-1. https://doi.org/10.1145/223904.223964.
18. ↑ Antle, Alissa N. (2013). "Research opportunities: Embodied child–computer interaction". International Journal of Child-Computer Interaction 1 (1): 30–36. doi:10.1016/j.ijcci.2012.08.001. ISSN 2212-8689. https://www.sciencedirect.com/science/article/pii/S2212868912000025.
19. ↑ "Internet-of-Things answering machine from 1992, with marbles / Boing Boing". 21 March 2013. https://boingboing.net/2013/03/21/internet-of-things-answering-m.html.
20. ↑ "Topobo construction kit with kinetic memory". http://www.topobo.com.
21. ↑ "jive - social networking for your gran.". http://jive.benarent.co.uk.
22. ↑ http://www.cs.rit.edu/~pns6910/docs/Tuple Space/A Tuple-Space Based Middleware for Collaborative Tangible User Interfaces.pdf
23. ↑ "The Evolution of Tangible User Interfaces on Touch Tables | Ideum" (in en). http://ideum.com/news/evolution-of-tangible-ui-on-touch-tables.
24. ↑ "Wireless I/O apparatus and method of computer-assisted instruction". https://patents.google.com/patent/US5991693/en.
25. ↑ "Tangible Media". MIT Media Lab. http://www.media.mit.edu/research/groups/tangible-media. Retrieved 10 December 2014.
26. ↑ Underkoffler, John; Ishii, Hiroshi (1999). "Urp". Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99. pp. 386–393. doi:10.1145/302979.303114. ISBN 978-0-201-48559-2.
27. ↑ Ishii, H.; Underkoffler, J.; Chak, D.; Piper, B.; Ben-Joseph, E.; Yeung, L.; Kanji, Z. (2002). "Augmented urban planning workbench: Overlaying drawings, physical models and digital simulation". Proceedings. International Symposium on Mixed and Augmented Reality. pp. 203–211. doi:10.1109/ISMAR.2002.1115090. ISBN 978-0-7695-1781-0.
28. ↑ Piper, Ben; Ratti, Carlo; Ishii, Hiroshi (2002). "Illuminating clay". Proceedings of the SIGCHI conference on Human factors in computing systems Changing our world, changing ourselves - CHI '02. p. 355. doi:10.1145/503376.503439. ISBN 978-1-58113-453-7.
29. ↑ Ishii, Hiroshi (June 2008). "The tangible user interface and its evolution". Communications of the ACM 51 (6): 32–36. doi:10.1145/1349026.1349034.
30. ↑ Hofer, Ramon; Kaplan, Patrick; Kunz, Andreas (2008). "Mighty Trace". Proceeding of the twenty-sixth annual CHI conference on Human factors in computing systems - CHI '08. p. 215. doi:10.1145/1357054.1357091. ISBN 978-1-60558-011-1.
31. ↑ Alexa, Marc (5 August 2007). "Tangible user interface for supporting disaster education". ACM SIGGRAPH 2007 posters. Siggraph '07. pp. 144–es. doi:10.1145/1280720.1280877. ISBN 9781450318280. https://dl.acm.org/doi/10.1145/1280720.1280877.
32. ↑ Jordà, Sergi; Geiger, Günter; Alonso, Marcos; Kaltenbrunner, Martin (2007). "The reac Table". Proceedings of the 1st international conference on Tangible and embedded interaction - TEI '07. p. 139. doi:10.1145/1226969.1226998. ISBN 978-1-59593-619-6.
33. ↑ Wall, Josh (2009). "Demo I Microsoft Surface and the Single View Platform". 2009 International Symposium on Collaborative Technologies and Systems. pp. xxxi-xxxii. doi:10.1109/CTS.2009.5067436. ISBN 978-1-4244-4584-4.
34. ↑ Hauert, Sibylle; Reichmuth, Daniel; Böhm, Volker (2007). "Instant city". Proceedings of the 7th international conference on New interfaces for musical expression - NIME '07. p. 422. doi:10.1145/1279740.1279846.
35. ↑ 35.0 35.1 Kaltenbrunner, Martin; Bencina, Ross (2007). "ReacTIVision". Proceedings of the 1st international conference on Tangible and embedded interaction - TEI '07. p. 69. doi:10.1145/1226969.1226983. ISBN 978-1-59593-619-6.
36. ↑ "Sourceforge TUIO User Exhibition". 20 February 2017. https://sourceforge.net/apps/phpbb/reactivision/viewforum.php?f=7&sid=a5ef9e8af6f70496591e4dd5552a7f5a.
37. ↑ Tangible Factory Planning, Diploma Thesis, Daniel Guse, http://www.danielguse.de/tangibletable.php
38. ↑ "Interactive Table with reacTIVision : ImpulsBauhaus". 20 February 2017. https://sourceforge.net/apps/phpbb/reactivision/viewtopic.php?f=7&t=59.
39. ↑ Skulmowski, Alexander; Pradel, Simon; Kühnert, Tom; Brunnett, Guido; Rey, Günter Daniel (January 2016). "Embodied learning using a tangible user interface: The effects of haptic perception and selective pointing on a spatial learning task". Computers & Education 92-93: 64–75. doi:10.1016/j.compedu.2015.10.011.
40. ↑ Fidalgo, F., Silva, P., Realinho, V.: "Ubiquitous Computing and Organizations", page 201. Current Developments in Technology-Assisted Education, 2006
41. ↑ Michitaka Hirose (2001). Human-computer Interaction: INTERACT '01 : IFIP TC.13 International Conference on Human-Computer Interaction, 9th-13th July 2001, Tokyo, Japan. IOS Press. pp. 337–. ISBN 978-1-58603-188-6. https://books.google.com/books?id=LoR_qZGX8IgC&pg=PA337.
42. ↑ Hamid Aghajan; Juan Carlos Augusto; Ramon Lopez-Cozar Delgado (25 September 2009). Human-Centric Interfaces for Ambient Intelligence. Academic Press. pp. 15–. ISBN 978-0-08-087850-8. https://books.google.com/books?id=m6Fa-d2JTZEC&pg=PA15.
43. ↑ Howard Rheingold (21 March 2007). Smart Mobs: The Next Social Revolution. Basic Books. pp. 104–. ISBN 978-0-465-00439-3. https://books.google.com/books?id=v0ZKQ7aCd2QC&pg=PA104.
44. ↑ Paul Dourish (2004). Where the Action is: The Foundations of Embodied Interaction. MIT Press. pp. 45–. ISBN 978-0-262-54178-7. https://books.google.com/books?id=DCIy2zxrCqcC&pg=PA45.
45. ↑ Mary Beth Rosson; John Millar Carroll (2002). Usability Engineering: Scenario-based Development of Human-computer Interaction. Morgan Kaufmann. pp. 316–. ISBN 978-1-55860-712-5. https://books.google.com/books?id=sRPg0IYhYFYC&pg=PA316.

## External links

* TUIO open framework for tangible multitouch surfaces at TUIO.org
* Encyclopedia entry on the history of Tangible Interaction and Tangible User Interfaces
* White paper on The Evolution of Tangible User Interfaces on Touch Tables

Template:User interfaces

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Tangible user interface. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Tangible_user_interface&oldid=4299293"