Technology-Augmented Perception and Cognition

In recent years, increasing evidence of the positive impact of Virtual Reality (VR) on neurofeedback training has emerged. The immersive properties of VR training scenarios have been shown to facilitate neurofeedback learning while leading to cognitive enhancements such as increased working memory performance. However, in the design of an immersive VR environment, there are several covariates that can influence the level of immersion. To date, the specific factors which contribute to the improvement of neurofeedback performance have not yet been clarified. This research aims to investigate the effects of vividness in a Cave automatic virtual environment (CAVE-VR) on neurofeedback training outcome, and to assess the effect on working memory performance. To achieve this, we recruited 21 participants, exposed to neurofeedback training inside a CAVE-VR environment. Participants were divided into three experimental groups, each of which received feedback in a different neurofeedback training scenario with increasing level of vividness (i.e., low, medium, high) while also assessing the effect of neurofeedback on working memory performance. Current findings show that highly vivid feedback in CAVE-VR results in increased neurofeedback performance. In addition, highly vivid training scenarios had a positive effect on user’s motivation, concentration, and reduced boredom. Finally, current results corroborate the efficacy of the neurofeedback enhancement protocol in CAVE-VR for improving working memory performance.

Perfect memory is a dream long sought and which has become especially prevalent in our knowledge society. While only a few are blessed with a photographic memory, most people compensate for the fallacies of their memory with the use of tools and tricks to organize information, schedule reminders and remember facts. Most aspects of our lives, however, leave behind some sort of data trail. We explicitly document our trips and experiences while our phones implicitly record our movements and messages. The sum of these recordings paints a holistic picture of our activities in a so-called lifelog. Lifelogging technologies have long been praised for extending our memory’s capacity by allowing us to recall and browse our recorded experiences. But despite positive, well-documented effects on memory and well-being, lifelogging largely remains a niche activity with few commercially successful applications. Over recent years, we built a series of prototypes to investigate the challenges and opportunities of lifelogging technologies. We investigated the positioning of on-body cameras, the collection of implicit and explicit lifelogging data, requirements for effective lifelog reviews, automated summaries and ways to navigate multimedia lifelogs. In this chapter, we discuss the potential of these technologies to augment memory and the obstacles of bringing memory prosthetics to a broad audience.

The technology and tools that we develop have always been transformative, but the pace of change, particularly in the last few decades is undoubtedly altering humans in ways we don’t understand. As researchers look to develop novel prosthetics and tools to enhance our memories and extend cognition, further consideration is needed to understand how technologies can help (or, indeed, hinder) our inherent abilities. In this chapter, we identify two distinct forms of cognitive risk associated with current and emerging technologies: memory inhibition and memory distortion. We describe how lifelogging, search engines, social media, satnavs and other developments are prompting us to retain less information for ourselves (inhibition), and present three specific examples of this phenomenon: the Google effect, photo-taking-impairment and alterations in spatial memory attributed to satnav use. We further consider cases in which technology actually increases the likelihood of errors in what and how we remember (distortion), including doctored evidence effects, creation of false memories for current or historical affairs (“fake news”) and retrieval-induced forgetting. Finally, we provide an exploration of these cognitive vulnerabilities in the context of human memory augmentation, including the reporting of a mixed design experiment with 48 participants in which we demonstrate both retrieval-induced forgetting and false memory creation for real-world experiences.

Learning on a mobile device in everyday settings makes users particularly susceptible for interruptions. Guidance (memory) cues can be implemented to support users in resuming a learning task after a distraction. These cues can take a wide range of forms and designs and, to work effectively, need to be carefully adapted to the mobile learning use case. In this work, we present a structured in-depth literature review on task resumption support for mobile devices. In particular, we propose a design space based on 30 carefully chosen publications to highlight well-evaluated design ideas as well as currently underrepresented research directions. Furthermore, we evaluate the causes of interruptions in the domain of mobile learning and derive design ideas for task resumption support on mobile devices. To this end, we conducted two focus groups with HCI experts (\(N=4\)) and users of mobile learning applications (\(N=3\)). Based on the literature review, focus groups, and further related work, we discuss ideas and research gaps for task resumption cues in mobile learning. We derive six design guidelines to support researchers and designers of mobile learning applications and emphasize promising research directions and open questions.

Individuals are voluntarily inserting devices inside their bodies for non-medical purposes. As research into these insertable devices continues, and the practice gains traction, it is clear that earlier accounts of hobbyist makers and tinkerers reveal edge cases of use. While transhumanists still aim to augment their bodies past “human norms”, they are no longer the only group interested in insertables. Beyond these innovators, early adopters of these devices are simply concerned with practical and convenience-based uses. The motivation of this chapter is threefold. The first is to present a background of insertable devices individuals are using for augmentation. The second is a more in-depth account of insertables use than previous research (\(n=115\)). This allows us to answer, more accurately: 1. What devices are individuals putting in their bodies; 2. What do they use these devices for; 3. What are the motivations for modifying their bodies in this way; and 4. How do users of these devices identify themselves? The third, and final, contribution is a discussion of the motivations and self-identifications of individuals with these devices, and the implications of these. We show the progression from individuals using insertable devices to augment themselves to become “cyborgs”. Now, “everyday” people are choosing to use insertables for the convenience and amenity-based purposes afforded by these devices. We contribute to the literature knowledge of these more commonplace uses, beyond augmentation. These are for access and authentication, and for storing and sharing information. Only a minority use them for augmentation purposes, which are now known to be edge cases. These are: extending senses; supporting human connections; acting as an alternative digital interface; and capturing biometric data. Motivations are evolving as device usage is moving from hobbyists and innovators to early adopters. Given they are less concerned with augmenting themselves, it logically follows that the majority also do not, or do not strongly, identify with any particular social movement. Users are no longer confined to the ‘fringes’. Understanding this not only contributes updated findings, but also helps normalize the use of such devices.

Humans are dependent on their sensory perception being built upon their classical senses. Accordingly, researchers have been envisioning for decades to augment and enhance the existing spectrum of human senses by technological means. In this chapter, we will give an overview of scientific work that has been occupied with augmenting human senses to facilitate or enhance our capabilities. Further, we will present three exemplary applications extending the powers of the visual, auditory, and tactile senses following a threefold evaluation concept.

In this chapter, we present a privacy and security framework for designers of technologies that augment humans’ cognitive and perceptive capabilities. The framework consists of several groups of questions, meant to guide designers during the different stages of the design process. The objective of our work is to support the need for considering implications of novel technologies with regard to privacy and security early in the design process rather than post-hoc. The framework is based on a thorough review of the technologies presented earlier on in this book as well as of prior research in the field of technology augmentation. From this review, we derived several themes that are not only valuable pointers for future work but also serve as a basis for the subsequent framework. We point out the need to focus on the following aspects: data handling, awareness, user consent, and the design of the user interface.

Evolution has always been the main driving force of change for both the human body and brain. Presently, in the Information era, our cognitive and perceptual capacities cannot merely rely on natural evolution to keep up with the immense advancements in modern technologies. But systems we use daily (e.g. computers, smartphones, etc.) remain mostly unaware about our current state, causing what has been described as the “cognitive gap”—the inability of systems to adapt to the current cognitive and circadian state of the user (Niforatos et al. 2017). In this edited volume, authors contribute ideas and investigations into bridging this gap by bringing the machine (system) closer to the human (user). From improving our working memory, our ability to retain and learn new information to extending our perceptual and executive capabilities with wearable or implantable hardware, modern technologies bear an unprecedented potential to seize the role of natural evolution for humans. One should tread lightly in this “Brave New World” of Human Augmentation, however. In this final chapter, we summarize the key contributions of each chapter in this book, assume a philosophical standpoint over augmentation technologies and share our vision on their future outlook.