Towards a guide for developers and novice researchers on human-centered design of the take-over request—Combining user experience and human factors

The requirements humans have regarding automobiles have kept changing constantly since their invention. In the beginning, very basic properties such as speed or oil consumption were decisive, whereas today, infotainment systems and driver assistance systems are in the focus of the customers (Gkouskos et al. 2014). Convenience and ease of use have become the automobile’s core values.

The role that the car plays for individuals depends, amongst other factors, on the region they live in, the generation they belong to, and their socio-economic status: Amongst adolescents living in Tirana, Albania, cars are considered a status symbol and even those individuals that do not enjoy driving plan to purchase a car in the future (Pojani et al. 2018). Owning a car is perceived as a necessity, although the city is built in a way that facilities for daily needs are within walking distance. In contrast, the car has been replaced as a status symbol by other consumer goods such as smartphones in other groups: Lenz (2013) states that for Germans between the age of 18 and 25, the car as a status symbol has significantly lost importance. Additionally, the pragmatism with which humans shape their own mobility is increasing, leading to a shift from owning to using and an increasing multi-modality of transport (Lenz 2013).

The growing necessity to fight global warming and the increased gas prices resulting from the war in Ukraine might have reinforced the need for profound changes in the field of mobility. Consequently, the automotive future is electrified, shared, connected, autonomous, and yearly updated (Kuhnert et al. 2018): Electrification of the vehicle’s drive train and using energy from renewable sources enable emission-free and CO2-neutral mobility. Connecting vehicles means that cars can communicate with each other as well as with the infrastructure, which is necessary for mobility providers to offer on-demand services. With shared vehicles available everywhere on demand, owning a car might be widely replaced. This development will be fostered by the automation of vehicles so that they can handle even complex traffic situations without human intervention. To implement new features quickly, development cycles within the automotive industry will become shorter, and regular software updates will be released frequently. Therefore, some authors ask the question, “Does the car as we know it still have a raison d’être?” (Wollstadt 2022), while others are convinced that “The farewell to the car as we know it will come.” (Hegmann 2019).

The automation of automobiles is a disruptive development that opens the opportunity to make mobility easier, more flexible, and more individual (Kuhnert et al. 2018). Traditional players in the automotive world must adapt in order to be able to manage the challenges and keep up with new players that focus mainly on electrified and automated vehicles (AVs). Software features, virtual validation, artificial intelligence, and connectivity move into focus. This implies a shift in their research and development activities as well as in the expertise of their developers and managers. Lastly, companies have to shorten their development cycles and make their working style more flexible and faster (Proff et al. 2019), i.e., by implementing agile methods and SCRUM teams instead of waterfall project management.

In the last years, the importance of human-vehicle interaction has increasingly come into focus: Instead of developing more features, solutions that address user needs and meet user requirements have to be invented. The driver-vehicle interaction, as it is in manual driving, is the background against which users perceive and assess new systems. With the changing capabilities of automation, the requirements regarding interaction and communication with passengers inside the vehicle and other road users outside the vehicle vary widely. To design a positive experience for all users, both inside and outside the vehicle, companies need to give space to this field of tension and adapt their development processes accordingly (Hassenzahl et al. 2021). They need to consult experts in this field who have specialized in designing human-machine interaction as well as the process of developing human-centered innovations.

One of the most urgent challenges is how to design the interaction between humans and partially, conditionally or highly AVs. The technical solution of automation must contain an overall concept of how tasks are divided between humans and vehicles and how responsibility regarding the driving task is transferred between them (Walch et al. 2017).

In this paper, we want to illustrate how Human Factors (HF) and User Experience (UX) Design can pave the way through this VUCA world that is full of Volatility, Uncertainty, Complexity, and Ambiguity. It is explicitly conceived for novices in the field of human-centered design who have a background in engineering, computer science, design, or other related disciplines. We consider it useful for both the application in the development departments of car manufacturers or suppliers as well as in application-oriented research. We will discuss the design of human-vehicle interaction in a take-over scenario, highlight which psychological constructs of the discipline of Human Factors are helpful in this context, and how the development process of automated systems can be facilitated when it is supplemented by methods of UX Design.

The field of HF emerged when experimental psychologists were consulted to explore aviation accidents or to improve military training (Lee et al. 2017). One of the most cited definitions of HF by Chapanis (1995) summaries, “Ergonomics or human factors is a body of knowledge about human abilities, human limitations and other human characteristics that are relevant to design.” (Chapanis 1995, p. 1625). Further, he defines the work field as “the application of ergonomic information to the design of tools, machines, systems, tasks, jobs and environments for safe, comfortable and effective human use.” (Chapanis 1995, p. 1626). Most definitions have in common that HF research is seen as the research field that investigates the interaction between humans and any form of machine: That might describe the interaction between a pilot and an airplane (Bergman 1976; Wise et al. 2016), between a worker and a nuclear reactor (Theureau 2000) or between a driver and a car (Lee 2008).

Pannasch et al. (2021) use examples in the context of the micro, meso, and macro level of HF to describe the importance of engineering psychology, a subdiscipline of HF: A well-known example on the macro level is the accident at the Chornobyl nuclear power plant in 1986. Here, errors in the action and information chain, but also the complexity of the control panel and the interrelationships within the nuclear reactor, were decisive for human failure, which had a major impact on the entire population. At the meso level, engineering psychology aims to design automated systems that meet humans’ concepts and mental models of, for example, automated systems (Pannasch et al. 2021). One target is to prevent misuse, disuse, or abuse. At the micro level, fundamental properties of complex behavior are investigated to facilitate systems that are sensitive to human capabilities, for example, regarding the design of a human-machine interface.

UX and its design were shaped by designers and software engineers: Dreyfuss (1950) explains UX Design as follows: “If the point of contact between the product and the people becomes a point of friction, then the designer has failed.” (Dreyfuss 1950, p. 80). Norman (1983) stated that designing an interface is a special discipline requiring both skills in programming and sophistication in human behavior. Another decade later, Norman et al. (1995, May) highlight that formulating user requirements needs to be one of the very first steps in product development. Additionally, they considered calling their discipline “human interface research” too narrow and shaped the term “user experience”: A working basis for this is the emphasis on UX starting at product conception, sensitivity to human needs, and inter-disciplinary as well as inter-divisionary collaboration.

In line with this, Hassenzahl et al. (2003) defined UX as a highly subjective and individual experience that encompasses all aspects of the interaction with a product. It includes “effectiveness […], efficiency […], emotional satisfaction […], and the quality of relationship with the entity that created the product or service […]” (Kuniavsky 2010, p. 14). Norman (2013) highlights how all the different experiences humans have with their senses determine their overall evaluation of their interaction with a product. McCarthy and Wright (2005) also state that emotion and experience are inseparable and argue that every action is connected to values, needs, desires, and goals. All in all, UX is dynamic, context-depending, and a subjective interpretation of the interaction with technology (Law et al. 2009).

To summarize, UX Design describes the process of deliberately designing experiences that are created by interaction with technology (Hassenzahl 2013). For example, car manufacturers shape their brand image and corresponding experiences to satisfy specific customer needs, e.g., by designing sportive cars and driving features. The more complex a product is, the harder it becomes to design a successful and delightful experience (Garrett 2011). Identifying design best practices has proven to be difficult due to the rapidly changing state of the art in technology (Kuniavsky 2010). While a certain feature might have led to excitement a few years ago, it has turned into a basic feature that humans expect to be there as a matter of course (Moser 2012).

We now want to highlight similarities between UX Design and HF and outline how the two approaches can benefit from each other: Firstly, both HF and UX Design are based on the concept of human-centered design: It is their basic principle to involve humans in all stages of the product development process. Conducting research with humans is used both at the very beginning of the development process when the solution space is very open and at more advanced stages when design solutions are tested against user needs. Secondly, both disciplines aim to develop technological solutions, products, and services that humans benefit from or to improve existing systems in a way that benefits the humans interacting with them (Dorton et al. 2021). This benefit might be on an individual, organizational, physical, or cognitive level (Wickens et al. 2022).

But what are the specific strengths of HF? As Wickens et al. (2022, p. 3) put it, “many different research methods can be employed to help discover, formulate and refine theory-based principles regarding ‘what works’ to support human performance”. Common methods include, amongst others, surveys, laboratory experiments, observational studies, case studies of major accidents, and also computational models to simulate human behavior and cognition. The scope of this research is human performance, i.e., in signal detection, decision-making, and action selection (Wickens et al. 2022).

The strength of UX Design is that it goes beyond the mere interaction of the human with the hard- and software and aims to create a pleasant overall experience. To be able to do so, the team needs to understand the users’ lifestyle, mindset, needs, pains and gains (Lewrick et al. 2020), as well as hopes and desires (IDEO 2015). By developing empathy, development teams can incorporate user needs in their work. Another important aspect is that UX Design considers not only the product, system, or service in focus but also the whole ecosystem, i.e., by considering a company’s customer service (Lee et al. 2017). UX Design often relies on qualitative studies focusing on the underlying needs of humans as well as the motives behind certain actions (Bargas-Avila and Hornbæk 2011). Common qualitative methods in UX Research are field visits, focus groups, and diary studies (Goodman et al. 2013). A frequently used formative evaluation technique in this field is usability testing which focuses on learnability, efficiency, memorability, errors, and satisfaction (Nielsen 1993). The focus of these studies is to iteratively refine the design of a certain product throughout the development process and to implement improvements quickly (Lee et al. 2017). Compared to sample sizes of 20–100 participants in typical experiments (Lee et al. 2017), often only 10–15 participants (Schrepp et al. 2017) are used included. Krug (2014, p. 114) even states that “testing a single user is 100% better than testing no user at all”.

Accordingly, the question arises of how both disciplines can benefit from each other in the best possible way: Firstly, we argue that HF provides the framework as well as theoretical background for UX activities. That means that by considering human abilities, developers know which concepts and theoretical models are relevant for a specific design challenge. For example, they are aware of general mechanisms of human-machine interaction, limitations of human performance, and how mental models influence the interaction with a system. Developers are then able to consider these boundaries when applying UX Design methods throughout the process. This knowledge is also helpful in increasing the significance of evaluations, user testing, and experiments by combining different methods and research approaches. We further elaborate on this train of thought in Sect. 3.

Secondly, the theoretical and analytical nature of HF might be beneficial for the quality of UX research. Bargas-Avila and Hornbæk (2011) argue that research in UX Design falls back on multiple methods. Mostly self-developed questionnaires are used, and some papers focus on a very particular situation—the authors call those studies “uniqueness studies”. Many of these papers do not report basic information on the underlying methodology, i.e., interview protocols or the methods used for data analysis. Additionally, new methods are often not compared with existing methods and not statistically verified, and therefore, the validity of the applied methods is often unclear. Hence, the quality of UX research might benefit from the application of HF methods. Vice versa, HF might also benefit from UX Design: Especially in the early phases of product development, small sample sizes, quick iterations, and the use of qualitative data deliver added value.

Thirdly, there is also much potential in the interdisciplinary work of the professionals in this field: The involved disciplines are, amongst many others, HF Engineering (HFE), Human Systems Integration (HSI), Human-Computer Interaction (HCI), UX, and Design Thinking (DT) (Dorton et al. 2021). When asking representatives about which tasks and roles their certain discipline includes, we see that all of the aspects of concept formulation, user research, system design, and human-in-the-loop/user testing are in the scope of each of these disciplines. Those disciplines can now benefit from each other from their different philosophies for system development: As John Winters is cited in Dorton et al. (2021) HF “requires structure and rigor and depends on sound application of science and foundational research” (Dorton et al. 2021, p. 1169). In contrast to this, Melissa Smith describes “understanding users’ motivations and needs and how they influence product usage” as key elements of UX Design (Dorton et al. 2021, p. 1169).

Based on this, we argue that the complex, multi-faceted challenges in designing human-vehicle interaction can only be solved with a holistic approach with collaboration across disciplines and divisions. Researchers and developers need to gain a comprehensive understanding of what humans’ abilities and needs are to be able to design innovative and human-centric automated systems and vehicles. In order to do so, we will now provide an overview of basic paradigms of the human relevant in the context of driving and mobility.

But how do people shape their mobility behavior? How do they decide on a certain means of transport, a certain travel route, or a certain travel time? And why do they decide to leave their house at all? This personal decision depends on the individual need to conduct an action that cannot be done at home and the infrastructure provided nearby (Becker 2016). Humans decide which needs can be satisfied with which activities and which means of transport they need to use in which way. The decision for a certain means of transport is based on certain factors or assumptions, for example, the required time, effort, costs and efficiency. Of course, human needs are not only a trigger for location changes but are also relevant during the decision process.

A very famous approach to understanding human needs is Maslow (1943) need pyramid, with physiological needs as the basis, followed by safety needs, social needs, esteem needs, and self-actualization needs at the top of the pyramid. Sheldon et al. (2001) propose ten human needs: self-esteem, pleasure-stimulation, physical thriving, self-actualization-meaning, security, popularity-influence, money-luxury, autonomy, competence, and relatedness, with the last three being especially noticeable in positive life events. Hassenzahl (2018) further investigated seven out of these ten needs (autonomy, competence, relatedness, stimulation, popularity, security, and meaning). Security for example, was found to be a “deficiency need” and matters especially when it appears to be restricted. His studies also showed that need fulfillment and experience correlate positively. The extent to which needs are fulfilled and the subjective rating of the hedonic quality of a product also correlate positively. The two dimensions, hedonic and pragmatic quality, together shape the subjectively perceived quality of a product and its overall attractiveness (Hassenzahl et al. 2000, 2008): In their definition, hedonic quality is abstract and less tangible and describes what the product symbolizes and which emotions it evokes. It consists of two characteristics: Stimulation describes to what extent a product can satisfy customers’ needs to improve their knowledge and skills, while identity is a product’s ability to strengthen the users’ self-esteem and communicate it to relevant others. It is concluded that hedonic quality is a motivator and pragmatic quality a hygiene factor for using a product (Hassenzahl et al. 2010).

According to Wright and Egan (2000), human needs perceived when driving are essentially the same as those identified by Maslow (1943). It states that human (or user) needs of all levels are satisfied: The inside of a vehicle is warm and safe, a room of privacy and space for social interaction. A vehicle is a powerful status symbol, a means of expression, and an extension of the body (McCarthy and Wright 2005). Tango et al. (2017) add that from 50 different identified needs of a driver, the ones related to the primary task and which enhance safety are valued the most.

Detjen et al. (2021) use a Stimulus-Organism-Response (SOR) model to explain human needs: Each vehicle has certain perceivable characteristics, e.g., its features, capabilities, and image. The corresponding stimuli are perceived by an organism, the human. The human then compares to what extent his or her individual needs are fulfilled in a certain imagined use case. Based on this evaluation, the attitude toward the vehicle is formed. A good fit and an imagined use case that is highly relevant for the human lead to a positive attitude. Transferred to automated driving, this means that the relevance of a need depends on the level of automation. At higher automation levels, new possibilities for humans to use their cars arise, and it is argued that hedonic and comfort-oriented qualities will become more relevant (Detjen et al. 2021). Lee et al. (2020) identified 12 human needs for autonomous vehicles: personalization & customization, connectivity, social needs, maintenance needs, accessibility, information, space, user interface, privacy, trust, health, and safety & security needs. They found out that drivers see fully AVs as private spaces that enable them to do the activities they feel like doing.

Frison et al. (2019a) investigated which needs were the most crucial for automated driving while traveling in different driving scenarios. They showed that the needs for security, autonomy, and stimulation were mentioned most. In a corresponding study, it was explored how four user needs (stimulation, autonomy, security, and competence) and affects differ between automated systems with different performance levels as well as different infotainment systems (Frison et al. 2019b). Participants’ need for security was less fulfilled when using the low-performance system, and negative affect was higher compared to the high-performance system. All investigated needs were significantly less fulfilled when subjects were driving in an AV with the so-called “ugly interface”. They conclude that since the performance of the automated system only affected the pragmatic qualities, system performance is a hygiene factor.

All in all, these studies show that human needs in automated driving correspond to basic human needs, that they are complex and multi-faceted, and that their relevance differs. Additionally, Garrett (2011) highlights that the identification of user needs can be complicated since users are quite diverse, even if they all originate from one certain user group. For automated systems, the subjective feelings of security, autonomy, and stimulation are especially important (Frison et al. 2019b). Before we delve deeper into this topic, we want to create a general understanding of driving.

Driving is a complex task with many tasks, which can be divided into different subtasks. Bubb (2003) distinguishes three levels of driving tasks: Primary driving tasks describe all subtasks directly involved in driving, such as speed control or steering. Secondary driving tasks are tasks that are still related to driving, and which increase the safety of all passengers and the environment, such as using indicators or activating the hazard warning lights. Tertiary tasks are all non-driving related tasks (NDRTs) in the car, such as setting the radio or air conditioning.

A frequently cited driver behavior model is the three-level hierarchy of driving tasks by Donges (1992). The highest level of the driving task is the navigation or strategic level, in which drivers need to decide which route they desire and in which time schedule they need to reach their destination (Winner et al. 2016). Here, the time horizon of the drivers’ actions is constant for about one minute up to several hours. The navigation level is based on knowledge-based behavior in which “the operator searches for problem-solving action alternatives based on knowledge already present or yet-to-be acquired” (Winner et al. 2016, p. 21). The resulting criteria, such as route and speed, flow into the guidance or maneuvering level. Here, the operator can fall back on known behavioral patterns to maneuver the vehicle. The modes of action at this level are controlled action patterns that are always aligned and adjusted with the maneuver. Drivers’ actions are in the range of seconds and the selection of the maneuvers to be performed is influenced by information from the environment. Furthermore, the feedback provided on the guidance level is used as a criterion for the stabilization or control level. Drivers use automatic action patterns to stabilize the vehicle based on skill-based, reflex-like behavior (Winner et al. 2016). The automatic action is continuously adjusted and refers to the influences of the feedback and other immediate environmental influences. The action typically takes place in the range of milliseconds.

The first driver assistance systems were developed in the early 20th century as brake force control for railroads and later for motor vehicles (Reichel 2003). The most important arguments for driver assistance systems are to support drivers in the physical as well as the psychological effort they have to spend when driving, to increase road safety, and to optimize the overall traffic (Vollrath and Krems 2011). From the car manufacturers’ point of view, the attractiveness of their products also increases with a wider range of functionalities.

Different classifications for driver assistance systems exist. Sheridan and Verplank (1978) set a theoretical basis for classifying assistance and automation levels. The proposed ten levels of automation for the interactions between humans and computers are: On level one, the lowest level, humans perform the task completely until they hand over control for the computer to implement the action. On the highest level, ten, the computer decides and performs the whole job. Humans are only informed about the actions of the computer if it decides to. Vollrath and Krems (2011) adapted these automation levels for manual and automated driving. On level one, the human is in manual control of the vehicle. On level five, the system performs an action if the driver confirms it, while on level ten, the system performs all tasks autonomously and ignores the driver. Until level five, the system can be classified as an assistance system (Hauß and Timpe 2002).

The Society of Automotive Engineers (SAE international standard J3016 2014) defines six levels of automation from level zero ‘No Driving Automation’ to level five ‘Full Driving Automation’ (Table 1). The levels of automation are divided into driver support functions (levels zero to two) and automated driving functions (levels three to five). These so-called SAE levels define which tasks belong to the driver/to the vehicle and which features are implemented at the different levels to support or take over the driving tasks. A recent example shows the state of development on the way to fully AVs: Mercedes Benz received approval from the German Federal Motor Transport Authority to put the first Level three Drive Pilot at speeds of up to 60 km/h into series production (Widmann and Müller 2021).

The exemplary assistance systems can be allocated to the levels of driving presented before: Systems such as the anti-lock braking system and the electronic stability program have the goal of supporting the driver on the control level. Many of the currently developed assistance systems target the maneuvering level, e.g., adaptive cruise control, lane centering, or traffic jam chauffeur. On the strategic level, navigation systems guide the driver to take the shortest route in terms of kilometers or time.

With an increase in automated functionalities, the role of humans is altered from the active controller to a passive monitor: In manual driving, humans control all the vehicle’s individual functions. In assisted or automated driving, they are in charge of monitoring and supervising some of these functions while they may be responsible for operating others (Merat and Louw 2020). On the strategic level, navigation is monitored, and potential hazards or alterations of the planned route are predicted by humans (Merat et al. 2019). Secondly, humans are responsible for monitoring lane maneuvering as well as the detection of and response to objects. On the control level, the vehicle motion control, i.e., lateral and longitudinal movement, must be monitored, especially regarding other road users and the road layout.

Hence, with increasing automation, the humans’ physical control decreases while reliance on warnings and communication provided by the vehicle increases (Merat and Louw 2020). These necessary additional aides are timely, intuitive, and accurate information that is provided by HMIs. They inform drivers of the vehicles and their system’s behaviors, capabilities, and limitations. Especially after longer system usage, situation awareness decreases while over-trust and distraction increase. In these cases, driver monitoring systems can be used to assess driver fatigue and distraction (Dong et al. 2011) to make sure that drivers are vigilant, actively monitoring the automation, and able to take over responsibility if necessary.

There are many different UX Design methods that can be used at the beginning of a project. One approach that is frequently used is to conduct workshops with representatives of all departments that are currently involved in the project:

Firstly, stakeholder maps help to gain an overview of all roles inside and outside the company that are, to some extent, relevant to the success of the project. Based on this, crucial roles can be identified in order to consider their needs during product development. Sometimes, this method also reveals that certain departments within the organization had not been involved in the activities up to this point and are therefore added to the team. The results of the stakeholder map(s) can be processed when restructuring the problem at hand. There are many different methods to do so—two related methods are Question Zero, 5 W-questions (Rosala 2021), and 6 W-questions (Lewrick et al. 2020). By answering certain questions, the team discusses their understanding of the problem and aligns on one common goal after a lively discussion that can also be communicated easily to the management.

The project team designing the take-over scenario might decide to define the problem using the 5 W-questions (Rosala 2021):

The team summarizes their problem at hand as follows: “Although automated driving aims at making travels safer, it presents drivers with new challenges: After longer periods of automated driving, their alertness may be reduced so that they may have trouble taking over control of the vehicle again. This might lead to reduced driving performance which could result in safety-critical situations or even accidents.” Based on this problem statement, the human-centered design process can be planned, e.g., designate responsible persons, identify suitable methods and activities, and integrate human-centered design into the overall project plan.

In the following two phases, the project team wants to gather information regarding their most important users, the drivers of passenger vehicles, and the context of use, i.e., the situation in which the take-over occurs. An overview of the current state of the art in research and technology serves as a starting point. Based on this, the project team gets into direct contact with users and collects original data. Consequently, requirements can be derived and prioritized to recognize user needs. Since these two steps are highly constructive on each other, we will elaborate on them together.

First, we tackle the problem from the HF perspective and cast an analytical eye on the take-over situation: In our field of interest, an overview of the technical environment and the task that users have to perform is especially relevant for the context of use (Maguire 2001): The operational design domain (ODD) describes the operating conditions for which a certain system is designed (Czarnecki 2018). The ODD of systems at the automation levels one to four are limited in terms of road environment (e.g., type of road), vehicle behavior (e.g., speed), and/or vehicle state (e.g., no trailer attached). By ensuring that the system does not exit its ODD, the residual risk of automated driving systems can be minimized (Gyllenhammar et al. 2020). If automation limits are reached, a transition of control is initiated by the automation, with the driver being in control after the transition (Lu and de Winter 2015). Nevertheless, even a well-designed take-over request (TOR) cannot ensure that all drivers regain control (Morales-Alvarez et al. 2020). If this is the case, the vehicle has to enter a safe state, meaning that it is stopped in a way that is visible to other road users and does not block emergency vehicles (Reschka and Maurer 2015).

The task of the driver is to take over. When doing so, two categories of take-over performance can be distinguished: take-over time and take-over performance (Weaver and DeLucia 2020). Take-over timing describes the time passed from the take-over request by the vehicle until a certain reaction of the driver. This can be the first input at the steering wheel or the first operation at the brake pedal (Gold et al. 2013). To do so, drivers need to glance at the road, grasp at the steering wheel (Kerschbaum et al. 2014), and place their feet on the pedals (Kuehn et al. 2017). The time that drivers require to resume control varies between studies and depends on the precise variable that is measured. To mention some results, drivers require a mean of 1.14 s ± 0.45 s (Zeeb et al. 2015) to 1.52 s ± 0.64 s (Zeeb et al. 2016) to put their hands on the steering wheel. Eriksson and Stanton (2017) calculated 4.56 s ± 1.63 s for drivers to resume full control, which is prolonged to 6.06 s ± 2.39 s if an NDRT is performed. A meta-analysis revealed that take-over time is shorter if perceived urgency is high and if drivers are not performing a visual NDRT and not holding a device such as a smartphone.

Take-over performance can be defined by the minimum distance to a forward hazard, braking and steering magnitude, lane positioning, and collisions (Weaver and DeLucia 2020). Lu and de Winter (2015) summarize that mental workload decreases if the level of automation increases. Also, reaction times were found to be higher in automated driving compared to manual driving, which they attribute to potential mental underload (Young and Stanton 2007). On the other hand, Perello-March et al. (2022) state that some NDRTs performed during highly automated driving may also increase arousal above the optimum and therefore decrease performance.

So how can the overall complexity of a take-over scenario and, therefore, also the level of arousal of the driver be determined? Objective complexity factors of a take-over scenario vary independently of the individual users of the system (Morales-Alvarez et al. 2020). Determining factors are the traffic situation, road conditions, as well as control transfer, e.g., if haptic guidance is provided or if the transition is abrupt. Subjective complexity factors are “affected by individual cognition adaptation processes” of the drivers (Morales-Alvarez et al. 2020, p. 5). One factor is the urgency of the situation: Depending on the SAE level, the take-over has to be carried out urgently (level 2) or leaves the driver more time to react (level 3). Also, the fact of whether, and if so, what kind of NDRTs are conducted can be a complexity factor. In a meta-analysis, Weaver and DeLucia (2020) summarized the results of 51 studies regarding take-over performance during conditionally automated diving. They concluded that performing NDRTs reduces take-over performance. There is some evidence that this is especially the case for visual NDRTs, e.g., watching a video, compared to nonvisual NDRTs, e.g., listening to music, since the resources required for the NDRT and the driving task overlap (Weaver and DeLucia 2020). Of course, also the human-machine interface providing the take-over request is a complexity factor (Morales-Alvarez et al. 2020). Lastly, situation awareness, the understanding of what is happening around them, is crucial for taking over as drivers need to decide quickly which actions to perform.

According to the most cited definition, situation awareness is “the perception of elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future.” (Endsley 1995, p. 66). Endsley (1995) describes the process of the development of human situational awareness as a three-stage model: In a first step, the relevant elements or cues of the system or the environment are perceived, which are, in the second step, combined into an understanding of the situation or the status of the situation. In the final step, a projection of the current state into the near future or a prediction of the system state in the future is possible. These three steps do not necessarily have to be built on each other. Endsley (2017b) explained that it is possible to predict a future state even without having perceived all relevant elements of a situation or without fully understanding a situation. However, the prediction might be better if perception and comprehension of the situation took place (Endsley 2015). When decision-making processes are automated, situation awareness of the system and environment can be reduced. This can result in poorer recognition and reaction to potentially critical situations. Different studies (e.g., Merat and Jamson 2010; van den Beukel and van der Voort 2013) have shown that automated driving can affect situation awareness negatively, especially if the human driver is kept out of the loop and relevant information is not shared which is a tendency that is found in the development of AVs (Endsley 2017a).

Now, the project team has a broad and general understanding of the concepts of take-over request, take-over time and quality, human performance, and situation awareness. Based on the collected insights, the requirements of the existing system can be reviewed critically, and some new user requirements can be formulated. Their nature will highly depend on the fidelity level of the underlying automated system, the specified use case, and the progress within the product development process. At this point, two limitations should be considered:

We now want to highlight how desktop research on quantitative data, theories and models can benefit from adding UX Design methods to give those results a “human touch”, to allow the team to identify with these numbers and statistics so that they can truly develop human-centered solutions. On the data level, this means that the quantitative results collected before are supplemented with qualitative data. There is a variety of methods at hand that can be used in this phase to get in direct contact with users, e.g., explorative interview (Lewrick et al. 2020), card sort (IDEO 2015), or empathy map (Osterwalder and Pigneur 2013). Many factors, e.g., project budget and experience level of the team, determine which approach has the greatest added value at this point, and often teams apply multiple methods to get a comprehensive picture.

For the development team’s task to evaluate an existing driver interface and develop new concepts, this might mean that they want to learn more about a specific user group: Their Stakeholder Maps (Phase 0) showed that drivers of premium class vehicles in Europe are most important for their success. We want to illustrate how using personas helps to develop empathy for this group: A persona is a meaningful archetype (Friis Dam and Siang 2022), a fictitious character that allows readers to engage and identify with users throughout the design process (Nielsen 2004). This approach provides a comprehensive picture and goes beyond concrete findings that are only related to the handover situation: By entering the lives of the users, the team gets a broad understanding of the users’ needs and how these needs create demands and requirements for the system (Nielsen 2004).

Depending on the specific target of the activity, there are different types of personas that can be created: The freestyle persona is created ad hoc, based on the memories of a user the team directly encountered, e.g., in an interview (Lewrick et al. 2020). Friis Dam and Siang (2022) outline the goal-directed persona to answer specific questions regarding what a typical user wants to do with the product. The engaging persona actively involves the design team in the lives of the personas without the risk of stereotypical descriptions (Friis Dam and Siang 2022). With this last type of persona, users, including their emotions, backgrounds, stories and characters, come to life. Since this type of persona is especially suitable for putting the development team into the users’ shoes, we want to pursue this approach. Generally, personas are equipped with a name, age, social background, family situation, and profession (Lewrick et al. 2020). The jobs-to-be-done covering all user activities relevant in the scenario are stated. Also, aspects such as trends, influencers, and pains and gains can be illustrated. For the engaging persona in particular, Nielsen (2004) proposes five areas that have to be covered: body, psyche, background, emotions, and cacophony. For example, the team might develop the persona “Sebastian”, a sales representative traveling a high annual mileage in his company car, a premium class limousine.

As stated before, the team now starts to concentrate on solutions rather than user problems and needs (Nessler 2018). The ground truth for this is all the information regarding the state of the art in research and technology, relevant theories and models on human behavior and performance, as well as results from original user research.

For the development team designing the take-over, the results of the first phases can be summed up as follows:

The ideation phase is a diverging phase, which means that it requires the team to widen their perspective, take anything into account and develop as many diverse ideas and solutions as possible (Nessler 2018). The method of the so-called How-might-we-questions (IDEO 2015) helps to get started. For this approach to be successful, it is important that the team relates to specific insights collected before and formulates nuanced questions (Hasso Plattner Institute 2022). Based on these provocative questions, the team can foster their creativity by using classic ideation techniques like brainstorming and SCAMPER (Michalko 2006) but also card sorting, paper and software prototyping, and storyboards (Maguire 2001). They ideate possibilities to answer the How-might-we questions and by doing so, open the solution space.

Based on these findings, the project team formulates How-might-we-questions, for example:

We assume that amongst many other possible focuses, the project team will set emphasis on the design of the TOR interface. The basis for their ideation is another specific literature review: The message that the driver shall resume control is often multimodal and can be of a visual, auditory, and/or haptic nature (Politis et al. 2015). Morales-Alvarez et al. (2020) provide an overview regarding different HMI modalities: Generally, HMIs with an auditory TOR design alone compared to visual-auditory TOR designs are preferable regarding take-over performance and workload (Roche et al. 2019). Bazilinskyy et al. (2018) found that multimodal TORs were preferred in high-urgency scenarios and auditory TORs in low-urgency scenarios. In line with this, reaction times were shorter and driver acceptance higher if the TOR was supplemented by semantic speech output explaining the reason for the TOR (Forster et al. 2017). This matches the conclusion by Greatbatch et al. (2020) that instead of simply alerting the driver, there should be more emphasis on how to provide additional context as to why the take-over is necessary.

In the ideation phase, the team might decide to use analogies and benchmarking (Lewrick et al. 2020) to come up with ideas, on how the reason for take-over can be communicated to drivers. This helps to get inspired by comparing the problem at hand with problems and their solutions from other areas and disciplines. The team can create an analogy inspiration board and apply elements of the solutions they found on the design of the TOR. Areas for inspiration might be aviation, rail and shipping traffic, but also the industrial context. In an iterative process, the team identifies the most promising solution ideas and outlines solution concepts.

Then, the most promising ideas are prototyped to make them visible, tangible, and testable (Lewrick et al. 2020). Different kinds of prototypes can be used depending on where the team is within the development process, the context of the product (e.g., app or automotive), and which hypotheses should be tested. Low-fidelity prototypes, such as paper prototypes, are very quick and easy to realize, while high-fidelity prototypes, like the implementation of a software prototype in the vehicle, require more effort (Pernice 2016). The development team might first use wireframes to review different graphical interfaces and then opt for a mid-fidelity prototype realizing the driver interface and TOR using a tablet and speakers.

The last step in human-centered design is the evaluation of the developed design concepts. Since the real-life usage of products is complex, it is crucial to test how humans perceive the system and how it supports them in the fulfillment of certain tasks. Evaluations can either be performed using guidelines, e.g., regarding usability and accessibility, or with the help of users. For the latter, exemplary methods are expert evaluation, user testing, satisfaction interview, post-experience questionnaire (Aghaeeyan et al. 2013; Maguire 2001), and field study (Roto et al. 2009). Which hypotheses are tested depends on the development phase the project is currently in Vermeeren et al. (2010): User testing as part of the iterative development process helps to improve the product during the process (formative testing) or to prove that the development process is completed successfully (summative testing). Ellis and Levy (2009) provide a guideline on how to identify relevant research questions, formulate hypotheses, and set up reliable and valid research. Environments for user testings are the laboratory, the field, or online (Vermeeren et al. 2010). Driving-related research can be conducted on public roads, test tracks, or in the driving simulator (Lindner 2017). In pre-defined scenarios, participants experience different HMIs for take-over while specific objective and subjective data are collected.

The development team can consult the studies discussed before when setting up their user testing. For example, they find information on which types of prototypes to use in which environments and which variables to investigate: Objective measures can be the timing and quality of take-over, e.g., acceleration, steering wheel angle, and lateral position (Weaver and DeLucia 2020). For the subjective data, there are various measures of special interest, some of which can be measured using standardized questionnaires:

These subjective assessments, which are based on self-reported data, can be accompanied by physiological parameters, e.g., electrodermal activity (EDA) for driver arousal (Li et al. 2021); EDA (Xu et al. 2022), respiration and skin conductance for the workload (Meteier et al. 2021); and eye tracking for visual distraction (Roche et al. 2019).

To make their mid-fidelity driver interface and TOR prototypes tangible, the development team mounts a tablet and speakers in a static vehicle mock-up. They test three different concepts and compare them against each other in two scenarios. Since their system is safety-critical, the team needs to have a reliable data basis and decides to collect both subjective and objective data of 30 participants. Quantitative data is analyzed using descriptive and inferential statistics to identify the best concept regarding take-over performance and usability. Qualitative data help to explain these results and understand the users’ experience.