HUMAN ROBOT INTERACTION
Humanoid Robots
David J. Bruemmer, Mark S. Swinson, in Encyclopedia of Physical Science and Technology (Third Edition), 2003
VII.D Service and Entertainment Robots
Human–robot interaction plays a crucial role in the burgeoning market for intelligent personal-service and entertainment robots. Applications include everything from robots that assist the elderly and severely disabled to entertainment robots at amusement parks. Increasingly, robots that can serve as mobile, autonomous tour guides and information kiosks will grace public places. One encouraging example is Minerva (Fig. 13), a popular tour guide at the Smithsonian National Museum of American History, which uses a rich repertoire of interactive capabilities to attract people and guide them through the museum. Minerva's creators at Carnegie Mellon University report that Minerva's facial features and humanoid form have had a profound effect on the way in which people respond to it.
In the Department of Research and Development at the National Center for Science Information Systems in Tokyo, Japan, Haruki Ueno has developed a service robot that can carry out high-level commands by exploiting sophisticated knowledge architecture. HARIS, with its robotic arm and human interface, is designed to help disabled people move and fetch objects. Using the human hand as a model, researchers created a robotic manipulation system capable of tasks such as picking up a coffee cup, grasping an egg, dialing a telephone call, and even pouring the coffee. HARIS is comprised of three separate arm segments and a hand. The arm has three joints and 8 dof. The hand has five fingers, 178 tactile sensors, and 17 dof. However, the mechanical arm itself is only the first problem that must be solved before service robots can be truly useful. The robotic system must also include a scene understanding system, three-dimensional vision system, a real-time motion scheduling system, an arm control system, and a knowledge architecture which allows it to capture and use information about its environment (Fig. 14).
To accomplish this, the designers tried to model human ways of storing information, communicating, and scheduling actions. The researchers have integrated this arm with stereo vision cameras that allow the robot to perceive several real-world objects such as cups and trays. The robot can discern the color of objects, their position and whether cups are empty or full. With the help of a natural language processing program, the robot can carry out commands such as “Get the blue cup!” To be truly useful as a service robot, HARIS must understand simple relationships between the elements of its environment. It must know, for example, that tea and coffee go into cups but not into plates. It must know that cups go on saucers and that they are easier to move when empty than when full. It is difficult for us to conceive just how much knowledge we draw upon even when we do simple tasks. In fact, the hard problem for service robots is not the vision system, the mechatronics, or even the natural language processing component, but rather the need for knowledge engineering (Fig. 15).
HARIS employs a semantic network to store names, roles, attributes and relationships for each object in the environment. Ueno used this network to create a complex world model comprised of a shape model, a functional model, an object model and a spatial model. In addition, there is a task model that relates to tasking and human interaction. One of the greatest advantages to using a term-oriented semantic network is that information is already stored in high-level semantic form and lends itself to verbal interaction with humans. Instructions from a human transition smoothly into a scheduling process based on transitive verbs such as “approach,” “hold,” “carry,” “put,” and “release.” Using its world model, HARIS can select a goal task to match the given command and can generate a sequence of primitive behaviors that will achieve this goal (Fig. 16).
This project exhibits a confluence of mechanical engineering, computer vision, knowledge engineering and natural language processing. Note that it requires that the robot be given a priori, a well-structured knowledge base that captures the objects and possible interactions within a defined domain. Thus, while HARIS's complex, rich knowledge architecture deals well with the few objects it is familiar with {red cup, blue cup, white tray, etc.}; new objects will stymie it. The idea of storing associations explicitly within frames may be functional for restricted tasks, but it bears little resemblance to the brain where associative connections are distributed implicitly (no hard-coded semantic representation) as spreading activation. This is the classic limitation of so-called, model-based approaches.
An ambitious effort at Vanderbilt University is also working toward intelligent service robots that can help the sick, elderly, and physically challenged live independently (Fig. 17). Researchers at Vanderbilt believe that to accomplish this aim, their robot must be task-general, and thus be able to cope with unstructured, dynamic environments, such as the home. ISAC hands are equipped with multifingered grippers that will allow the robot to pick up a variety of objects. To pick up a spoon, for instance, ISAC employs sensitive touch sensors that help it place the spoon between a thumb and three fingers.
To deal with the complexity inherent to humanoid bodies and tasks, ISAC is designed as a multiagent system where a separate agent is devoted to each functional area. For instance, one agent deals with arm movement, while another is devoted to interacting with humans. Using a Data Base Associative Memory (DBAM), ISAC has the ability to store and structure the knowledge it acquires. To mimic long-term memory, DBAM uses a spreading activation network to form associations between database records. To efficiently structure its memories, ISAC uses a Sensory Ego Sphere (SES) that processes incoming perceptual data according to spatial and temporal significance.
Humanoid robots are also surfacing in the entertainment industry. Designed by Florida Robotics, “Ursula the Female Android” is a remote-controlled, full-size robot that walks, talks, dances, plays music, and more. Ursula makes for great entertainment. Her unique ability to enthrall a crowd makes her an effective communicator. Florida Robotics can make specialized androids like Ursula for a wide variety of commercial purposes. Special features of “Ursula” include fiberoptic hair and on-board video cameras (Fig. 18).
SARCOS, a Utah-based company with considerable experience in entertainment engineering has developed some of the world's most sophisticated humanoid robots and virtual reality interfaces. SARCOS entertainment robots are constructed not only to be high performance but also to be sensitive and graceful. SARCOS has placed a great deal of emphasis on the aesthetics of their humanoid as well as the engineering. Their corporate staff includes leading designers, artists, and craftspeople who style the robots. Concept development and graphic renderings are supported by a complete sculpting facility where high-performance skins and other coverings are produced. The robots can be telecontrolled by a remote operator or by a computer-controlled playback of a preprogramed show (Fig. 19). Recently, a SARCOS robot named DB has been used by the Kawato Dynamic Brain Project in Japan to enable motion learning. DB has 30 dof, and yet is nicely packaged in an 80-kg, 1.85-m body. Currently, a tether is needed to connect DB with its hydraulic air supply. Nonetheless, this is one of the most capable robots of its kind ever built (Fig. 20).
Read more
On the social perception of robots: measurement, moderation, and implications
Steven J. Stroessner, in Living with Robots, 2020
Machineness and robot perception
Numerous HRI researchers and theorists have emphasized how humanlike physical attributes of robots, even features that play no functional role, are central in fostering human engagement (Duffy, 2003; Eddy, Gallup, & Povinelli, 1993; Fink, 2012). Humanlike physical features are believed to enhance familiarity, affinity, and feelings of warmth (Branscomb, 1979; Caporael, 1986; Mori, 1970/2012). These responses are presumed to arise from anthropomorphism, the attribution of human cognitions and emotions to human-looking inanimate objects. Theoretical accounts of the psychology of anthropomorphism highlight the importance of physical features in promoting the humanization of nonsocial entities:
The perceived similarity of targets to humans should likewise influence the extent to which people anthropomorphize nonhuman agents…Readily observable humanlike features should therefore influence the accessibility of egocentric or anthropomorphic knowledge structures, thereby increasing the likelihood that such knowledge is applied to a nonhuman target of judgment.
Within robotics, it has been argued that the humanlike outer appearance of a robot can lead people to respond automatically to social cues emitted by a robot, applying human–human social schemas and norms in such interactions (Lee, Lau, Kiesler, & Chiu, 2005; Schmitz, 2011). Conversely, visible physical indicators that robots are in fact machines should reduce engagement and affinity by decreasing the degree they are seen as humanlike.
Affective Human–Robot Interaction
Jenay M. Beer, ... Sujan Pakala, in Emotions and Affect in Human Factors and Human-Computer Interaction, 2017
Conclusions
In closing, affective HRI is emerging as a critical and necessary field of study as robots become more commonplace in operating in social environments and domains, such as caring for older adults, companionship, and education. In this chapter, we provide a review of affective HRI, highlighting the importance of considering both the human and robot as social entities. Through this review, we identify common affective themes important across application domains, such as the importance of robot facial expression and a match between robot social capability, user expectations, and user acceptance. However, research on individual differences, varying user needs, long-term evaluations, and culturally specific studies are needed. Through continued research and development of affective HRI, the application of social robotics will continue to evolve and promote assistance and enrichment in users’ lives.
All about human-robot interaction
Kiran Jot Singh, ... Balwinder Singh Sohi, in Cognitive Computing for Human-Robot Interaction, 2021
Learning and processing
The heart of HRI resides in learning and processing block. It takes valuable info from task model, interaction model and input block for processing and perform desired task be generating control signals. Further, it also makes utilization of diverse calculation approaches like optimization techniques, neural systems, deep learning (DL), etc. in order to enlarge and improve knowledge and learning of robot for accomplishing forthcoming tasks. The biggest challenge of this block is to maintain system firmness and dynamic behavior of the robot, which can be applicable to machine learning as well.
DL in robotics deals with very specific problems like robot learning, reasoning, embodiment challenges and many more which are typically not addressed by machine learning and computer vision. It takes immediate outputs (object segmentation, detection, depth estimation etc.) into account in order to generate actions in real world. Because of advent of DL many new features have been introduces in HRI. Features of HRI with or without involvement of DL are shown in Table 11.3 (Zhang, Qin, Cao, & Dou, 2018).
Table 11.3. Human-robot interaction (HRI) contrast with or without deep learning (DL).
FeaturesHRI without DLHRI with DLAutonomyLowHighInformation inputComplete, accurate commandIncomplete, vague, and noisy dialogInformation typeInherent informationInherent information, Impromptu information (based on knowledge graph etc.)Information environmentPhysical space (low dimension)
Physical space (low dimension)
Cyberspace (high dimension)
Interaction medium
Point-to-point, end-to-end mapping
Contact-based user interface
Peer-to-multi or multi-to-peer
Multimodal interfaces
Information processingSequential processingParallel processingToleranceLowHighAffordanceLowHighOutput and feedbackNear-timeIn-time real-time
Deep reinforcement learning (DRL) techniques are widely used for achieving state-of-art results for learning, navigation and manipulation tasks in robotics (Tai, Zhang, Liu, Boedecker, & Burgard, 2016). MDP is used for formalizing robotic task in DRL by making a series of observations, leading to actions and generating rewards. MDP has five key parameters namely states, actions, transition dynamics, rewards and discount factor. The MDP can be described as a process which is in a state at each time step. The decision maker can take any action in present state. As decision maker moves to next state it receives a corresponding reward which is evaluation of action-based on performance. To extract better reward form environment the agent, learn to make decisions by making use of trial and error paradigm.
Read more
What Is Emotional About Emotional Robotics?
Thorsten Kolling, ... Monika Knopf, in Emotions, Technology, and Health, 2016
领英推荐
Affect in Human-Robot Interaction
Early research in human-robot interaction (HRI) showed that humans tend to “personify” computers (Scheibe & Erwin, 1979). Subjects in Scheibe and Erwin’s study used personal pronouns (“you” or “him/her”) for computers and were very emotionally involved when playing a simple strategic game with a computer. This behavior was not related to actual game performance, showing the dissociation between normative, rational interaction with the computer and the specific socioemotional responses of the human user. In line with these findings, Reeves and Nass (1996) developed the media equation theory which asserts that, as human-machine interaction is inherently natural and social, the rules of human-human interaction also apply to human-machine interactions.
Relying on the ideas of media equation theory, Nass and colleagues (Nass & Moon, 2000; Nass, Moon, Morkes, Kim, & Fogg, 1997b; Nass, Steuer, Henriksen, & Dryer, 1994) demonstrate that human users tend to interact with computers as if they were independent social actors (computers as social actors, CASA). The CASA theory posits firstly that humans use the same rules and heuristics when interacting with a computer as they do when interacting with humans, and secondly that being “mindless,” human actors are not aware of their behavior toward computers. Using theories of social psychology and empirical tests, a number of studies have provided support for the ideas behind CASA. In student samples, for example, certain stereotypes that were expected to apply only to humans were also used in interactions with computers. In one study (Nass, Moon, & Green, 1997a), for example, participants showed stereotypic gender behavior when interacting with a computer with a male, female, or neutral voice. In another study (Nass, Isbister, & Lee, 2000), participants were shown videos of persons of either the same or a different ethnicity to their own. In one case, they were told they were interacting with a human, in the other they thought they were interacting with a software agent. In both cases, participants showed stereotypic in-group preferences. It has been concluded from studies like these that humans react mindlessly to social cues, whether or not they are derived from a machine.
The CASA model maintains that people will tend to interact with computers as if they were human, even though the computers do not resemble them at all. Nass and Moon (2000) add that the tendency of the human user to show human-like behavior toward a machine rises when the degree of human-likeness (e.g., a personalized interface; language use; display of a human face; emotion expression; interactivity) is increased.
The idea that human-likeness in a machine increases the human tendency to positively interact with it has also been put forward in other research into human-robot interaction (see, for example, Cappella & Pelachaud, 2002). Research by Bruce, Nourbakhsh, and Simmons (2001) showed that humans are more prone to interact with a robot if the robot has a human-like face and if it moves its head (a moveable screen). Factors that contribute to the human tendency to anthropomorphize robots are independent movement (see, for example, Tremoulet & Feldman, 2000) and a face with eyes (see Scassellati, 2002). Human-likeness, however, has its limits, as is demonstrated in studies using the uncanny valley hypothesis. Originally proposed by the Japanese roboticist Masahiro Mori (1982), the uncanny valley hypothesis states that humans react positively to an increase in the human-likeness of robots, but that in the case of a small deviance from perfect human-likeness, they tend to react dismissively. Empirical research in recent years has analyzed the uncanny valley effect from different research perspectives, e.g., developmentally (Lewkowicz & Ghazanfar, 2012) and by using eye-tracking (Cheetham, Pavlovic, Jordan, Suter, & J?ncke, 2013), as well as functional brain imaging (Cheetham, Suter, & J?ncke, 2011).
So far, however, little evidence on humans actually responding emotionally to robots is available. Studies investigating the emotional reactions of humans when harming a robot or watching a robot being harmed (Bartneck & Hu, 2008; Rosenthal-von der Pütten, Kr?mer, Hoffmann, Sobieraj, & Eimler, 2013; Slater et al., 2006) demonstrated that the willingness of participants to “harm” a robot is higher than their willingness to harm a person in the classical Milgram experiment, and that almost all participants were prepared to destroy the robot, which they probably wouldn’t have been if it had been a “real” living being. At first glance, these findings would appear to contradict the idea of human-likeness in human-robot interactions. But at the same time, almost all healthy participants reported the same negative feelings, the same moral concerns, and the same arousal as well as distress that they would have experienced if asked to harm or kill a living being, even if the “harmed” robot is just a robot with a functional, nonemotional embodiment. Rosenthal-von der Pütten et al. (2013) argue that this emotional-behavioral dissociation is due to the knowledge that no real harm is being done because the robot is not a living being, and that this self-justification offsets the emotional reaction. It has also been demonstrated that people do attribute emotional states to robots and adjust their behavior accordingly (Eimler, Kr?mer, & von der Pütten, 2011). These findings are in line with the conclusion drawn in Scheibe and Erwin’s study mentioned above, that there is a dissociation between rational behavior on the one hand and the socioemotional reaction on the other. One explanation that may account for these seemingly contradictory findings is the content of the mental models people employ when considering a robot. These mental models contain schemas, beliefs, and heuristics that define the category of objects to which the one under consideration belongs. They are formed automatically when a new object triggers the retrieval of experiences and pieces of knowledge from long-term memory that guide the reaction to the object under consideration (see Gentner & Markman, 1997, for more information about the cognitive processes involved). By combining these experiences and pieces of knowledge, a mental model of the object emerges more or less instantaneously within the first couple of minutes of interaction (Powers & Kiesler, 2006). It is assumed that the cues trigger simultaneous retrieval, whether or not the categories retrieved are mutually exclusive (Hintzman, 1986). Thus, a mental model of a robot may very well contain characteristics that are typical of mechanical robots and living creatures at the same time. As mental models involve both emotions and reasoning, the integration of disparate features into one mental model may account for the dissociation between rational behavior and the socioemotional reaction (see Kiesler & Goetz, 2002; Powers & Kiesler, 2006).
Although rational reactions to the robot’s mechanical features sometimes seem to be stronger than emotional reactions to its life-like features, this is not always the case. A number of studies (Kidd, Taggart, & Turkle, 2006; Klamer & Ben Allouch, 2010; Turkle, 2005; Wada, Shibata, Saito, Sakamoto, & Tanie, 2005) document that humans show empathy toward a robot, look after it, and form attachments resembling those with real animals and human babies. Interestingly, Kidd et al. (2006) report that most elderly participants in an intervention study with an animal-like robot reported not bothering to answer the question whether the robot was alive or not, but liking it nevertheless. Negative emotional reactions are reported for a number of nursing home residents who are frightened of interacting with an animal-like robot because they worry that it might bite (although this is mainly reported for persons with dementia), or become distressed when they are expected to interact with an animal- or baby-like robot over an extended period of time because they feel that the responsibility is too much for them.
Overall, research on HRI indicates that humans tend to interact with computers and robots in the same way they do with humans. The greater the human-likeness, the stronger the tendency of the human user to react to a machine as if it was a living being. However, this tendency declines sharply and suddenly when the robot is almost but not perfectly indistinguishable from a real living being, and it is replaced by aversive emotional reactions. Complicating this relationship, scarce research on emotions toward robots shows that while humans do show empathy toward robots, take care of their well-being, and form attachments resembling those to real animals or babies, cognitive reasoning can offset these tendencies. A possible explanation for conflicting results may be the formation of mental models about robots that contain aspects of both machines and living beings.
Read more
Transparent interaction and human–robot collaboration for military operations
Shan G. Lakhmani, ... Jessie Y.C. Chen, in Living with Robots, 2020
Working with a robot
In the military, HRI is often focused around teleoperation. This approach styles the interaction between humans and robots as one between operator and tool. In this approach, the robot is a piece of equipment, and the operator needs to know about its operating procedures, functions, and limitations (Cannon-Bowers, Salas, & Converse, 1993; Ososky, Schuster, Phillips, & Jentsch, 2013). This approach increases operators' capabilities, allowing them to execute tasks more safely, accurately, and reliably than they would alone (Parasuraman et?al., 2000). Fully manually controlled robots are extensions of the operators’ will (Sheridan, 1995). While this form of interaction keeps the operator in the loop, the task is engrossing, and the operator can only control a single robot at a time (Adams, 2005). A supervisory control approach allows the operator to allocate tasks to one or more robots, monitor their performance, and intervene when necessary (Inagaki, 2012; Johnson et?al., 2014; Sheridan, 1995). Increasing the autonomous capabilities of a robot reduces the amount of work an operator has to do to direct a single robot, leaving them free to supervise more robots or to engage in other, separate tasks (Lyons, 2013).
Task delegation frees the operator to do more, but having both robots and humans doing tasks near one another is hardly an efficient use of resources. Collaboration between the two groups, the interdependent pursuit of shared objectives, allows humans and robots to complete tasks together that they could not do separately (Bradshaw, Feltovich, & Johnson, 2012b; Fan & Yen, 2004). For humans and robots to successfully engage in a joint activity, however, they must share a common ground—a shared set of knowledge, beliefs, and assumptions (Johnson et?al., 2014; Stubbs, Wettergreen, & Hinds, 2007). To establish and maintain this common ground, humans and robots must be able to communicate in some way (Chen & Barnes, 2014). With this understanding in place, both parties can predict one another's future actions and use that knowledge to coordinate on a shared task (Bradshaw et?al., 2012b; Sycara & Sukthankar, 2006). This need for mutual understanding differentiates autonomous robots from their more teleoperated brethren (Chen et?al., 2018). Robots with autonomous capabilities have their own “understanding” of a situation and use that to make decisions independently from their human counterparts (David & Nielsen, 2016). Given the agency of these robots, humans may need to treat robots more like their human teammates (Morrow & Fiore, 2012).
The role of consumer robots in our everyday lives
Heather C. Lum, in Living with Robots, 2020
Applications of companion robots
Children
One area, where human–robot interaction has been studied is the application of robotic pets for children and how they distinguish robotic entities with live ones. One such study observed the spontaneous interaction behavior when children and adults interacted with either a Sony AIBO robotic dog or a real dog. There was no difference in amount of time before first tactile interaction with the entity or length of time spent touching the real dog versus the AIBO (Kerepesi, Kubinyi, Jonsson, Magnusson, & Miklosi, 2006). However, another study focusing on children's beliefs about robots revealed that they do indeed make a distinction between live and robotic entities (Bernstein & Crowley, 2008). After interacting with eight separate entities, two of which were robotic, the children made unique classifications for the humanoid robot and rove along the biological and intelligence characteristic spectrum. This was also true for the psychological characteristics attributed to the robotic entities when compared with the people, cat, computer, and doll. In this instance, the robotic entities were considered to have more psychological characteristics attributed to them than the computer or doll, but far less than the cat or people. Children in this study, did make unique distinctions between live, robotic, and other entity types on behavioral and psychological markers.
Another study examined the beliefs that preschool-age children had of AIBO and a stuffed animal dog. When these children interacted with either entity their beliefs about the “animal” and ways that they behaved with it were not consistent with each other. In this study, the same proportion of preschool children attributed mental states, social rapport, and moral standing to both AIBO and the stuffed animal. From the behavioral interaction, it is suggested that preschool children treated AIBO like it was more capable of making its own decisions than the stuffed dog (Kahn Jr., Friedman, Perez-Granados, & Freier, 2006). Yet another study found that when children were put in an interaction situation, they were just as likely to talk to AIBO as they were a real dog. Despite this, the children conceptualized the real dog as having more physical essences, mental states, sociality, and moral standing than AIBO (Melson et?al., 2009). This suggests that while people may treat an AIBO and real dog similarly, they may have different beliefs or attributions about them.
Interestingly, although children are encouraged to take part in a wide range of robotic competitions, robots are still frequently designed from an adult perspective, ignoring children's perceptions and attitudes about robots (Druin, 1999). If successful robots are to be designed and used within educational curriculum activities, children should be at the forefront of the design course and suitable methods for obtaining children's views should be designed and utilized.
Intent Recognition for Human–Robot Interaction
Richard Kelley, ... Mircea Nicolescu, in Plan, Activity, and Intent Recognition, 2014
14.2.2 In Robotics and Computer Vision
Unfortunately, both the logical and the Bayesian approaches just described are difficult to apply in human–robot interaction. There are a number of difficulties, largely dealing with resource constraints and the need to produce estimates at a rate of up to 30 hertz (Hz). We detail these issues later but here provide some discussion of methods that computer vision researchers and roboticists have used to predict intention in humans.
Previous work on intent recognition in robotics has focused on significantly simpler methods capable of working with sensor data under challenging time constraints. Much of the early work comes from the computer vision community or makes extensive use of computer vision techniques. Many of the systems that have aimed for real-time operation use fairly simple techniques (e.g., hidden Markov models).
Whenever one wants to perform statistical classification in a system that is evolving over time, HMMs may be appropriate [4]. Such models have been successfully used in problems involving speech recognition [5]. There is also some evidence that hidden Markov models may be just as useful in modeling activities and intentions. For example, HMMs have been used by robots to perform a number of manipulation tasks [6–8]. These approaches all have a crucial problem: They only allow the robot to detect that a goal has been achieved after the activity has been performed. To the extent that intent recognition is about prediction, these systems do not use HMMs in a way that facilitates the recognition of intentions. Moreover, there are reasons to believe (see Section 14.5.1) that without considering the disambiguation component of intent recognition, there will be unavoidable limitations on a system, regardless of whether it uses HMMs or any other classification approach.
The problem of recognizing intentions is important in situations where a robot must learn from or collaborate with a human. Previous work has shown that forms of simulation or perspective-taking can help robots work with people on joint tasks [10]. More generally, much of the work in learning by demonstration has either an implicit or an explicit component dealing with interpreting ambiguous motions or instructions. The work we present here differs from that body of research in that the focus is mostly on recognition in which the human is not actively trying to help the robot learn—ultimately, intent recognition and learning by demonstration differ in this respect.
The use of HMMs in real-time intent recognition (emphasizing the prediction element of the intent-recognition problem) was first suggested in Tavakkoli et al. [9]. That paper also elaborates on the connection between the HMM approach and theory of mind. However, the system proposed there has shortcomings that the present work seeks to overcome. Specifically, the authors show that in the absence of addition contextual information, a system that uses HMMs alone will have difficulty predicting intentions when two or more of the activities the system has been trained to recognize appear very similar. The model of perspective-taking that uses HMMs to encode low-level actions alone is insufficiently powerful to make predictions in a wide range of everyday situations.
Sensors and actuators
Ercan Altinsoy, ... Hans Winger, in Tactile Internet, 2021
10.3.2 Smart vision
In addition to the development of audio- and haptic-feedback sensors and actuators, visual feedback is important for completing the entire human–robot interaction within TP2. Therefore strong collaboration of the Chair of Acoustic and Haptic Engineering and Fraunhofer FEP is required. In this respect, wearables (such as near-to-eye displays), also referred to as eyeables or smart glasses, as given in Fig. 10.6, are developed at Fraunhofer FEP. A typical OLED-on-silicon microdisplay architecture is schematically shown in Fig. 10.7. The microdisplay consists of a silicon wafer with integrated Complementary Metal Oxide Semiconductor (CMOS) electronics and the patterned anode. Different OLED layers are deposited by thermal evaporation in ultrahigh vacuum. Finally, color filters and encapsulation are put on top. Typically, OLED microdisplays consist of a white emission layer and lithographically etched color filters. Within the last years, several micropattering approaches (e.g., fine metal masks, lithography, e-beam direct writing) have been investigated to realize red, green, and blue OLED pixels instead of the white OLED in order to increase efficiency, color gamut, and contrast. However, all these techniques came along with new challenges, such as resolution limitations or yield issu
s.
Increasing pixel density for high-resolution, reducing latency and power consumption, and the application of multicolor displays are the key challenges for OLED microdisplays nowadays. Therefore new OLED-on-silicon display backplane architectures are required and need to be implemented in a deep-submicron process node. Besides new backplane circuitry concepts (see Section 10.4.2), the OLED structure itself needs to be highly efficient and stable. This requires a careful design of the electronic, optical, and excitonic properties. To meet the criterion of long lifetime, typically fluorescent emitters are used in OLED microdisplays, which allow only for maximum 25% internal quantum efficiency due to spin statistics. Phosphorescent emitters allow for 100% internal quantum efficiency, but they are expensive and the lifetime, especially of blue emitters, do not meet industry requirements.
Read more
Human–robot cohabitation in industry
Uwe A?mann, ... Ariel Podlubne, in Tactile Internet, 2021
3.1.2 Categorization of cobotic cells
Not surprisingly, we find that the definition of classic robotic assembly cells commonly used in the robotic literature [168,169] to be slightly in disagreement with cells defined in more human–robot interaction/collaboration-related literature [170–173]. The former tend to regard robotic assembly cells in the sense of static entities assigned to one specific task, more or less located within a cage, where humans are absent. The latter, the collaborative robotic cell, is a product which operates with a given feature set to provide value to its customers [168] and is thus considered a “value-adding” process [168].
Cobotic cells as collaborative robotic cells? In accordance to the literature, we assume human interaction to be unconditionally involved in the manufacturing process in a collaborative robotic cell. Though a distinction can be made between a conventional collaborative manufacturing cell and a hybrid manufacturing cell, both involve human–robot interaction in a shared space to carry out a collaborative task. For example, as stated in [170–172], a hybrid manufacturing cell has the primary motivation to minimize the cost of resources by adopting the human flexibility to carry out specific tasks that are too difficult for a robot to perform. In hybrid cells, the workspace between robots and humans is explicitly shared, and direct interaction with each other is desired. To give an example, the hybrid assembly cell is regarded as a workshop “where the semi-finished products are assembled by humans and robots together” [171, p. 1067]. In contrast, in a conventional collaborative manufacturing cell, the human and robot work alternatively [171], hence, “each cell [is] responsible for a complete unit of work” [174, p. 477]. This implies also a distinct separation of the workspace between both coworkers, robots, and humans. We can observe that this strict separation may also be nullified to consider disruptive nonpredictable events, such as path intercepts as investigated by Unhelkar et al. [173] in the course of performing a logistic-related task (e.g., collecting items from a depot, which the robot fills). Such work infers the inclusion of human-aware motion planners solely for safety reasons, but not for interactive collaboration.
Nevertheless, in both forms of manufacturing cells, the robot is aware of the human to varying extents, whether it be simple collision detection and path replanning, or direct interaction, adaption, and optimal rescheduling (more sophisticated approaches concerning interaction) to adapt to the continuously changing human behavior and performance, or unexpected internal/external events.
In conclusion, we can give an informal definition of a cobot and a cobotic cell with reference to [175]. Cobot are robots tightly collaborating with humans; this is realized in a cobotic cell, which is a hybrid cell without a safety cage. Therefore these robots must be aware of human movements and autonomously adapt their behavior to prevent accidents with humans or other robots. In our use-case a cobot could be either a human-controlled or an autonomic working robot.
Distributed cobotic cells and avatar stations? In addition to cobotic cells situated in one location, the Tactile Internet also enables their distribution. Two stages of distributed human–robot coworking are discussed, the spacial distribution of sections of a cobotic cell as well as immersive control of a remote robot using avatar stations.
Distributed cobotic cells A cobotic cell can be local, so that workers can see all robots and work pieces. However, there are industrial scenarios for remote work, in which experts in remote locations have to inspect a cobotic cell (remote inspection) or certify the quality of a product from remote (remote certification), and other experts have to maintain a machine in the remote cell (remote maintenance), or in which they have to repair a machine remotely (remote repair). Furthermore, more complex scenarios with both experts and robots spread across different locations are possible. In all these situations, a distributed cobotic cell is needed, which couples at least two subcells in different locations over the Tactile Internet. Such distributed cobotic cells must guarantee the same functional and nonfunctional properties as local cobotic cells. Due to the increased complexity of a distributed system, more advanced concepts for design, programming, and error handling must be used.
Avatar stations in immersive cobotic cells As outlined in the introduction, a cobotic cell may incorporate avatar stations. Then, we speak of an immersive cobotic cell. An avatar station provides both motion-tracking and manipulation mechanisms to control a remote robot avatar and sensory feedback to the operator. Thus the avatar station is used as an entry point for a human user to immerse into the remote robotic body located in the cobotic cell. The aim is to completely immerse the users, i.e., give them the illusion as if they were physically at the remote location. In Section 3.2.3, we will present a reference architecture for immersive cobotic cells.
With the development of an anthropomorphic robotic hand–arm system, we are building on novel concepts and aiming to fully project the user to a remote location, at an appropriate level for industrial use cases. To immerse the user, the robotic system has to be able to provide proper tactile and audiovisual feedback to the user and must possess real-time responsiveness. Such specifications present not only a major challenge to the communication infrastructure, but also to the technologies used to establish the impression of remote presence. High-performance wearable technologies,