HUMAN-MACHINE INTERFACE RELEVANCE AND NEW REQUIREMENTS
The adoption of IoT, AI, Robotics and Unmanned Vehicles has resulted in an increased in the range, complexity and number of automated tasks conducted by machines. This is does not imply the full replacement of human labor rather a change of human jobs to provide direction and collaborate with machines which requires a new and improved human-machine interaction.
KETTL enabled machines in warehousing, logistics and manufacturing bring now workflows that include both physical and also digital work information. Their use requires constant and intense exchange of information, increasing the complexity of the interaction between human workers and machines.
Traditional human-machine interfaces are insufficient to facilitate enough context and efficient interaction between human and advanced machines, for that reason a wave of new interface technologies powered by AI and IoT and pioneered in the creative industries have been already applied to manufacturing and adoption, and it is rapidly extending to Transport and Logistics bringing an operational efficiency to a level not imagined a decade ago.
EXTENDED REALITY, DEFINITION AND CATEGORIES
This new wave interface technology is included under the umbrella of Extended Reality. These technologies enhance human senses, providing additional information, either about the actual world or through simulated worlds for humans to experience, with the objective of monitoring, managing and making decisions working with advanced machines. It includes Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) technologies.
Virtual Reality (V.R.)
VR implies a complete immersion experience in a computer-generated environment out the physical world. Even as VR has been used to simulate some physical operations or environments it has limited applicability in Transport and Logistics, and its development and adoption is taken place mainly in the creative sector and to a minor extend in retailing.
Augmented Reality (A.R.)
AR Is the next-generation human-machine interface to bridge the workflow gaps in tasks and maximize efficiencies of human and machine interactions. Unlike VR which require the user to inhabit a virtual environment, AR applications superpose digital information like digital 3D, computer generated graphics and images of the equipment on top of the user’s physical environment. AR then provides a more natural environment for workers to instruct advanced machines to actuate, often though human physical motion. Additionally, when machine tasks have to be altered AR offers a virtual, immersive and interactive dashboard that avoids costly stops for reprogramming.
Mixed Reality (M.R.)
Mixed Reality is an emerging technology that combines VR and AR facilitating working in a real-world environment using some virtual objects for example an immersive rendering of the internal components and functioning of a device that the worker is using.
The VR, AR and Mixed Reality elements powered by AI and IoT can be combined in different proportions to allow for specificities of the machine, workflow and tasks being conducted. Also proportions are adjusted for the tasks related to the machine being conducted by the human operator like monitoring, scenario testing, task modifications and overall optimization real-time. For example, recently improved VR simulations are used to make proactive, dynamic and computerized adjustments without written reports nor physical direct work on the machines.
NEW EXTENDED REALITY ADOPTION AND EFFECTS
The following table summarizes some of the uses of AR and VR in Transport and Logistics by companies, operators, planners and administrations. Also, the use of advanced human-machine interfaces is a requirement for the optimal use of other KETTLs already deployed, like IoT enabled machinery, unmanned vehicles and drones and advanced robotics.
| Data Access and Environment Interaction | Problem Diagnosis | Action Taking | |
|---|---|---|---|
| Enhance Human Capabilities | Equipment assembly, use and maintenance, Human training | Remote support, Fulfillment Claims | Navigation by virtual display in autonomous vehicles and drones |
| Manage Spaces | Functional design optimization in warehouses, urban spaces, factories and public use infrastructures, Inventory and in-transit monitoring, Disruption and safety response, Consumer virtual navigation of facilities or travel | Space and environment monitoring, Buildings and corporate or public facilities inspection | Spatial optimization for order packing, and shipment organization, Order picking |
| Manage Devices | Product, construction, urban and infrastructure design | Operations monitoring, Production line optimization, Quality Control, Equipment monitoring and diagnostics | Operator instruccions for large equipment and also precision processes,Task sequence and assembly work instructions, Asset installation, Maintenance instruction and guidance |
CETMO Analysis, adapted from Boston Consulting Group (1)
Regarding its maturity and adoption, creative and entertainment sectors are at the forefront followed by retailing. Also, the adoption of other KETTLs has generated the need for improved interfaces and accelerated its adoption by the Manufacturing and Transport and Logistics sectors.
While these new human-machine interfaces are a logical continuation of the deployment of other KETTLs, they also have some potential strong effects in customer relationship, adding a degree of empathy absent from automated digital interactions and the possibility to offer new customer experiences in particular in passenger transportation.
Regarding social and Country Disparity, AR, VR and MR are potent tools to improve economic, urban, agricultural and health care planning, plus they make for first time possible the management of infrastructures and services in underdeveloped and remote areas by specialized human operators located in distant central or regional operational centers.
CETMO Analysis, adapted from McKinsey & Company (2)
References:
- Yusuf, Lukic, Boston Consulting Group-BCG. Unleashing the Power of Data with IoT and Augmented Reality. 2020
- McKinsey & Company. Ashutosh, Hastings, Murnane, Neuhaus, Automation in Logistics: Big Opportunity, Bigger Uncertainty, 2019. link
This technologies, less mature the presented before are starting to reduce the risk of work accidents, improve human resource conditions and increase their skills. In passenger transport it allows to offer new services to users and improve their travel experience. Additionally, the simulation of real conditions and situations opens the door to new possibilities in planning, operations, business relations and support. Some effects include:
Freight Transport
Increased Use of Information and Efficiency: Augmented Reality facilitates the interaction between employees and things and allows to locate, extract relevant information, finalize tasks and solve problems more efficiently. For example, more agile physical search for cargo and merchandise, with less risk of losing merchandise, less time spent searching department stores, and less time for cargo operations.
Error Reduction: in particular Picking errors and general, any documentary and physical errors in shipments. It also allows to increase quality, efficiency and safety, in tasks such as maintenance or supervision.
Staff Training: simulating real situations in areas or contexts that are risky or expensive to reproduce or deal with. deal with in another way. Also offers the possibility of optimized cost-efficient training in all areas
Service Quality Increase: by creating simulations situations that allow to identify inefficiencies in processes and protocols, plus generating scenarios and situations not contemplated before. Additionally, simulations can be created to optimize operations and service by virtual trial and error at near insignificant additional cost.
Increased Risk Management Capabilities: simulations allow to generate risk situations not contemplated until then that facilitate the creation of contingency and emergency plans .
Passenger Transport
Increased Use of Information and Efficiency: similarly, to Freight Transportation, augmented reality facilitates the interaction between employees and things, extract relevant information, finalize tasks and solve problems more efficiently. In Passenger transport will, for example, facilitate and improve the process of identification and location of vehicles, etc., allowing more speed and greater control, both for the operating company and for the user.
Marketing, Pre-testing and Sales: Smart HMI change the way the products are sold (both from the supplier company to the operator and from the company to the user), as it will recreate reality in a more accurate way (for example, showing the simulation of future conditions or the visual comparison with the current ones).
Staff Training: similar case to Freight Transport.
Customer Experience Improvement: can improve the service experience for the user, facilitating the recreation and simulation of processes related to the different stages of the trip, such as the arrival at the door or platform, vehicle access and installation, vehicle exit and station access, or station exit. Additionally, it can be done while reducing support tasks and cost.
Increased Risk Management Capabilities: same case as Freight Transport.
Sources: CETMO and “Impacte de les KETs en la digitalització dels diferents àmbits del transport”, CENIT-CINESI – December 2020
With the support of:
Subscribe to our newsletter
Stay updated about all news and events