Using Augmented Reality to Improve Productivity and Safety for Heavy Machinery Operators: State of the Art

The machinery used in industrial applications, such as in agriculture, construction, and forestry, are increasingly equipped with digital tools that aim to aid the operator in task completion, improved productivity, and enhanced safety. In addition, as machines are increasingly connected, there are even more opportunities to integrate external information sources. This situation provides a challenge in mediating the information to the operator. One approach that could be used to address this challenge is the use of augmented reality. This enables the system-generated information to be combined with the user’s perception of the environment. It has the potential to enhance the operators’ awareness of the machine, the surroundings, and the operation that needs to be performed. In this paper, we review the current literature to present the state of the art, discuss the possible benefits, and the use of augmented reality in heavy machinery.


INTRODUCTION
Industrial machinery and heavy vehicles, like those used in agriculture, construction, and forestry, hereafter referred to as heavy machinery, is increasingly equipped with more sensors and higher computation capacity. This evolution is driven by the needs for new features, some of which could improve the operation's productivity and safety. The transition into more digitally powered Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. VRCAI '19, November 14-16, 2019, Brisbane, Australia © 2019 Association for Computing Machinery.  systems subsequently means that the information exchanged with the user, i.e. the operator, is progressively becoming a designed interaction between the system-generated information and the operator. Moreover, as machines are connected, more information could be exchanged between the machine and external information systems, as well as between machines. This evolution has many benefits, but there might also be negative effects in terms of divided attention, information overload, and operator's stress. Operating modern heavy machinery already requires significant cognitive workload from the operator. For example, the workload to operate a forest harvester is comparable to the workload to operate a fighter plane [Burman and Löfgren 2007]. A further increase in the information load can potentially lead to a higher risk of human errors, which could harm not only the operator, but also people and property near the machine [Akyeampong et al. 2014;Chen et al. 2011].
One approach to mitigate the negative effects of the increase in information being presented is the use of augmented reality and see-through interfaces. Using augmented reality, where the system-generated information is blended with the user's perception of the environment, has the potential to enhance the operator's awareness of the machine, the surroundings, and operation at hand. In addition, using see-through interfaces allows the operator to see the presented information without completely obstructing the operator's view on their surroundings. Following is a short fictive scenario to illustrate this: The excavator operator is tasked to reshape the ground on a construction site. Due to the augmented system, the operator can see directly the different levelling areas and their needed adjustment, based on the blueprints and scanning of the surroundings. The system also presents and alerts about underground infrastructures, such as pipes and power lines. Hence, the operator can focus on the operational area instead of diverting attention to traditional head-down display based precision systems. Meanwhile, an autonomous dump truck is approaching the excavator. The dump truck is carrying the gravel to fill an area. Upon arrival, the excavator system displays an alert in the visual range of the operator, indicating there is movement in the vicinity. The planned passage and the designated unloading area for the dump truck operation are also presented and the operator can acknowledge the planned passage. This example can be further extended with additional information, for example, pedestrian information, sensory data, full 3D visualization of the construction site, etc.
Fully realizing such a scenario may be a long way off, since it requires a lot of information integration from different systems and well-designed interactions. However, how close have we come? In this paper, we look at the current state of the art in research that utilizes augmented reality and see-through interfaces to aid the operator of heavy machinery. Section 2 describes the way we collected the relevant articles. Section 3 details the different augmented information depending on the type of operation and heavy machinery, including the results from user evaluations, if any. Section 4 discusses the findings based on technological approaches, existing user evaluations, design implications, and users' acceptances. Section 5 concludes the study in this paper.

METHOD
The information presented in this is paper is based upon a semistructured way to collect relevant literature. We utilized Google Scholar to find relevant articles. Since there are various types of heavy machinery and Google Scholars has a limitation on the length of the search string, we could not cover everything in one attempt. To mitigate this issue, we searched the relevant articles using the following search string: ("augmented reality" OR "mixed reality" OR "headup display" OR "head up display" OR "immersive display" OR "transparent display" OR "transparent screen" OR "see-through display") AND We then wrote the name of the heavy machine after the word 'AND' in the end of the search string above, for example, 'excavator'. The process was then repeated for other types of heavy machinery. We also used general terms, such as 'heavy machine', 'off-highway vehicle' and 'earthmover' in order to ensure that we have covered almost everything. Following is the complete list of additional strings that we used: tractor, harvester, forwarder, excavator, digger, crane, dumper, dump truck, dozer, loader, driller, mobile drilling rig, pile driver, paver, grader, scraper, forklift, industrial vehicle, off-highway machine, off-highway vehicle, heavy vehicle, heavy machine, heavy equipment, mobile machine, utility vehicle, work machine, work vehicle, and earthmover. There was no timeframe limitation set when searching for relevant articles. Therefore, articles published until September 2019 and accessible were covered during the search process.
For each search attempt, we manually checked the titles of the articles given by Google Scholar in order to determine whether they were related to heavy machinery. This process gave us 233 articles in total. Lastly, we manually checked whether the articles describe something about augmenting the operator's view, either as concepts, within simulated environments, or in real-world settings. For articles describing mixed reality or virtual reality simulations for training new operators, but with no description about information being augmented to the operator is mentioned, these articles were also discarded. This last step left us with 39 relevant publications that we then reviewed in detail.

RESULTS
Based on the type of operation, we found that augmented information is provided in two types of operation: in-cabin operation and remote operation. Within these areas, the kind of information being augmented is diverse and it greatly differs depending on the type of heavy machinery. Some articles also describe user studies, where the proposed systems were tested by test users in order to measure the impact on users' performance and preference. More information is further described in the following sub-sections.

Type of Operation
The traditional setting to operate heavy machinery is by controlling the machine inside the cabin. While this setting is still generally applicable, some efforts have been made to move the operator from working inside the cabin to working in a remote control station [Chi et al. 2012[Chi et al. , 2014Wang and Dunston 2006;Yang et al. 2015], which is also referred to as tele-operation. Both in-cabin and tele-operation have its own challenges in terms of having sufficient visibility of the environment. This sub-section further describes the difference on augmented information between in-cabin operation and tele-operation.
3.1.1 In-cabin Operation. Regarding in-cabin operation, it is known that the machine's physical structure can obstruct the operator's view . For example, the presence of the excavator's boom and attachment and the loader's bucket may occlude the operator's view, thus the operator is unable to have a clear and complete view of the environment around the machine. Having see-through visualization could allow the operator to have a clearer view on what is in front of the machine.   proposed using diminished reality in order to produce see-through visualization, where the machine's physical structure can be made transparent from the operator's point of view (see Figure 1). If augmented reality enhances the physical view of the user [Caudell and Mizell 1992], diminished reality removes some real objects from the user's view, by replacing them with background textures [Kawai et al. 2013]. In their study using a simulated environment, although having transparent boom is beneficial for the operator,  found that there is no significant difference on participants' performance between 50% transparent and 80% transparent. The participants also noted that higher transparency could cause more collisions, since they are not aware of the machine's structure. Alternatively, the occluded view can also be mitigated by visualizing the environment around the machine (see Figure 2). Palonen et al. [2017] proposed using point clouds to visualize the view in front of the machine's structure, without making it transparent.
3.1.2 Tele-operation. One of the main motivations for tele-operation is to enhance the operator's safety. When working inside the cabin, the operator is also exposed to any accidents that occur near the machine. Using tele-operation, the operator is completely protected from accidents around the machine that is being controlled.
Having sufficient visibility is a bigger challenge in tele-operation, since the operator's vision is limited to the field of view of the cameras attached on the machine [Fang et al. 2018]. In addition, since the view is obtained through the video stream displayed using 2D displays, the view that the operator receives often lacks the depth of field perception of normal vision, which could diminish the operator's ability to pinpoint the exact location or trajectory of an object.
Augmented reality solutions have also been evaluated for teleoperation. The solutions can be broadly described as one of two kinds: using camera views with overlaid augmentation, or as complete virtual representations of the remote machine environment. The example of augmented overlays of the video stream for remote operators can be seen in Sarupuri et al. [2016], where the forklift operator is supported with visual cues to navigate the forks while operating a forklift remotely. The overlaid video stream includes the visualization of fork alignment towards the pallet, depth cues of forks depth into pallet, and visual guidelines for pallet racking (see Figure 3).  [Sarupuri et al. 2016] The latter type of visualization, where the whole scenery is virtually presented, should more suitably be categorized as augmented virtuality. Such example is presented in Yang et al. [2015], where visual cues in form of guide laser, reachable sphere, distance arrow, distance labels, and rotation hints that are used to guide the remote operator to determine the machine's physical position in relation to the working area and the objects being worked with. Even though these use cases are about support to remote operators, the information that they show could also provide inspiration to design visual support for in-cabin operations. Thus, these studies are highly relevant since they can indicate possible use cases for operators working within the real machine.
As automation is expected to be progressively prevalent in the future, the operator's role may change from the controller of the machine to the supervisor of the machine. In this case, the operation is not only done remotely, but also reduces the controlling workload. Halbach and Halme [2013] developed a prototype of an augmented reality tele-operation system, where the operator can remotely assign work to an automated wheel loader, and then the operator received the augmented information on the progress of the work being done by the machine (see Figure 4).

The Use of Augmented Information and Its Impact in Different Types of Heavy Machinery
Since there are various types of heavy machinery, the kind of augmented information relevant to the operator is also different. The following sub-sections describe various kinds of information being augmented in different machines. The results of user evaluations are also described, if any.
3.2.1 Cranes. Cranes are one of the typical machines found on construction sites. Cranes play a vital role in lifting and distributing materials across working sites. Different types of cranes can be utilized depending on the objective. For the sake of simplicity, cranes are generally divided into two categories here: fixed-position cranes and mobile cranes. Since cranes are considered to be dangerous machines, the augmented information is often about parameters that can prevent collisions or the loss of balance. In mobile cranes, the augmented information is related to collision, excessive load, obstructions, and recommended lifting path [Fang and Cho 2017] (see Figure  5). Similar information can also be found in assistive systems in fixed-position cranes, such as tower cranes [Chen et al. 2011;Park et al. 2011] and off-shore cranes [Kvalberg 2010]. The recommended lifting path is not limited to showing the safest path, but also the most efficient path for lifting and moving the object.
In a study that compared a traditional indication system and an augmented reality system in real mobile cranes, Fang et al. [2018] found that real operators have shorter response time and higher rate of correct responses when using the augmented reality system. The result from the situation awareness measurement method SPAM indicates that operators were able to maintain a higher level of situation awareness by using the augmented reality system. In the context of tele-operated cranes,  found that the presence of augmented information provides insignificant impacts on participants' performance. In their study, participants were asked to operate a tower crane remotely using multiple views with augmented reality and multiple views without augmented reality. Based on participants' completion time and number of collisions, there is however no significant difference between using multiple views with augmented reality and multiple views without augmented reality.

Underground Drillers.
Underground drillers are used for making tunnels. Unlike other heavy machinery, underground drillers are used in dark environments.  proposed to augment the operator's view with information related to the drilling progress and the distance between the drilling bit and the drilling target (see Figure 6). In their study, they also utilized a simulated environment for visualizing the augmented information in two different places: on the machine's windshield and on the target area. According to participants' completion time and drilling accuracy, there is no significant difference observed whether the information is augmented on the windshield or on the target area.

Rollers.
Rollers are machines used for compacting asphalt that has recently been laid down by a road paver. Vahdatikhaki et al. [2019] proposed a training simulation that augments information related to the asphalt's compactness level and temperature directly on the recently paved asphalt (see Figure 7). Recently paved asphalt is hot and knowing the temperature is essential, since there is a range of temperature where it is best to compact the asphalt. Figure 7: The left image shows the compactness level of newly paved asphalt, where red color and yellow color respectively mean the asphalt needs to be compacted three and two times more. The right image shows the temperature of the newly paved asphalt, which is visualized using the gradient color from red to blue [Vahdatikhaki et al. 2019] 3.2.4 Trucks and Tractors. The augmented information for trucks and tractors differs greatly depending on the context. Trucks and tractors are versatile machines, since their purpose can be changed depending on the attachment being used. In general, the most common augmented information in trucks and tractors is related to navigational information (see Figure 8), such as recommended routes, turning signals, and lane boundaries [Kaizu and Choi 2012;Rakauskas et al. 2005;Santana-Fernández et al. 2010]. This kind of information can also be found in augmented reality applications for conventional cars [Narzt et al. 2006]. What makes trucks and tractors different from cars is the fact that the operator needs to drive the machine precisely. In case of snowplowing trucks, the entire road could be covered with snow, which makes it difficult for the operator to know where is the boundary between the road and the sidewalk. Therefore, it is beneficial to know whether an area of the road should be plowed or not Rakauskas et al. 2005]. In case of agricultural tractors, the operator needs to know which areas that have been treated and have not been treated, since the difference between treated and untreated areas are unclear in some cases [Santana-Fernández et al. 2010].
In their study, Rakauskas et al. [2005] asked operators to drive a snowplowing truck in a low-visibility condition with and without the use of an augmented reality head-up display. The results show that the presence of the augmented information leads to slower speed, more responses on the steering wheel, and more precise driving position compared to when the augmented reality head-up display is not present. However, the result from the RSME questionnaire indicates that operators have higher mental workload when the augmented reality head-up display is present. This is probably caused due to the additional effort to process the presented information and then act accordingly.

Forklifts.
Forklifts are often used in warehouses for moving and lifting materials. The augmented information in forklifts is related to navigating around working sites, the location of the target material, and the destination for the target material [Pereira et al. 2016;Pettersson and Stengård 2015] (see Figure 9). In addition, Pettersson and Stengård [2015] also proposed to include cargo's related information, forklift's indicating values, and proximity warnings. In case of tele-operating forklifts, Sarupuri et al. [2016] also proposed to visualize the fork's alignment and depth cues towards the pallet in order to assist the operator when picking up a material from the rack. Figure 9: The left image shows which object that should be picked. The middle image illustrates the recommended route for navigating within the warehouse. The right image represents the destination where the object should be placed [Pereira et al. 2016] In the study that compares the augmented reality visualization using a virtual head-up display and a virtual head-mounted display, Pereira et al. [2016] found that there is no significant difference on participants' performance. In addition, participants also do not have any preference towards head-up display or head-mounted display. The opposite results were found in a study in the context of tele-operating forklifts. Sarupuri et al. [2016] found that there is a significant reduction of bumps when participants were equipped with augmented reality information. Moreover, the participants also felt more comfortable in operating a remote forklift in the augmented reality condition.
3.2.6 Excavators. Excavators are another versatile vehicle for doing different tasks on site, such as digging trenches, preparing property grounds, loading trucks, and moving materials on working sites. An excavator can even do more tasks, using different attachments to its boom, such as drilling, tree harvesting, etc. Modern excavators are already equipped with Geographic Information Systems that let the operator see the blueprint of the working site on displays inside the cabin and cameras to see what is behind the machine. Still, operating an excavator is a demanding task with high demands on efficiency and precision, while maintaining awareness on the surroundings to avoid hitting into objects or humans.
Augmentation in excavator applications uses head-up displays [Wallmyr et al. 2018], video-based displays [Akyeampong et al. 2012;Talmaki et al. 2010], and projection displays [Behzadan and Kamat 2009]. Examples of use are notifications, augmented views of hidden objects, and blueprints of the area presented on the ground [Su et al. 2013].
Based on the collected articles, there is still limited knowledge regarding the real setting, since all of the existing evaluations were done in a simulated environment. Using transparent surfaces, such as head-up displays, has been shown to reduce the operator's workload and increase their detection of presented information (see Figure 10). Akyeampong et al. [2014] conducted a study which compared the workload when using head-down-based design and head-up-based design. They found that the head-up-based design reduced up to 35% workload. Moreover, [Wallmyr et al. 2019] evaluated how operators can benefit from having warning and navigational information near line of sight. The results showed that having information near the operator's line of sight leads to significantly higher rate of information acquisition and lower rate of mental workload.  pong et al. 2014]. The right image shows navigational information and obstacle warning that are also presented near the line of sight [Wallmyr et al. 2018] 3.2.7 Forest Harvesters. Forest harvesters are one of the most advanced heavy machines, especially the ones used in the cut-tolength (CTL) method. In the CTL method, the operator drives the machine, while moving the boom and the harvester's head to the target tree. The target tree is then cut to optimal lengths, taking account data about pricing, tree length, diameter, and quality. The operator is assisted by advanced process systems that present operational data on head-down displays inside the cabin.
The use of augmented reality support has been envisioned to support navigating the machine, as well as visualizing the boom's reach and detailed information of the stem's health [Nordlie and Till 2015]. However, the main evaluations so far are related to moving the information currently presented on machines head-down displays into the operator's field of view using head-up display technology [Erikssohn and Oscarsson 2005] (see Figure 11).
Furthermore, what may be interesting is how forestry, as one of the few areas of industry, have performed operator studies in real world settings. These studies show that operators are positive about moving the information related to the processing task from head-down displays into the line of sight [Englund et al. 2015].
Figure 11: The image shows the view from the cabin operator's position when doing the cutting operation. The operator is supported by information from the bucking optimization system presented on a head-up display, seen in pink in the middle of the image [Englund et al. 2015] In terms of performance and workload, the findings are mixed. Lundin et al. [2005] found no significant difference in terms of performance, while Järrendal and Dillekås [2007b] found that operational time and mental workload were respectively reduced up to 13% and 27%. Although significant improvements were found, the study is limited in terms of the number of participants and should only be seen as an indication.

DISCUSSION
In this section, we discuss technological approaches, impacts of augmented information on users, design implications, and users' acceptances on a general level across different heavy machinery.

Technological Approaches
Realizing augmented reality solutions in the heavy machinery industry is related to both technical challenges and usage challenges. There are several technological solutions to visualize augmented reality in the context of heavy machinery, each with its pros and cons [Wallmyr 2014].
Using see-through technologies, such as head-mounted displays and head-up displays, is an approach that is gaining interest, representing roughly 25% of the cases reviewed. This way, the user does not have to look away from the operation area to acquire system information, and it would also provide a larger perceivable area for information presentation.
Head-mounted mixed reality displays [Kress and Starner 2013] have, for example, been used to replace blueprints at property building sites [Nohrstedt 2017]. However, these devices still have severely limited operation time and the operator needs to take extra effort including putting on the equipment before being able to operate or to move in and out of the machine.
Another see-through technology studied is the head-up display, where the whole cabin windshield is used to present information instead of only the head-down display, opening up more possibilities in terms of possible interaction designs. Although head-up displays have become more available, there are some limitations that inhibit their full potential. They currently lack a big enough display area to cover the large windshield of a heavy machine, in order to present all the necessary information in the relevant field of view. Also, most heavy machinery lacks dashboards, where automotive head-up displays are currently placed. It would thus require more development of mechanical and optical solutions, to fit them into the cabins of heavy machinery and present the information in the optimal way.
Alternative technologies, such as transparent OLED displays have issues sustaining in the harsh environment and are not seen in the reviewed literature. This technological development is a challenge since advanced machines are also produced in relatively moderate volumes, thus they cannot underwrite the large cost to make the fundamental technology development needed [Järrendal et al. 2007]. There is a need for research of visualization technologies with an affordable cost that can fit the environment of heavy machines and support a wide range of machines and use-cases.
Video-based augmented reality is also an alternative. This approach is more commonly evaluated in tele-operation environments to support the remote operator's lack of sensory input. However, it could also be used to aid operators inside the machine. One of the benefits is that many parts of the required technology are already available and maturing. Modern heavy machinery is now commonly equipped with graphical displays and cameras that present information from the surrounding area or from the production process. Using video-based augmented reality, virtual information can be overlaid on the video feed captured by cameras around the vehicle. The combined video feed is then presented to the operator. Using augmented reality to enhance the information presented on the cabin's displays would let the operator be aware of the surrounding environment, while simultaneously receiving benefits from the information available via the machine's information systems.
Another approach is the use of projected augmentation, where information is projected over the objects of interest in the real world. The main benefit of this approach is that the users do not need any display to see the virtual content. However, it also limits the visualization to certain locations, and it has to cope with visibility and lighting in bright environments.
Another alternative, which is already maturing, is the use of mobile devices, such as smart phones and tablets. These can be used, for example, in mobile mixed reality interfaces where information is overlaid on the display of the device [Seichter et al. 2013;van Krevelen and Poelman 2010]. However, this alternative is limited in terms of practicality, since the user's hands are occupied in order to hold the device. Based on the collected articles, we did not find any applications that utilize or evaluate this approach. Still, it can be a good complement when the operator is outside of the machine or not actively in hands-on activities.
Moreover, the use of transparent displays is more common in simulated environments, indicating that these technologies are of interest for future designs to support safe and efficient operator interaction. More research is therefore needed to fully understand the requirements and to evaluate the potential of transparent displays, such as head-up displays and head-mounted displays, in the heavy machinery domain.

Will Augmenting Information Benefit the
Operator?
Although not all articles have involved users to evaluate the proposed systems, a meta-review can still be done by looking at the results of user studies that are described in Sub-section 3.2. Here, the evaluations are grouped into three categories: with or without augmented reality, evaluations of different placement of information, evaluations that involved professional operators, and evaluations that were conducted in real-world settings. Firstly, we look at the studies that compare the condition with and without augmented information. In the context of dump trucks, we can see that the presence of augmented information leads to reduced driving speed, shorter response times, and more frequent responses made on the steering wheel [Rakauskas et al. 2005]. Similar positive results were also found in mobile cranes [Fang et al. 2018], forklifts [Sarupuri et al. 2016], and excavators [Wallmyr et al. 2018[Wallmyr et al. , 2019. However, the opposite findings were found in the context of tele-operating cranes.  found that the there were no significant impacts on participants' performance whether the multiple views are augmented or not.
Secondly, we investigate the studies that compare different locations for presenting the augmented information. In the context of underground drillers, there was no significant difference on users' performance whether the information is presented on the windshield or directly on the object of interest . A similar finding is also found in the context of forklifts [Pereira et al. 2016]. While presenting the information closer to the user's line of sight definitely improves the user's rate of information acquisition, there is a little difference whether the information is presented on the windshield or directly in the environment [Wallmyr et al. 2019].
Thirdly, most studies so far have been performed in simulated environments. This is understandable since experiments can then be standardized, reproduced, and controlled under more stringent conditions [de Winter et al. 2012]. The potential effect on mistakes, crashes, or the risk to human life can also be mitigated when moving away from the real world. The involvement of professional users is generally also low, since only 11 articles involved real operators in their evaluations. Among these articles, only three studies involved more than 10 operators and the average number of operators involved in these 11 articles is six operators. The rest of the studies only tested the technical capability or involved non-professionals. There are two articles that involved both real operators and nonoperators in the studies within simulated environments, thus their performance can be compared. While there are similarities on the performance between real operators and non-professional users, there are differences as well. Chi et al. [2012] found out that real operators looked at the presented information more frequently than non-professional users, while  discovered that real operators acted more safely, thus they required longer time to complete the given tasks. Real-world evaluations are even more limited with the biggest real-world study involved nine operators [Rakauskas et al. 2005].
Most of studies in real-world settings utilized different kinds of head-down displays and video-based augmented reality. Based on the reviewed literature, there are only a few real-world examples of augmented reality for heavy machinery using transparent displays. A reason for this can be the lack of technological readiness. The cases, where see-through displays have been used in real-world settings, are also diverse with different types of machines and with a diverse set of displays, such as dump trucks with projection displays [Rakauskas et al. 2005], off-shore cranes with electroluminescence displays [Kvalberg 2010], tractors with head-mounted displays [Santana-Fernández et al. 2010], and forest harvesters with head-up displays [Englund et al. 2015]. This makes it difficult to draw any firm conclusions for these kind of visualization technologies. Still, the indications tend to be positive with improved performance in terms of safe operation, higher or similar efficiency, and lower workload.

User Experience and Design Implications
For see-through visualization technologies and augmented reality interfaces to become successful in the context of heavy machines, there are several usability challenges that need to be considered and further evaluated. One of the major design implication is that information should be presented in a safe way, where the important information in each situation is clearly highlighted and information overload should be avoided [Politis et al. 2014].
The selected technology also needs to be on par with the usability requirements. Different kinds of technical solutions, such as hand-held devices, head-up displays, and head-mounted displays, etc., for presenting information are more or less appropriate for different usages. For example, head-down displays offer high fidelity in visual quality, but the operator must actively look at the screen. On the other hand, the projection displays enables information visualization within the operator's field of view at the place of operation, but it is more difficult to implement and environmental conditions should be taken into account more carefully.
Based on the diverse range of machines, applications, and evaluations, we feel that it is still too early to draw a clear guidance on how to develop an interface for augmenting information in the context of heavy machinery. Still, the reviewed literature gave input on operators' experiences and preferences. Some examples are that grouping of information and having a clear placement was something positively perceived by operators [Erikssohn and Oscarsson 2005], although there are different findings on where the information should be best placed. The use of colors was also something that the users appreciated [Järrendal and Dillekås 2007a]. When to show and not to show information shall also be considered for supporting operators' confidence [Nordlie and Till 2015]. To be able to show the system's confidence on the presented information is also something that can increase users' trust Su et al. 2013].
Moreover, while augmented reality can be used to present important information, such as warnings, that could be overlooked when using head-down displays, it does not guarantee that the information will always be detected. The study, which presents warnings using head-up display and projection display, shows a higher rate of information acquisition and lower mental load, but critical information is still overlooked [Wallmyr et al. 2019].
Additionally, operators of heavy machinery spend long hours sitting in the cabin and the ergonomic factors have to be considered, where augmented reality technologies can provide benefits [Akyeampong et al. 2013;Järrendal et al. 2007]. This also includes the way the user should provide input to the information system, where more advanced visualizations will probably create a need for renewed ways of interacting with the new interface, depending on the situation and task at hand. This discussion is however outside the scope of this paper.

Users' Acceptances
For augmentation technologies to be widely used in the context of heavy machinery, the operators should also be open to adopting new technologies. There are some studies that involved real operators for evaluating the proposed systems. While the feedback on the proposed systems is generally positive, there were also some operators that expressed reluctance in adopting new technologies. In the context of mobile cranes, although the operators were able to perform the given tasks just by relying on the assistive system, they do not believe that the proposed system can replace the role of signalers [Fang et al. 2018]. In addition to the lack of trust in the assistive system, the operators also expressed that, when an accident occurs, the responsibility will be shared between the operator and the signaler. If the signaler is no longer needed, then it is completely the operator's fault if an accident occurs. Another example of reluctance can also be found in the context of trucks. Although all truck drivers drove more safely and perceived the head-up display as useful, only 5 out of 9 drivers would want to have the head-up display in their vehicles [Rakauskas et al. 2005].

CONCLUSION
This paper has reviewed the current applications and state of the art of augmented reality and see-through interfaces in heavy machinery. The type of applications and approaches vary diversely, such as moving existing data and notifications presented at displays into the users' field of view, operator guidance integrated in the environment, and x-ray like applications where the structures of the machine are made transparent. The studies performed in the reviewed articles gave an indication that augmented reality can improve operators' productivity, confidence, and safety. Further studies are however needed to verify this, especially in real-world situations. Moreover, successful application of augmented reality applications in the heavy machinery domain will require further development of visualization technology and deeper understanding on how to create a safe, efficient, and good experience for the operator.