The construction industry is not immune to the effects (both positive and negative) caused by robotics, automation, and algorithms. These technological disrupters are going to shake up the industry from top to bottom. They will affect job site safety, operational efficiency, material productivity, operator training requirements, and necessary skill sets. This all will require changes to job site operations, labor force utilization, and managerial coordination. These advances may seem like something out of science fiction, but they are arriving faster than expected. Eventually, the current machine control and guidance system will evolve to true artificial intelligence applications.
Current Machine Control Technologies
What do we mean by “machine control”? “In civil engineering, machine control is used to accurately position earthwork machinery based on 3D design models and GPS (Global Positioning System), and thus aid machine operators to e.g. control the position of a road grader’s blade.” Broadly speaking, the primary means of machine control are laser-based display and grade control systems, three-dimensional (3D) grade control utilizing either GPS guidance systems or Advanced Tracking Sensor (ATS) Robotic Total Stations.
Laser-based displays provide visual guidance to the manual operation of the excavation bucket or dozer blade. Simpler and easier to use than other machine guidance systems, these systems can be customized to a particular job site or task while providing operational flexibility. Their core is a battery of receivers installed above the operator’s cab. Powered by a battery, the display system provides additional information to the operator to ensure efficient operation.
Laser-based grade control systems take this capability to the next level to create a fully automatic system. Utilizing integrated valve controls on the machine’s hydraulic systems allows for direct control of the dozer blade by the incoming laser signal. The information received from the laser signal controls the blade’s proportional hydraulic valve, correcting and adjusting the dozer blade.
3D Grade control achieves another level of sophistication by coordinating GPS signals with digital terrain models. These are preloaded in the machine control system and show both existing ground conditions and proposed grading, excavation, or trenching. The northing, easting, and elevation (X, Y, and Z) coordinates of the edge of the dozer blade or tip of the bucket are provided by sensors that measure these locations using the GPS system. GPS receiving antennas are mounted on the blade or excavator stick. These coordinates are compared with the X, Y, and Z values of the digital terrain model which then guide the equipment until the surface matches the proposed design grades. This type of GPS driven grade control system can be accurate to 30 millimeters (0.1 feet), with additional accuracy down to 5 millimeters when utilizing an advanced tracking sensor.
For very precise finished grade work, there are ATS Robotic Total Stations. This utilizes a target mounted on the blade or bucket and the ATS continuously measures the location of this target. As it does so, it transmits this location data to the in-cab guidance system which then determines how to move the equipment to the desired location for dozing or digging. If the target is obscured by people, earth, or other equipment, the ATS will engage a search intelligence that will quickly reacquire its target.
Future Machine Controls
We find ourselves in an era of increasing automation that benefits operators (through ease-of-use) and the overall job site in terms of increased machine production. For example, we’ve seen a huge increase in dozing efficiency compared to the previous automation that was considered advanced only five years ago. As a result, the role of the equipment operator is evolving. The dozer operator’s focus has changed from primarily being concerned with blade control to the point where the operator allows the automation to control the blade so he can focus on the big picture of the overall work area operation.
The most innovative offerings in machine control today are all related to automation. But when reading the tea leaves of any technological advance, one should ask about the current status of the technology and when it will reach full acceptance and maturity. Not all technology gets accepted. Sometimes the better technology does not win (Betamax vs. VHS) nor is the optimum configuration chosen (the QWERTY keyboard, for example), but it becomes accepted and embedded into industry usage and public acceptance. With these caveats in mind, we asked several industry leaders the following questions concerning machine automation controls:
- First, what are the latest offerings when it comes to machine control? What are the capabilities in improving safety, efficiency, and production and operator ease of use?
- Second, how far away are we from automated equipment and artificial intelligent equipment? How does developing automation and AI take over where machine control capabilities end? Where is the line drawn between machine control and automation/AI? Basically, what is the difference and will automation/AI be the new machine control of the future?
Scott Crozier, General Manager of Trimble’s Civil Engineering and Construction Division, Trimble, provides the following insights. Today, machine control systems control the blade or cutting edge of a machine to grade. The operator still needs to know how to optimally move material around a site. We see machine learning techniques and a richer understanding of the environment (real-time terrain mapping) providing the operator with information on how to optimally complete the task. This gives less experienced operators the knowledge of those with many years moving dirt, therefore enabling them to complete a job more efficiently.
A robotic machine will be able to navigate safely around a site but will need the knowledge of an operator to efficiently complete tasks. Construction has the complexities of unknowns and variations from task to task and project to project. It will take time to build up the intelligence to command a machine to complete a task at the effectiveness of a skilled operator for all but the most basic applications.
Machine control from the perspective of controlling the blade to a digital model will always be at the core of an automated machine. Advancement will come through adding additional controls, from steering to velocity to coordinated sequences, and then safety solutions being introduced today with lockouts and object detection alerts will complete the robotic machines. It is the artificial intelligence that powers automation, turning a robotic machine into an autonomous machine. The intelligence is created and continually improved through machine learning techniques on large amounts of data and real-time machine and site sensing, and then applied to the constructible model. The outcome is an efficient and safe autonomous machine.
According to Sean Mairet, Product Marketing Manager, Grade Control at John Deere Construction Equipment, in general, machine control has significantly increased efficiency and productivity by reducing the operator’s workload. This has a compounding effect, as contractors are still able to produce quality results with less experienced operators and more experienced operators can focus on more complicated tasks. Additionally, the accuracy being achieved by grade control solutions helps contractors reduce the number of passes required to complete a job, saving fuel, time, and wear and tear on machines.
Jason Anetsberger, Senior Product Manager, Komatsu America, states that “We’re seeing another significant evolution in machine control capabilities coming to market for the benefit of all operators. There is often a misconception in the marketplace that machine control is simply machine control, and that you can lump it all together regardless of OEM or when the machine was purchased. The truth of the matter is that today’s machine control technology is not the same as the technology you would have purchased even five years ago.”
According to Fred Rio, Construction Digital and Technology Product Manager for Caterpillar Inc., artificial intelligence will be leveraged in two main areas: real-time optimization and coordination of all assets on the job site. “Today, job sites are planned and designed using high tech tools. However, when it comes to getting the work done, a lot of the coordination still relies on walkie-talkies and hand gestures. In the current construction environment, the highly sophisticated digital work has not transitioned into the physical construction work. It is awaiting adoption by the industry. AI will also be used to help the individual machines become autonomous by allowing them to interpret their surroundings and make decisions on how to best accomplish their mission most efficiently and safely.”
Full autonomy is not yet fully ready or available. However, there is a tremendous amount of step changes that can be taken now by balancing people, processes, and technology. As technology and processes change, the sophistication and adoption of automation will grow into full autonomy. Fred Rio states, “We’re committed to exploring all the options and continuing to innovate in a space where we’ve long been the industry leader. Whether it be automation at the machine level, semi-autonomous operations at the system level, or a total autonomous solution, complete connectivity and mobility paired with robust data analytics will deliver actionable insight to ensure optimum job site performance with positive enterprise outcomes. The technology is there—the next step is adoption from the construction industry.”
Matthew McLean, Product Manager for Excavators of Volvo Construction Equipment, sees the line between machine control and automation as being similar to what’s happening with cars. “Right now, cars have features to help the driver stay in their lane, not get too close to the car in front of them when using cruise control, and warn them when they are backing up too close to something. Machine control is quite similar in that we prevent operators from digging too far down or from putting the machine where they shouldn’t—like swinging into a lane of traffic.”
“In the future, when you get to level 5 automation in cars, you will tell it where to drive, and that’s your last interaction until you open the door to get out at your destination (assuming the car doesn’t open it for you). On the heavy equipment side, this would not only mean you would load a design plan and let the machine dig, but the machine should also have the ability to recognize potential hazards, such as another machine getting too close, and it would stop safely on its own. And just like cars, it will likely take a few design iterations/generations to get there.”
“To the best of my knowledge, the AI part of the equation is how the cars (or excavators) learn and improve. Cars will learn what obstacles to brake for (someone crossing the street) and what it can ignore (a pile of leaves). Construction equipment will be similar in that machines will need to learn when to stop swinging or digging. Perhaps when loading a truck, the excavator will learn how to spread the material evenly, while also maximizing the load that the truck can safely haul.”
Machine Control and The Limits of Artificial Intelligence
One near future application for machine control is in long-distance trucking, an industry whose equipment operations resemble those of construction, earthmoving, and hauling equipment. The plans for robot trucks (already tested in short drives in Florida by Starsky Robotics, a San Francisco-based developer of autonomous truck technology, which drove one of its semi-trucks 9.4 miles through live traffic on Florida’s Turnpike) intend for fully autonomous driving in freeway conditions with humans taking over the truck’s operations like drone pilots in urban environments. The reason for this is, despite its high rate of speed, freeway traffic is quite predictable and regular. Most vehicles traveling along a freeway do so at roughly the same speed, operating roughly within the same mile-per-hour window, plus or minus the posted speed limit. Lane changing occurs over long distances, traffic stoppages are easily apparent, and there is no cross traffic to worry about.
This last point is especially important because cross-traffic (not to mention pedestrians, balls, or pets running into traffic) represents a level of uncertainty that automation has difficulty with. A robot is only as good as its programming algorithms, and this software is incapable of coping with sudden surprises. So predictable freeway driving will be left to autonomous robots. Computers have been described as “high-speed idiots” that do tasks within the limits of their programming with much greater efficiency, safety, and speed than a human can ever hope to achieve. And they can do so without having to stop to eat, use the restroom, or sleep.
However, once the truck exits the freeway onto the off-ramp, a human driver will take over to guide it through relatively chaotic city traffic until it reaches a delivery destination or warehouse operations. The driver will do so from the remote comfort of an air-conditioned office using the same technology as drone pilots (more on that below). The truck “pilot” will utilize all of the radars, cameras, and sensors utilized by the onboard robotic computer to provide all of the visual and audial information he would need if he was sitting in the truck cab. Also, he will be fed traffic information, traffic status, road conditions, vector analyses of moving objects and people, and other information that will enhance the safety and efficiency of his drive.
Similar divisions of labor will be apparent on the construction site. The work zone within construction can be rather chaotic depending on the number and type of equipment. But each piece of equipment performing a particular task on any given day is operating within a relatively narrow work zone. Dozers will be spreading dirt along a particular alignment or within the current working face. Excavators will perform minimal maneuvers while digging in a particular location. Furthermore, there are also routes of routing and regular hauling of earth and materials. As such, a construction site can represent a hybrid work zone that can successfully integrate automated equipment provided that sufficient planning is performed. The trick will be to check that the software ensures nonoverlapping work areas while the equipment is being choreographed—all the while preventing the occurrence of the same type of cross-traffic that impacts urban truck driving. Additional efficiency and safety will be ensured when the various AI systems in each piece of equipment can “talk” to each other.
What Does This Do to Jobs and Productivity?
Technology destroys jobs; there is no way around that. But it also creates newer, better, and better-paying jobs. The disruption occurs going from point A of the initial job destruction to point B with the newly created jobs because these new jobs require extensive training, re-training, and instruction before a workforce can become proficient in their performance.
A perfect example is the “unmanned” Predator drone flown by the US Air Force. “For example, the replacement of human pilots by drones has eliminated some jobs but created many new opportunities in maintenance, remote control, data analysis, and cybersecurity. The US armed forces need 30 people to operate every unmanned Predator or Reaper drone flying over Syria while analyzing the resulting harvest of information occupies at least 80 people more. In 2015 the US Air Force lacked sufficient trained humans to fill all these positions, and therefore faced an ironic crisis in manning its unmanned aircraft.” (Yuval Noah Harari, “21 Lessons for the 21s Century”). So over 100 people are needed to fly an unmanned drone. A manned F-15, by contrast, has a ground crew of only six to a dozen personnel. Both support staffs have completely different training requirements and education profiles. And so, it will be as automation spreads throughout the entire economy and across the job site.
Ironically, these labor-saving devices will also require a greater commitment from equipment operators to keep their skill up to date with advancing technology. This will require regular training classes and re-education training to keep up. So the bottom line is more jobs and better jobs—but different jobs that require greater commitment.
Major Suppliers
Hemisphere GNSS, an Arizona-based innovative high-tech company that designs and manufactures heading and positioning products, services, and technology, offers GradeMetrix, a valuable machine guidance solution for high-precision GNSS-based machine control and guidance applications and systems. Hemisphere’s GradeMetrix machine control and guidance positioning systems offer unprecedented value for dozer and excavator earthmoving applications. Hemisphere continues to stride forward with its next-generation GradeMetrix solutions through its network of machine control dealers and value-added resellers. Whether it is grading, mining, excavating, or compaction applications, GradeMetrix solutions provide superior performance at an exceptional value.
The complete solution, backed by a three-year warranty, is powered by the VR500 all-in-one smart antenna (dozers) or the VR1000 GNSS receiver (excavators). The VR500 and VR1000 have undergone extensive shock, vibration (MIL-STD-810G), and environmental testing to ensure they stand up to the harshest conditions. The total solution VR500 has a set antenna separation, provides a consistent and reliable position and heading accuracy, and is installed easily with a single cable. The VR1000 supports antenna separations up to 10 meters, offering heading accuracy to 0.01 degrees RMS in addition to RTK position accuracy and full support for Hemisphere’s Atlas GNSS global correction service. The IronOne rugged display features an 8-inch high-brightness touchscreen for visibility in direct sunlight. The TS2 tilt sensors are industry proven and designed to perform in challenging environments.
Both the VR500 and the VR1000 have the latest firmware and hardware to allow them to track and use all the signals available from GPS, GLONASS, Galileo, and BeiDou constellations. This advanced technology allows the average useable number of satellites for the GradeMetrix machine control solutions to be significantly higher than older technology systems which use only GPS and GLONASS. As a result, the GradeMetrix system will operate in the most challenging conditions and offer the user the best possible performance by tracking and processing virtually every satellite in the viewable sky. Key features of this system include dynamic cut/fill to tracks, blade, or bucket; customizable views include plan, profile, cross-section; import formats include: dwg, dxf, and LandXML; 2D model free operations for simple site work tasks; collect and stake points; in-field design; global map projections; geoid support; intuitive, user-friendly software design; and machine operations in remote areas without an RTK base utilizing Atlas H10 satellite correction service.
In 2017, Trimble introduced Trimble Earthworks, their next-generation grade control platform, which gave the industry its first integrated 3D aftermarket excavator automatics capability. Using the “autos” functionality in Earthworks, the operator controls the stick and the technology controls the boom and bucket to stay on grade. The Trimble Earthworks platform now supports Dozers and Motor Graders, bringing the same increased levels of performance and usability for increased safety efficiency and productivity to job sites around the world.
Improvements continue, with new capabilities improving usability and productivity that are regularly being introduced to the Earthworks platform. Recently added functionality includes infield designs allowing operators to increase their utilization of technology by building their model on the machine. The ability to bring in surveyed or drone-created terrain maps informs operators of progress and where to move material from and to.
Trimble’s integrated weighing capabilities on excavator systems ensure trucks are optimally loaded, preventing the dangers of overloading, and connected machine control systems enable progress monitoring, keeping more people off the site and out of the dangers of moving equipment. The latest advances in Connected Construction automatically ensure the right models are active on the right machines on the right jobs and provide warnings when machines are non-compliant.
Trimble has focused on offerings that make machine control more accessible to smaller contractors, and all contractors using compact machines. In 2019, they introduced the new Trimble Earthworks GO! Grade Control Platform, which is a 2D grade control solution for compact machine grading attachments. This platform enables high-accuracy grading in an easy-to-use, portable platform. It is the only Bring Your Own Device (BYOD) compact machine control solution available today that runs on Android or iOS smartphones.
In 2018 at the Trimble Dimensions User Conference, they demonstrated the operation of an autonomous dozer and an autonomous compactor, both of which had no operator in the cab and were instead controlled by an operator standing in a tent 100 yards away. Although full automation is still years away, new offerings in machine control are moving us closer to full automation now and will continue to do so into the future.
John Deere Construction Equipment has been at the forefront of these technological changes. In the summer of 2019, John Deere launched the 850L Crawler Dozer, which is available as SmartGrade, the John Deere fully integrated, mast-less machine control solution. The 850L SmartGrade features a new operator station, making this dozer the premier integrated grade control solution on the market. The all-new operator station moves the integrated grade control monitor up from the previous position to ensure optimal viewing angles without obstructing visibility. The 850L Dozer also features software updates to the grade control system, adding provisions to incorporate blade wear without sacrificing accuracy. Another recent product launch was the 470G Excavator with integrated grade guidance. This is the largest John Deere excavator to feature the integrated grade guidance solution. Available in either a 2D or 3D configuration, grade guidance puts the power of technology into the cab, allowing an operator to maximize his productivity. A launch from earlier this year, but still notable, are their SmartGrade Motor Graders. The ability to run the grader without limiting configuration has been a game-changer for anyone who is frustrated by current system limitations. Combine that freedom with the ability to add aftermarket systems like laser receivers, sonic tracers, LPS prism, or an mm GPS receiver—the innovation and versatility embedded in these machines is truly unrivaled.
Komatsu continues to push to aggressively increase automation where they can find a win-win that increases both productivity and an operator’s desire to use the machine. Komatsu America Corporation’s proactive dozing control, introduced last year, has revolutionized machine control. With this system, the dozer learns the terrain it is tracking over and automatically uses that information to plan the next pass. The biggest challenge with previous automation was that the machine did not have the eyes of the operator and could not anticipate what was coming. That is no longer the case. Now grade breaks, elevations, and slopes are all recorded by the machine as it tracks along, and the machine automatically uses that data to make sure the next pass is optimally executed in terms of depth/trajectory. With Komatsu’s new proactive dozing logic, the productivity levels of a seasoned dozer operator are consistently delivered to anyone inside the cab. This automation is appreciated by novice and veteran dozer operators alike for all dozing aspects—first to last pass, rough dozing to fine grading, grass to finish grade. These dozers effortlessly do it all and the auto usage rates we track are jumping accordingly.
With their newly introduced intelligent Machine Control 2.0, Komatsu has additionally delivered an “easy button” for another operation that previously was not well-automated. Many dozers equipped with machine control today are frequently utilized to place materials in lifts or layers of a predetermined height for proper compaction. Historically, this operation has been done manually because it was cumbersome or too time-consuming to map out the existing surface and program the machine to be able to auto-spread material to grade in those accurate layers. Machine Control 2.0 changes that by utilizing the aforementioned capabilities that enable the dozer to learn the existing terrain as it maps the ground with each pass, thereby allowing it to spread in layers of accurate thickness. By “spreading” the use of automatics to this key dozer operation, consistent compaction is ensured, and re-work is minimized, improving overall productivity.
For many years, the world’s mines have looked to Caterpillar’s autonomous products to help lead the way for improved operations. Caterpillar equipment provides numerous other advantages, including better efficiency, greater productivity, improved safety, and lower costs. Building on existing Cat technologies (Link, Detect, Grade, Payload, Compact, and Command), Caterpillar is developing solutions that will optimize and coordinate job site activities. Their machine control system, Cat Command, is making its way into the construction industry improving safety and productivity. Automated features built into today’s machines ensure even the newest operators can work accurately and efficiently. Cat dozers and excavators often work in challenging environments (under high walls, on steep slopes, around unstable surfaces, or near other hazards) that create difficult operating conditions that can hinder a customer’s operation or put their crew’s safety at risk. Cat Command’s remote-control solutions take operators far away from the hazards of the job. Cat Command for dozing is structured as a series of building blocks leading to semi-autonomous dozing. The solution is scalable and incremental, allowing the customer to enter at whatever level of technology and infrastructure they choose.
Volvo Construction Equipment is carrying the legacy of Dig Assist by incorporating machine control with Volvo Active Control, their new automated machine control system for their EC220E, EC250E, and EC300E excavators. Volvo Active Control automates boom and bucket movements, making the digging process more efficient and more accurate, with less effort. On the performance side, it ties into Dig Assist—Volvo’s machine control system for excavators—allowing an operator to set a target grade and preventing over-digging. Additionally, the operator can use the Active Control functions to set swing limits for truck loading. When digging, the operator makes a couple of quick choices. Does he want to control the bucket angle, or have the machine hold an angle? Does he want a finely graded surface or just a rough grade at a faster speed? Then the operator sets a target depth, pushes the ON button, and goes to work. On the safety side, an operator can set boundaries for the machine not to cross (when working inside a building to avoid hitting the ceiling or walls, establishing a height limit or a swing fence, avoid hitting an underground utility, set a depth limit to prevent the machine from digging too far down).
Some of the latest advancements in machine control for Topcon include the new automated edition of its 3D modular aftermarket solution for excavation—the X-53x Automatic. The X-53x Automatic Excavator is designed to increase seasoned operators’ productivity and bring newer operators up to speed faster. Features such as grade matching, increment/decrement of vertical offset, topo survey, bucket mode, and more are all accessed via the Topcon JS-1 Joysticks. This ergonomic approach helps operators effortlessly execute accurate, sophisticated designs that would be difficult even for highly experienced operators. This solution allows operators to work from design files produced offsite or create and execute their designs in the field using the bucket to take topo shots. Plus, the system reduces the need to have site personnel in or next to trenches and holes where accidents are more likely to occur. Automatic Excavator functionality can be added to machines already equipped with Topcon’s 2D or 3D systems to the new fully automatic excavation package. Additionally, Topcon has introduced a revolutionary solution for machine control with our 3D-MCMAX bulldozer system. This three IMU sensor system (Inertial Measurement Unit) uses state-of-the-art sensors to measure the movement of the cutting blade. This eliminates the need for a mast with a GPS antenna on the cutting blade to establish a position. The new system both improves safety by eliminating the mast on the blade and greatly increases grading performance and accuracy at high operating speeds and when operating on tight turns and steep slopes.
