Preparing Software Engineers to Develop Robot Systems

Carl Hildebrandt, Meriel von Stein, Trey Woodlief, Sebastian Elbaum

Robotics is a rapidly expanding field that needs software engineers. Most of our undergraduates, however, are not equipped to manage the unique challenges associated with the development of software for modern robots. In this work we introduce a course we have designed and delivered to better prepare students to develop software for robot systems. The course is unique in that: it emphasizes the distinctive challenges of software development for robots paired with the software engineering techniques that may help manage those challenges, it provides many opportunities for experiential learning across the robotics and software engineering interface, and it lowers the barriers for learning how to build such systems. In this work we describe the principles and innovations of the course, its content and delivery, and finish with the lessons we have learned.

Published in 2022 IEEE/ACM International Conference on Software Engineering (ICSE), 2021

World-in-the-Loop Simulation for Autonomous Systems Validation

Carl Hildebrandt, Sebastian Elbaum

Simulation is at the core of validating autonomous systems (AS), enabling the detection of faults at a lower cost and earlier in the development life cycle. However, simulation can only produce an approximation of the real world, leading to a gap between simulation and reality where undesirable system behaviors can go unnoticed. To address that gap, we present a novel approach, world-in-the-loop (WIL) simulation, which integrates sensing data from simulation and the real world to provide the AS with a mixed-reality. The approach executes multiple instances of the AS in parallel, one in the real world and at least one in simulation, performs configurable transformations, filtering, and merging operations on the body of sensed data in order to integrate it, and provides the pipelines to distribute the original sensor data and the integrated sensor data back to the executing AS. We present a study on multiple scenarios and two simulators that demonstrates how WIL reduces the simulation-reality gap and increases the chances of exposing failures before deployment.

Published in 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021

Feasible and Stressful Trajectory Generation for Mobile Robots

Carl Hildebrandt, Sebastian Elbaum, Matthew B. Dwyer, Nicola Bezzo

While executing nominal tests on mobile robots is required for their validation, such tests may overlook faults that arise under trajectories that accentuate certain aspects of the robots behavior. Uncovering such stressful trajectories is challenging as the input space for these systems, as they move, is extremely large, and the relation between a planned trajectory and its potential to induce stress can be subtle. To address this challenge we propose a framework that 1) integrates kinematic and dynamic physical models of the robot into the automated trajectory generation in order to generate valid trajectories, and 2) incorporates a parameterizable scoring model to efficiently generate physically valid yet stressful trajectories for a broad range of mobile robots. We evaluate our approach on four variants of a state-of-the-art quadrotor in a racing simulator. We find that, for non-trivial length trajectories, the incorporation of the kinematic and dynamic model is crucial to generate any valid trajectory, and that the approach with the best hand-crafted scoring model and with a trained scoring model can cause on average a 55.9% and 41.3% more stress than a random selection among valid trajectories. A follow-up study shows that the approach was able to induce similar stress on a deployed commercial quadrotor, with trajectories that deviated up to 6m from the intended ones.

Published in ISSTA 20: 29th ACM SIGSOFT International Symposium on Software Testing and Analysis Proceedings (ISSTA), 2020 - Distinguished Artifact Award

Blending Kinematic and Software Models for Tighter Reachability Analysis

Carl Hildebrandt, Sebastian Elbaum, Nicola Bezzo

Reachable sets are critical for path planning and navigation of mobile autonomous systems. Traditionally, these sets are computed using system models instantiated with their physical bounds. This exclusive focus on the physical bounds belies the fact that these systems are increasingly driven by sophisticated software components that can also bound the variables in the system models. This work explores the degree to which bounds manifested in the software can affect the computation of reachable sets, introduces an analysis approach to discover such bounds in code, and illustrates the potential of that approach on two systems. The preliminary results reveal that taking into consideration software bounds can reduce traditionally computed reachable sets by up to 91%.

Published in 2020 IEEE/ACM 42nd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), 2020

Fire-Aware Planning of Aerial Trajectories and Ignitions

Evan Beachly, Carrick Detweiler, Sebastian Elbaum, Brittany Duncan, Carl Hildebrandt, Dirac Twidwell, Craig Allen

Prescribed fires can lessen wildfire severity and control invasive species, but they can also be risky and costly. Unmanned aerial systems can reduce those drawbacks by, for example, dropping ignition spheres to ignite the most hazardous areas. Existing systems, however, lack awareness of the fire vectors to operate autonomously, safely, and efficiently. In this work we address that limitation, introducing an approach that integrates a lightweight fire simulator and a planner for trajectories and ignition sphere drop waypoints. Both components are unique in that they are amenable to input from the system’s sensors and the fire crew to increase fire awareness. We conducted a preliminary study that confirms that such inputs improve the accuracy of the fire simulation to counter the unpredictability of the target environment. The field study of the system showed that the fire-aware planner generated safe trajectories with effective ignitions leveraging the fire simulator predictions.

Published in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018 - IROS Best Paper Award on Safety, Security, and Recue Robotics