Immersive Semi-Autonomous Aerial Command System (ISAACS)
A new way to team up with drones.
Overview
ISAACS is an open-source project to envision new ways for human users to intuitively interface and collaborate with aerial drones around augmented reality (AR) technologies.
Updates
RFS Flight Test
We have begun flight tests at B164 and B100 in Richmond Field Station (RFS). These flight tests prove the capability of our software beyond what can be achieved in simulations. We have conducted tests on the DJI Matrice M200 and M600 series with various payloads, single drone, and multi drone configurations.
Server Architecture Development
Starting Fall 2020, our software is being migrated onto a central server that will enable our solution to effectively scale. A central server will make it easier to connect new drones and VR clients to our platform and enable the world of true multi-drone flight control.
Open Source Support
We are also in the process of making our work available for the open source community to use. With the new Server Architecture, we can easily support drones from different manufacturers.
Most recently we began development on new Ardupilot and MavROS drones. We expect the hardware to be arriving in Spring 2021.
Process and Features
Localization and Visualization
ISAACS utilizes real-time SLAM solutions to localize the 3D coordinates of aerial vehicles and the operators wearing HoloLens. The first version of the platform will be developed in collaboration also with DJI based on the Matrice-100 platform.
Human-Robot Interface (HRI) via AR
A new 3D, contextualized robot interface allows operators to control and command a drone fleet directly from HoloLens. It represents a new vision of how human-robot interface (HRI) can be, which can provide seamless and intuitive extension of human operators’ perception and control capabilities to a fleet of aerial vehicles in complex environments.
Vehicle Safety Assurance
ISAACS also interfaces with drones' low-level controller to enhance the safety of the drone operation even in the case of human error. Carefully designed vehicle safety information will be also augmented in 3D space via HoloLens to enhance the operators’ situation awareness.
News and Press
ISAACS was awarded the First Microsoft HoloLens Research Grant
Today, we’re excited to announce the award recipients of the Microsoft HoloLens Academic Research Grant Program. Earlier this year, we challenged academic institutions to submit their ideas to harness the potential of Microsoft HoloLens and push the boundaries of possible applications for holographic computing.
Who is Isaacs?
ISAACS is inspired by the famous game theorist Rufus Isaacs, who contributed to the fundamental development of the classic pursuit-evasion game theory and dynamic programming, among many other notable work.
For more information, please contact:
Dr. Allen Y. Yang <yang@eecs.berkeley.edu>