Using insights from the human visual system, this linked document describes thoughts on making use of built-in inertial sensors and the rich texture prevalent in natural scenes to drive the construction of a multi-scale model for from live drone video.

The basic approach is to construct a 3D model of the environment in which a set of one or more cameras is moving and to use that model to estimate motion of the cameras and other sensors with respect to the model. The color and/or albedo of locations on the model can also be estimated as a function of viewing direction; this intrinsic property determination can be done using a separate approach from the geometry determination. For speed and robustness, both the geometry and the color of the model can be estimated at multiple scales. The model estimates can be improved over time based on multiple camera measurements, improved estimates of motion, and other data.

Including measurements and modeling into a unified framework enables more efficient use of computational resources, which can decrease energy use and/or decrease the time taken to produce models. This forms a “cycle of goodness” – where reduced solution time means reduced time between measurements, which means reduced change between measurements, which means reduced computational resources, which repeats the cycle.

Including error estimation and statistical motion distributions provides more‐robust models, which enable more reliable planning based on the resulting models. It also enables additional focusing of computational resources on the most‐likely solutions and/or reducing the largest errors, which can both improve the model and make more efficient use of computational resources.

Providing model‐based estimates also increases the effectiveness of data transmission from a drone during deployment, enabling algorithms, a pilot, and/or analysists to determine where the drone should acquire new data before returning to base. Providing a set of drones that cooperate to build a model could enable a set of analysts to survey an area more rapidly, enabling faster planning of responses in disasters and more effective and rapid activities in other situations. Model‐based estimation also makes more efficient use of deployment time even in the absence of communications back to base.