With feature-based Vision Navigation, the system doesn’t try to learn to “see” the imagery directly, rather it does a further abstraction of the imagery into discrete identified features, then tries to infer the position and pose based on the arrangement of those features in 2D space. In many cases, feature-based Vision Navigation can be superior to holistic-based architectures because it disconnects the process of feature identification from localization. In a Holistic system, if the appearance of features changes, retraining the entire vision navigation stack may be required. In a feature-based system, only the underlying feature recognizer might need updating. By way of analogy, in a holistic system that learns from actual input pixels, a red brick Post Office building landmark is very different from a tan siding Post office building. So different, that unless the vision navigation system had picked up sufficient other identifying marks (like the Post Office sign, door and window arrangement, etc) it might no longer recognize the post office as a landmark. In which case, the whole system would need retraining. Breaking apart the feature recognition and localization components allows one to be updated independently of the other. Additionally, a system capable of both capabilities is often simply larger and more performance intensive than a multi-component system.

A feature-based vision navigation system can also be composed of a neural-network-based feature identification system and a more conventional algebraic localization based on Perspective-n-Point. While many object identification systems like YOLO produce a bounding box surrounding an identified feature, other systems like BRIEF, SIFT, SURF and ORB) can be taught to focus on producing an XY point coordinate representing a consistent location within the identified feature, and potentially a scale/distance metric. This allows for computing angles and distances to known landmarks using trilateration, triangulation and/or PnP. In this case, no retraining of the localization network is needed at all if features change in position, appearance or number. Just update the feature almanac and then retrain the lower-level networks that recognize features. Going one step further, an implementation could even segregate the feature identifiers into a collection of independent recognizers, such as “Recognize Smallburg Post Office” and “Recognize Anytown Grocery Store”. Each of these could be a much smaller and more efficient neural network. An optimizing visual navigation system could then use a rough navigation solution (based on known prior location, degraded GPS/IMU solutions, etc) to load and execute only the specific landmark-recognizers that are relevant to the region at hand. This also allows for minimal and efficient regeneration of recognizers if the “Bigville Mall” is replaced with condos or the “Hill Valley Courthouse” is struck by lighting and burns. In practice, while YOLO is popular, well-known and impressive, it isn’t actually that well suited for navigation feature recognition because it is designed to generalize recognition of classes of features – “a dog”, “a post office”, “a lawyer billboard” rather than identify a particular unique specimen. Semantic segmentation networks that break a scene up into categories and extract features from the patterns are better for navigation.

In general, any neural network capable of recognizing a larger set of locations will be more complex than a smaller one. The ratio is not linear either, so deconstructing into specific smaller recognizers (for example by region) is an advantage.