Publications

ICRA 2023 - Boosting Performance of a Baseline Visual Place Recognition Technique by Predicting the Maximally Complementary Technique

One recent promising approach to the Visual Place Recognition (VPR) problem has been to fuse the place recognition estimates of multiple complementary VPR techniques using methods such as SRAL and multi-process fusion. These approaches come with a substantial practical limitation: they require all potential VPR methods to be brute-force run before they are selectively fused. The obvious solution to this limitation is to predict the viable subset of methods ahead of time, but this is challenging because it requires a predictive signal within the imagery itself that is indicative of high performance methods. Here we propose an alternative approach that instead starts with a known single base VPR technique, and learns to predict the most complementary additional VPR technique to fuse with it, that results in the largest improvement in performance. The key innovation here is to use a dimensionally reduced difference vector between the query image and the top-retrieved reference image using this baseline technique as the predictive signal of the most complementary additional technique, both during training and inference. We demonstrate that our approach can train a single network to select performant, complementary technique pairs across datasets which span multiple modes of transportation (train, car, walking) as well as to generalise to unseen datasets, outperforming multiple baseline strategies for manually selecting the best technique pairs based on the same training data.

Performance benefits of dynamically selecting complementary VPR techniques.

RA-L 2022 - Improving Road Segmentation in Challenging Domains Using Similar Place Priors

Road segmentation in challenging domains, such as night, snow or rain, is a difficult task. Most current approaches boost performance using fine-tuning, domain adaptation, style transfer, or by referencing previously acquired imagery. These approaches share one or more of three significant limitations: a reliance on large amounts of annotated training data that can be costly to obtain, both anticipation of and training data from the type of environmental conditions expected at inference time, and/or imagery captured from a previous visit to the location. In this research, we remove these restrictions by improving road segmentation based on similar places. We use Visual Place Recognition (VPR) to find similar but geographically distinct places, and fuse segmentations for query images and these similar place priors using a Bayesian approach and novel segmentation quality metric. Ablation studies show the need to re-evaluate notions of VPR utility for this task. We demonstrate the system achieving state-of-the-art road segmentation performance across multiple challenging condition scenarios including night time and snow, without requiring any prior training or previous access to the same geographical locations. Furthermore, we show that this method is network agnostic, improves multiple baseline techniques and is competitive against methods specialised for road prediction.

Semantically similar but geographically separate places from WildDash.

ACRA 2021 - Improving Semantic Segmentation with Calibrated Whole Image and Patch-Based Similar Place Priors

Semantic segmentation is a crucial but challenging capability for many autonomous systems, especially in challenging environmental conditions like rain, snow or night time. Current solutions achieve high performance in these scenarios by learning domain specific information through domain adaptation or style transfer, but don’t generalize and require both anticipation of and access to representative data. Here we present a domain-agnostic approach that uses Visual Place Recognition (VPR) techniques to find image patches from similar (but geographically distinct) places to the query location, but obtained under ideal conditions, and fuses the open loop segmentation results with those prior place segmentations. We present our new patch-based place similarity methods along with comparison to whole image-based methods as a baseline, and a new entropy-based calibration method for weighting which methods to use. Results are presented for several benchmark datasets demonstrating the effectiveness of using similar place priors, and analysing the performance of whole- and patch-based approaches.

Example of patch-based similar place generation using the Dark Zurich Dataset.