Long-range UAV Thermal Geo-localization with Satellite Imagery

Jiuhong Xiao1, Daniel Tortei2, Eloy Roura2, Giuseppe Loianno1,
1New York University, 2Technology Innovation Institute
arXiv Video Code

Abstract

Onboard sensors, such as cameras and thermal sensors, have emerged as effective alternatives to Global Positioning System (GPS) for geo-localization in Unmanned Aerial Vehicle (UAV) navigation. Since GPS can suffer from signal loss and spoofing problems, researchers have explored camera-based techniques such as Visual Geo-localization (VG) using satellite RGB imagery. Additionally, thermal geo-localization (TG) has become crucial for long-range UAV flights in low-illumination environments. This paper proposes a novel thermal geo-localization framework using satellite RGB imagery, which includes multiple domain adaptation methods to address the limited availability of paired thermal and satellite images. The experimental results demonstrate the effectiveness of the proposed approach in achieving reliable thermal geo-localization performance, even in thermal images with indistinct self-similar features. We evaluate our approach on real data collected onboard a UAV. We also release the code and Boson-nighttime, a dataset of paired satellite-thermal and unpaired satellite images for thermal geo-localization with satellite imagery. To the best of our knowledge, this work is the first to propose a thermal geo-localization method using satellite RGB imagery in long-range flights.

Boson-nighttime

We used FLIR's Boson thermal imager (8.7 mm focal length, 640p resolution, and 50-degree horizontal field of view). The collected images are nadir at approx. 1m/px spatial resolution. We performed six flights from 9:00 PM to 4:00 AM and label this dataset as Boson-nighttime, accordingly. To create a single map, we first run a structure-from-motion (SfM) algorithm to reconstruct the thermal map from multiple views. Subsequently, orthorectification is performed by aligning the photometric satellite maps with thermal maps at the same spatial resolution. The ground area covered by Boson-nighttime measures 33km^2 in total. The most prevalent map feature is the desert, with small portions of farms, roads, and buildings. The low-contrast self-similar thermal features in the desert regions have made it difficult for geo-localization at a finer scale. The Bing satellite map (https://www.bing.com/maps/aerial) is cropped in the corresponding area as our satellite reference map. We tile the thermal map into 512x512 px thermal image crops with a stride of 35 px. Each thermal image crop pairs with the corresponding satellite image crop. Areas covered by three flights of Boson-nighttime are used for training and validation. The remaining areas, covered by the other three flights are used for testing. The train/validation/test splits for Boson-nighttime are 10256/13011/26568 pairs of satellite and thermal image crops, respectively.

SGM Results - Green is Correct, Red is Wrong

Input

Baseline

Ours

TGM Generated Thermal Images

Satellite

Enhanced Generated

Enhanced Ground Truth

BibTeX


      @INPROCEEDINGS{10342068,
        author={Xiao, Jiuhong and Tortei, Daniel and Roura, Eloy and Loianno, Giuseppe},
        booktitle={2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, 
        title={Long-Range UAV Thermal Geo-Localization with Satellite Imagery}, 
        year={2023},
        volume={},
        number={},
        pages={5820-5827},
        doi={10.1109/IROS55552.2023.10342068}}