Recent developments in the use of AI foundation models in open source GIS
Machine learning (ML) is a subfield of Artificial Intelligence (AI) in which computers are utilized to learn from data without being explicitly told what to do. In practice, ML methods are used for solving problems that might be impractical for humans due to time or resource constraints.
ML approaches are divided into three main categories: supervised, unsupervised and reinforcement learning. In supervised learning, models are trained on labeled datasets, learning to map inputs to corresponding outputs. This enables accurate predictions on new, unseen, but similar data. Supervised learning is used for instance in email filtering (junk mail). In contrast, unsupervised learning involves models learning on their own, finding patterns, relationships or structures in the data. Typical examples of unsupervised learning techniques are some clustering algorithms, such as K-Means clustering. Finally, reinforcement learning is a type of ML where models receive feedback on interactions with their environment aiming to learn a policy guiding actions towards specific objectives. Reinforcement learning can be utilized in e.g. recommendation systems.
OpenAI’s chatbot, ChatGPT, is a good example of a widely popular AI application. It is based on foundation models GPT-3.5 and GPT-4, fine-tuned using both supervised and reinforcement learning. Foundation models are large scale ML models pre-trained on a large quantity of data. “Pre-trained” is pivotal here; finding a suitable dataset or creating one from scratch can be very time and resource consuming, let alone training the model. Since the heavy lifting is already done, users can focus on fine-tuning the model for more specific downstream tasks.
In the realm of open source GIS, NASA and IBM have released in August 2023 a geospatial AI foundation model for Earth observations called Prithvi. It is trained with NASA’s Harmonised Landsat Sentinel 2 satellite data and unlike most remote sensing models, it can handle time series of images. The model can be fine-tuned for tasks such as burn scars segmentation or land cover classification. There are a few demos to try here. Just to show an example, here’s how Prithvi can be used for flood segmentation:
The second picture is the model prediction on the input raster (the first) where white area is water while black is land. Images downloaded from https://huggingface.co/spaces/ibm-nasa-geospatial/Prithvi-100M-sen1floods11-demo.
Another example of open source foundation model is in April 2023 released Meta’s Segment Anything Model (SAM) which can be used for image segmentation. SAM can be utilized for segmenting geospatial data, for instance, with a Python package segment-geospatial. There is also a QGIS plugin Geometric Attributes that uses the said package. The plugin can be used to calculate the centerline, width, deviation, shape and adjacency of polygons, and to segment geospatial data. However, the segmentation process is quite slow in the plugin (at least it was for me), so I’d recommend using segment-geospatial directly and then QGIS for viewing the result. You can check the plugin and some examples here.
With the recent developments in AI foundation models, progress has been made in harnessing their power to address geospatial challenges. AI foundation models are becoming useful tools, simplifying the application of machine learning techniques to a wide range of tasks. These advancements hold promise for enhancing geospatial analysis in the future.
This article was initially published at www.gispo.fi by Mika Sorvoja (Gispo Ltd.).
