Visualizing a million lightning strikes with deck.gl and React
What’s this blog post about?
The beginning of summer 2024 was particularly stormy in Northern Europe. For example, Finland saw over three times the average amount of lightning strikes in May, and just the first days of June had more lightning observations than the entire month’s average total (read more in Finnish).
In this blog post I build a small web map application for interactively visualizing and animating the thunderous early summer in Northern Europe. The text is an overview of what implementing such a visualization might look like using the deck.gl visualization framework. You can use the map here (do note that some older or less performant devices might not support the map application).
If data visualization, web mapping or web development in general interests you, keep reading! If not, feel free to just play around with the map!
What’s deck.gl?
Deck.gl is an open source framework for building data visualizations for the web. Its main focus is combining high interactivity with the performance needed to enable responsive visual exploration of large datasets. While not strictly limited to map-based presentations, visualizing geospatial data is perhaps its most prominent use case.
Building an interactive web map
In the following I’ll go over the main parts of the map application. While there are some simplified code snippets below, you can refer to the source code of the map for the details and a working application. For example, all styling is intentionally left out here to keep this post shorter.
Accompanying technologies
In addition to deck.gl, I use a few other tools and libraries. Here are the most important ones:
- React: React is a common choice for implementing general-purpose UI components and managing application state. Integration with React is also a design priority of deck.gl, which makes this a robust choice when your map application gets more complex.
- Vite: Vite provides an effortless project setup and, as the name suggests, speedy builds.
- Material UI: Material UI has ready-to-use templates and styling for React components. This saves a potentially massive share of development time and boilerplate code.
- Maplibre / react-map-gl: I use Maplibre with react-map-gl to provide a basemap for the visualization.
The data
The lightning data shown on the map is the Finnish Meteorological Institute’s open data. To create the dataset used in this particular visualization, I downloaded every lightning observation (location, time, peak electrical current) from late May to early June. The resulting dataset has 1 245 392 lightning observations.
To provide a way for the map application to access the data, I simply used a public GitHub repository to host a CSV file containing the dataset (Do note that this is a bad idea for any larger files).
Project setup
Vite’s React + Typescript template is a straightforward option to set up the initial project scaffolding. Vite has excellent documentation on the topic, but in this case everything boils down to:
npm create vite@latestAfter answering some prompts you have a functional React app with Typescript and some useful scripts all configured.
If using Maplibre, one potentially needed trick is to alias Mapbox to Maplibre, as some tools might mysteriously expect Mapbox to be present. You can do this in Vite’s configuration file:
import { defineConfig } from "vite";
...
export default defineConfig({
...
resolve: {
alias: {
"mapbox-gl": "maplibre-gl",
},
},
});A skeleton of the map application
Loading data
To access the dataset I simply use the fetch API to get the data, and then the loaders.gl library to efficiently parse the resulting (quite large) CSV into an array. To make things more explicit, I also define a type for the data.
import { parse } from "@loaders.gl/core";
import { CSVLoader } from "@loaders.gl/csv";
import type LightningObservation from "./types";
DATA_URL = <url to access your data>
const loadData = async () => {
const data = await parse(fetch(DATA_URL), CSVLoader);
return data.data as LightningObservation[];
};
...type LightningObservation = {
latitude: number;
longitude: number;
peak_current: number;
time: number;
};
...App component
As I am using React, I structure the code into components. In the top level component (App) I load the data, and pass it down to the MapComponent. Note that the data is managed with React’s useState and useEffect hooks.
import { useState, useEffect } from "react";
import MapComponent from "./components/Map";
import loader from "./loader";
import type LightningObservation from "./types";
const App = () => {
const [data, setData] = useState<LightningObservation[] | null>(null);
useEffect(() => {
loader.loadData().then((data) => setData(data));
}, []);
return (
<div>
<MapComponent data={data} />}
</div>
);
};
...Map component
The Map component (named MapComponent to avoid confusion with react-map-gl’s Map) defines the layer responsible for presenting the lightnings, and returns a DeckGL component that presents the layer. Deck.gl has quite a few options when it comes to layers for visualizing all kinds of datasets, but in this case I went for a simple ScatterPlotLayer to present each observation as a dot on the map.
Of course, each layer type has lots of options to further spice up the visualization. Here I set the radius of each dot based on the peak current of said lightning strike. Note that this, like many things in deck.gl, is done by simply passing a function. This allows for great flexibility: in theory, there are no limits to how simple or complex the functions controlling various aspects of a layer can be. Static values can of course be used just as well (like the getFillColor below).
import { DeckGL, ScatterplotLayer } from "deck.gl";
import Map from "react-map-gl/maplibre";
import { BASEMAP } from "@deck.gl/carto";
import type { MapViewState } from "deck.gl";
import type LightningObservation from "../types";
...
const MapComponent = ({ data }: { data: LightningObservation[] | null }) => {
...
const layers = [
new ScatterplotLayer({
id: "ScatterplotLayer",
data: data,
getPosition: (d: LightningObservation) => [d.longitude, d.latitude],
getRadius: (d: LightningObservation) => d.peak_current,
getFillColor: [225, 210, 255, 180],
radiusScale: 10,
}),
];
const INITIAL_VIEW_STATE: MapViewState = {
longitude: 20,
latitude: 60,
zoom: 5,
};
return (
<>
<DeckGL
initialViewState={INITIAL_VIEW_STATE}
controller={true}
layers={layers}
>
<Map mapStyle={BASEMAP.DARK_MATTER}> </Map>
</DeckGL>
...
</>
);
};
...Adding interactivity
The above components make for a map with all the lightnings rendered. However, zooming and panning are the only ways to interact with the map. Let’s add the capabilities for filtering and animating the lightning strikes and for customizing the visualization.
Filtering data
Deck.gl’s DataFilterExtension provides highly efficient filtering capabilities to layers. To filter the lightnings based on observation time I added the extension to the layer, and defined an array (filterRange) to hold the min / max values to filter by. To prepare for everything being interactive, the array is a state variable that starts out as null, waiting to be defined by user input. The purpose of the getTimeRange function is to make sure that the filter always has a min / max value to go by, defaulting to the bounds of the observation times.
const getTimeRange = (data): ... => {
...
};
const MapComponent ... => {
const [_filterRange, setFilterRange] = useState<
[start: number, end: number] | null
>(null);
const timeRange = useMemo(() => getTimeRange(data), [data]);
const filterRange = _filterRange || timeRange;
...
const dataFilter = new DataFilterExtension({
filterSize: 1,
});
const layers = [
filterRange &&
new ScatterplotLayer({
...
getFilterValue: (d: LightningObservation) => d.time,
filterRange: [filterRange[0], filterRange[1]],
extensions: [dataFilter],
}),
];
...
return (
...
<DeckGL>
...
</DeckGL>
...
{timeRange && filterRange && (
<FilterSlider
min={timeRange[0]}
max={timeRange[1]}
filterRange={filterRange}
setFilterRange={setFilterRange}
animationSpeed={animationSpeed}
/>
)}
...
)
};
...Now we just need a UI component to change the filterRange array. The one below is a slider made using material UI.
import { Slider, Box, ... } from "@mui/material";
import formatTimeStamp from "../format-timestamp";
const FilterSlider = ({
min,
max,
filterRange,
setFilterRange,
}: { ... }) => {
...
const handleSliderChange = (_: Event, newRange: number | number[]) => {
setFilterRange(newRange as number[]);
};
return (
<Box ... >
...
<Slider
min={min}
max={max}
value={filterRange}
onChange={handleSliderChange}
valueLabelDisplay="auto"
valueLabelFormat={formatTimeStamp}
/>
</Box>
);
};
...Animating the map
Now that we have the slider, animating the map is quite straightforward. The trick is to animate the slider: the map will then react to the filterRange changing and update too. The animation itself runs inside a useEffect hook.
Note the use of request / cancelAnimationFrame to produce the animation (instead of using timeouts or intervals, for example). Among other things, this makes sure the animation matches with the display refresh rate. You can read more on the topic for example here.
import { useState, useEffect } from "react";
import { Slider, Box, Button } from "@mui/material";
import { PlayArrow, Pause } from "@mui/icons-material";
...
const FilterSlider = ({ ... animationSpeed }: { ... }) => {
const [isPlaying, setIsPlaying] = useState<boolean>(false);
const isPlayEnabled = filterRange[0] > min || filterRange[1] < max;
useEffect((): any => {
let animation: number;
if (isPlaying) {
animation = requestAnimationFrame(() => {
const span = filterRange[1] - filterRange[0];
let nextValueMin = filterRange[0] + animationSpeed;
let nextValueMax = nextValueMin + span;
if (nextValueMax >= max) {
nextValueMin = min;
nextValueMax = nextValueMin + span;
}
setFilterRange([nextValueMin, nextValueMax]);
});
}
return () => animation && cancelAnimationFrame(animation);
});
...
return (
<Box ... >
<Button
color="primary"
disabled={!isPlayEnabled}
onClick={() => setIsPlaying(!isPlaying)}
title={isPlaying ? "Stop" : "Animate"}
>
{isPlaying ? (
<Pause fontSize="large" />
) : (
<PlayArrow fontSize="large" />
)}
</Button>
<Slider
...
/>
</Box>
);
};
...One thing to note when it comes to these types of controlled animations is that different animations might not play nicely with each other: In this case, if material UI tries to apply its own transitions on every change of the slider, while the slider is already changing constantly, the animation will look wonky at best and completely stop working at worst. You can fix this particular issue by defining your own theme for material UI, for example:
import { createTheme } from "@mui/material/styles";
const myTheme = createTheme({
transitions: {
create: () => "none",
},
});
...Even more interactivity
To recap: You can use your own functions in controlling the visualization (and of course the entire application), and you can expose different parts of it to be interactively controlled (for example by defining them as state variables if using React). As a result, the control you have over creating not just a visualization, but also the surrounding interface is quite limitless.
In the example map the user can control the animation speed, and the size multiplier of the dots on the map. This is, again, achieved with the same pattern as above: define state, use it in controlling the visualization, and add UI components for controlling the state. The relevant state variables in the MapComponent are animationSpeed and radiusScale, and the UI for interacting with them could be something like this:
...
import { Slider, ... } from "@mui/material";
...
const InfoPanel = ({
...
animationSpeed,
setAnimationSpeed,
radiusScale,
setRadiusScale,
}: { ... }) => {
...
return (
...
<Slider
aria-label="Animation Speed"
value={animationSpeed}
min={10}
max={3000}
onChange={(_, value: number | number[]) =>
setAnimationSpeed(value as number)
}
/>
<Slider
aria-label="Point Size"
value={radiusScale}
min={1}
max={100}
onChange={(_, value: number | number[]) =>
setRadiusScale(value as number)
}
/>
...
);
};
...Those were the main functionalities of the map explained! Again, you can find the application code here in case you want to dig deeper or even experiment yourself.
Conclusion
While the question of what data is “big” and what is not is a topic entirely of its own, there certainly is value in interactive exploration when it comes to reasoning with most information. Deck.gl provides one option to enable such exploration with large geospatial data, all from the comfort of your web browser. This is certainly valuable as both the datasets and the requirements placed on visualizing them keep growing, and more often than not the expectation for software is to be accessed on the web.
Helpful resources
- deck.gl has extensive and well written documentation, and an example gallery with all the examples you could wish for
- React docs are always helpful
- Vite is also very well documented. For example, how the live version of this map is deployed is directly from their examples.
This article was initially published at www.gispo.fi by Eemil Haapanen (Gispo Finland Ltd.).
