I got to play with a hyperspectral camera! So much fun. Normal cameras capture three buckets: RGB. Hyperspectral cameras capture a bajillion buckets. And they do it across a way wider spectrum -- so you can see into infrared and ultraviolet.But cooler than the increased range of a hyperspectral camera, is the drastically increased granularity: even within the normal visible range, we chop up each of those RGB buckets into many much smaller buckets. Each one, only a few nanometers wide.
generated via Wikimedia → Nano banana. decently good actually
This granularity means that, instead of a 2d-plane (where you collapse RGB-values along the spectral dimension into a single pixel), hyperspectral cameras yield 3d-cubes as images: each pixel now hosts its intensity over the entire spectrum.That's what's visualized above: a hyperspectral image.
Using this spectral dimension, we can extract spectral signatures from what we image. Different materials reflect and absorb light differently across the spectrum, so by jumping from a scalar-per-pixel to a vector-per-pixel, we gain all sorts of wacky new discriminatory power. (I think of it kinda like bad mass-spec for surfaces.)
Here's an example.
#notscuffed
All of these leaves (gaffers-taped to a spare 2x4) are real, except for one. Instead of plant matter, it's plastic. (Something something, camouflage detection). I can't tell which. Can you?Now, let's image these suckers with a hyperspectral camera.(A quick tangent: these hyperspectral cameras are generally line-scanning, meaning they only image in 1d. You can scan this line over a region to image and stitch them together in post. Normally, this would be drone-mounted, which you can fly across a region. But we didn't wanna stick it on a drone, so we just stuck whatever we wanted to image to that 2x4 on the chairs, and slid it during recordings.)
The data!
Scrub through the spectral axis with the slider below
Spectral row 0 / 199
726
Spatial pixels
0
0324
Lines
Wavelength slice slider0 / 199
At a spectral range higher than what we can see, there exists a region where the plastic leaf is entirely visible, while all the real leaves are entirely gone! We have de-camouflaged the fake leaf, by utilizing its unique spectral signature.
Hello-world for hyperspectral imaging, complete :)
A huge, huge thanks to the great folks over at Lore+ Technology Wilco Imaging + Syzygy optics / SensIR.
Specifically, Mike Sullivan, Evan Ihlenfeldt, Christopher Clemens, and Darlene Theriault, for letting me borrow their awesome camera and lenses (and nerding out about cool optics with me :). All the tech worked beautifully, the API was great, and everyone was so incredibly helpful. These folks are stellar.
And a big thanks to Lucas! For spending a rainy afternoon hacking and collecting leaves with me. It's been so great having you in the bay, after taking the leap to leave college together. I can't wait to learn more about percolation and self-avoiding walks next time we hang out :)
(lucas, locked *tf in* on positioning the camera.)
