If Spectra is a Cube

When I interned at the Carnegie Mellon Vision Science Labs the summer after my junior year, I struggled to run the code in a way that wouldn’t crash my tiny computer. So, Professor Sankaranarayanan told me to ask one of his graduate students for ways to make more efficient pipelines. I told him I was working on hyperspectral images. He paused for a moment and asked, “Are those cubes?” I answered that they were. We finally came up with a solution to the problem.

If you are familiar with spectroscopy, yes vision science in general, you might be wondering why they call hyperspectral images ‘cubes’. I will explain why. Let’s say you want to have a black and white image stored on a computer. The data appear to be squared. Each value in the square is a value from 0 to 1, with 0 being black and 1 being white. This is what it looks like:

What about color pictures? It’s less complicated than black and white because it doesn’t take much for the human eye to perceive color. We call red, green, and blue the ‘primary colors,’ but that doesn’t mean that all colors are made up of red, green, and blue. If you produce light with a wavelength of 570–580 nanometers, you will see yellow. Red, green, and blue are missing. However, if you have red light (620 to 750 nm) and green light (490–580 nm), your brain will think you are seeing yellow light. Not you! But our eyes are designed for basic tasks, so this is all we need. As a result, most computer screens have red, green, and blue diodes. That means color images are stored on computers like this:

With our limited eyes, we have a good idea of ​​the world around us. But what if we could get a computer eye that could distinguish a narrower range of colors than the human eye? There is one matrix for 490 to 495 nm, another for 495 to 500 nm, and so on. That’s what a hyperspectral image is! You can’t really show it on a standard screen because that’s not what computers are designed to do, but you can get a lot of data on what component you’re observing. And, since there are hundreds or thousands of matrices all stacked on top of each other, it looks like a cube! And so the graduate student called it a cube.

A year after this, I worked at Specere Labs, studying light and spectroscopy. I explained to other researchers what I studied at CMU. But they did not see hyperspectral images as cubes; they see them as groups of spectra.

Specere labs study spectroscopy, but rarely involve themselves in the study of hyperspectral images. More focused is the actual mechanisms of the infrared spectroscopy equipment in the experiment itself. Thus, the focus is more on material science, heat transfer science, and chemistry.

When a spectrometer is used, it usually returns a spectrum like this:

The plot above may appear as a cross-section of a hyperspectral image, but almost no one thinks of it that way because that is not how spectral data is usually captured. So at Specere Labs, hyperspectral images are not cubes. They are groups of spectra taken from regions close to each other.

So while I don’t think this information is earth-shattering, I think it illustrates why I enjoy cross-disciplinary work and learning from multiple professors in multiple labs.