Abstract Hyperspectral cameras are advanced imaging devices that capture images in many different wavelengths of light, providing rich information about the properties of objects and materials that cannot be obtained using conventional RGB cameras. Unlike RGB cameras, which capture only three bands of the electromagnetic spectrum (red, green, and blue), hyperspectral cameras can capture hundreds of narrow spectral bands, allowing for highly detailed and accurate spectral analysis of the objects being imaged like their chemical composition, physical properties, and biological characteristics of the objects and materials being imaged. This makes hyperspectral imaging a powerful tool for a wide range of applications, including remote sensing, smart farming, medical imaging, industrial inspection, and surveillance.

One of the key challenges in hyperspectral imaging is, however, dealing with noise and low-light conditions, which can severely degrade image quality and limit the effectiveness of subsequent analysis and interpretation. To address this challenge, advanced denoising algorithms are needed that can effectively remove noise while preserving important spectral information. In this context, the proposed research project aims to develop novel end-to-end approaches for hyperspectral image enhancement. Furthermore, we will also explore the exciting possibility of synthesizing hyperspectral images from standard RGB images, which has received limited attention in the literature, but has the potential to greatly expand the capabilities of hyperspectral imaging. The conversion of RGB and broadband visible near infrared (VNIR) images to hyperspectral images using labeled data obtained through this process will pave the way for studies that will enable machine learning algorithms to perform better.

To achieve our goals, we will leverage recent advances in deep learning and computer vision, including vision transformers, self-supervised learning, and diffusion-based generative modeling. We will also develop novel loss functions and optimization strategies to ensure that our models are robust and effective. To evaluate our methods, we will use a variety of benchmark datasets, including both simulated and real-world hyperspectral images. Moreover, we will collect our own dataset for hyperspectral image reconstruction from standard and VNIR camera images.

Overall, the proposed research project has the potential to greatly advance the state-of-the-art in hyperspectral imaging, and to enable new applications in a wide range of fields. By developing end-to-end approaches for denoising, enhancing, and synthesizing hyperspectral images, we can unlock the full potential of this powerful imaging technique, and "see the invisible" in ways that were previously impossible.

Related Publications Hyperspectral Image Denoising via Self-Modulating Convolutional Neural Networks
Signal Processing
Orhan Torun, Seniha Esen Yuksel, Erkut Erdem, Nevrez Imamoglu, Aykut Erdem