research-article

3D imaging spectroscopy for measuring hyperspectral patterns on solid objects

Authors:

Todd Alan Harvey,

David S. Kittle,

Holly Rushmeier,

Richard O. Prum, and

David J. BradyAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 31, Issue 4

Article No.: 38, Pages 1 - 11

https://doi.org/10.1145/2185520.2185534

Published: 01 July 2012 Publication History

Abstract

Sophisticated methods for true spectral rendering have been developed in computer graphics to produce highly accurate images. In addition to traditional applications in visualizing appearance, such methods have potential applications in many areas of scientific study. In particular, we are motivated by the application of studying avian vision and appearance. An obstacle to using graphics in this application is the lack of reliable input data. We introduce an end-to-end measurement system for capturing spectral data on 3D objects. We present the modification of a recently developed hyperspectral imager to make it suitable for acquiring such data in a wide spectral range at high spectral and spatial resolution. We capture four megapixel images, with data at each pixel from the near-ultraviolet (359 nm) to near-infrared (1,003 nm) at 12 nm spectral resolution. We fully characterize the imaging system, and document its accuracy. This imager is integrated into a 3D scanning system to enable the measurement of the diffuse spectral reflectance and fluorescence of specimens. We demonstrate the use of this measurement system in the study of the interplay between the visual capabilities and appearance of birds. We show further the use of the system in gaining insight into artifacts from geology and cultural heritage.

Supplementary Material

ZIP File (a38-kim.zip)

Supplemental material.

Download
126.68 MB

MP4 File (tp126_12.mp4)

Download
21.32 MB

References

[1]

3DCoForm, 2012. MeshLab. http://meshlab.sourceforge.net/.

[2]

Andersson, S., ornborg, J., and Andersson, M. 1998. Ultraviolet sexual dimorphism and assortative mating in blue tits. In Proc. R. Soc. Lond., vol. 265, 445--450.

[3]

Arnold, K. E., Owens, I. P. F., and Marshall, N. J. 2002. Fluorescent signaling in parrots. Science 295, 5552, 92.

[4]

Attas, M., Cloutis, E., Collins, C., Goltz, D., Majzels, C., Mansfield, J., and Mantsch, H. 2003. Near-infrared spectroscopic imaging in art conservation: investigation of drawing constituents. J. Cultural Heritage 4, 2, 127--136.

[5]

Battle, D. R. 1997. The measurement of colour. In Colour Physics for Industry, R. McDonald, Ed., 2nd ed. Soc. Dyers Col., Bradford, 57--80.

[6]

Bennett, A. T. D., Cuthill, I. C., Partridge, J. C., and Maier, E. J. 1996. Ultraviolet vision and mate choice in zebra finches. Nature 380, 433--435.

[7]

Bernardini, F., and Rushmeier, H. 2002. The 3D model acquisition pipeline. Comput. Graph. Forum 21, 2, 149--149.

[8]

Bernardini, F., Martin, I. M., and Rushmeier, H. E. 2001. High-quality texture reconstruction from multiple scans. IEEE Trans. Vis. Comput. Graph. 7, 4, 318--332.

Digital Library

[9]

Bioucas-Dias, J., and Figueiredo, M. 2007. A new twist: two-step iterative shrinkage/thresholding for image restoration. IEEE Trans. Image Processing 16, 12, 2992--3004.

Digital Library

[10]

Brady, D. J. 2008. Optical Imaging and Spectroscopy. Wiley-interscience, New Jersey.

[11]

Brusco, N., Capeleto, S., Fedel, M., Paviotti, A., Poletto, L., Cortelazzo, G. M., and Tondello, G. 2006. A system for 3D modeling frescoed historical buildings with multispectral texture information. Machine Vision and Appl. 17, 6, 373--393.

Digital Library

[12]

Burns, P. D. 2002. Slanted-edge MTF for digital camera and scanner analysis. In Proc. PICS Conf., IS&T, 135--138.

[13]

Chambolle, A. 2004. An algorithm for total variation minimization and applications. J. Mathematical Imaging and Vision 20, 1--2, 89--97.

Digital Library

[14]

Chen, D. M., and Goldsmith, T. H. 1986. Four spectral classes of cone in the retinas of birds. J. Comparative Physiology A 159, 4, 473--479.

[15]

Chen, D. M., Collins, J. S., and Goldsmith, T. H. 1984. The ultraviolet receptor of bird retinas. Science 225, 4659, 337--340.

[16]

CIE. 2001. Improvement to industrial colour difference equation. CIE Pub. 142, Commission Internationale de l'Eclairage, Vienna.

[17]

Devlin, K., Chalmers, A., Wilkie, A., and Purgathofer, W. 2002. Tone reproduction and physically based spectral rendering. Eurographics 2002: State of the Art Reports, 101--123.

[18]

Du, H., Tong, X., Cao, X., and Lin, S. 2009. A prism-based system for multispectral video acquisition. In Proc. Int. Conf. Comput. Vision (ICCV), IEEE, 175--182.

[19]

Elgazzar, S., Liscano, R., Blais, F., and Miles, A. 1997. 3-D data acquisition for indoor environment modeling using a compact active range sensor. In Proc. the IEEE Instrumentation, Measurement and Technology Conference, 1--8.

[20]

Farouk, M., Rifai, I. E., Tayar, S. E., Shishiny, H. E., Hosny, M., Rayes, M. E., Gomes, J., Giordano, F., Rushmeier, H. E., Bernardini, F., and Magerlein, K. 2003. Scanning and processing 3D objects for web display. In Proc. Int. Conf. 3D Digital Imaging and Modeling (3DIM), 310--317.

[21]

Figueiredo, M., Nowak, R., and Wright, S. 2007. Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE J. Selected Topics in Signal Processing 1, 4, 586--597.

[22]

Fischer, C., and Kakoulli, I. 2006. Multispectral and hyper-spectral imaging technologies in conservation: current research and potential applications. Reviews in Conservation 7, 3--16.

[23]

Habel, R., Kudenov, M., and Wimmer, M. 2012. Practical spectral photography. Comput. Graph. Forum 31, 2, 1--10.

Digital Library

[24]

Hart, N. S. 2001. The visual ecology of avian photoreceptors. Progress in Retinal and Eye Research 20, 5, 675--703.

[25]

Holroyd, M., Lawrence, J., and Zickler, T. 2010. A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance. ACM Trans. Graph. (Proc. SIGGRAPH 2010) 29, 3, 99:1--12.

Digital Library

[26]

ISO. 2000. Photography -- electronic still-picture cameras -- resolution measurements. Tech. Rep. ISO 12233:2000, International Organization for Standardization (ISO).

[27]

Kawakami, R., Wright, J., Tai, Y.-W., Matsushita, Y., Ben-Ezra, M., and Ikeuchi, K. 2011. High-resolution hyperspectral imaging via matrix factorization. In Proc. IEEE Conf. Comput. Vision and Pattern Recognition, 2329--2336.

Digital Library

[28]

Kittle, D., Choi, K., Wagadarikar, A., and Brady, D. J. 2010. Multiframe image estimation for coded aperture snapshot spectral imagers. Appl. Opt. 49, 36, 6824--6833.

[29]

Labshpere, 2011. Spectralon diffuse reflectance standards. http://www.labsphere.com/uploads/datasheets/diffuse-reflectance-standards-product-sheet.pdf.

[30]

Land, M. F., and Nilsson, D.-E., Eds. 2002. Animal Eyes. Oxford University Press, Oxford.

[31]

Lensch, H., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H. 2003. Image-based reconstruction of spatial appearance and geometric detail. ACM Trans. Graph. (Proc. SIGGRAPH 2003) 22, 2, 234--257.

Digital Library

[32]

Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S. E., Davis, J., Ginsberg, J., Shade, J., and Fulk, D. 2000. The digital Michelangelo project: 3D scanning of large statues. In Proc. SIGGRAPH 2000, 131--144.

Digital Library

[33]

Mansouri, A., Lathuiliere, A., Marzani, F. S., Voisin, Y., and Gouton, P. 2007. Toward a 3D multispectral scanner: An application to multimedia. IEEE Multimedia 14, 1, 40--47.

Digital Library

[34]

Mcgraw, K. J. 2006. Mechanics of uncommon colors in birds: pterins, porphyrins, and psittacofulvins. In Bird coloration, Vol. I, Mechanisms and measurements, G. E. Hill and K. J. Mcgraw, Eds. Harvard University Press, Cambridge, Massachusetts.

[35]

Newsome, D., and Modreski, P. 1981. The colors and spectral distributions of fluorescent minerals. J. the Fluorescent Mineral Society 10, 7--56.

[36]

Next-Limit-Technologies, 2012. Maxwell Render. http://www.maxwellrender.com/.

[37]

nVidia, 2012. About iRay, physically correct GPU rendering technology. http://www.mentalimages.com/products/iray/about-iray.html.

[38]

Qin, J. 2010. Hyperspectral imaging instruments. In Hyperspectral Imaging for Food Quality Analysis and Control, D.-W. Sun, Ed. Elsevier, 129--175.

[39]

Rapantzikos, K., and Balas, C. 2005. Hyperspectral imaging: potential in non-destructive analysis of palimpsests. In Proc. Int. Conf. Image Processing (ICIP), vol. 2, 618--621.

[40]

Saunders, D., and Cupitt, J. 1993. Image processing at the National Gallery: The VASARI project. National Gallery Tech. Bull. 14, 72--86.

[41]

Stoddard, M. C., and Prum, R. O. 2008. Evolution of avian plumage color in a tetrahedral color space: a phylogenetic analysis of new world buntings. The American Naturalist 171, 6, 755--776.

[42]

Stoddard, M. C., and Prum, R. O. 2011. How colorful are birds? Evolution of the avian plumage color gamut. Behavioral Ecology 22, 5, 1042--1052.

[43]

Sugiura, H., Kuno, T., Watanabe, N., Matoba, N., Hayashi, J., and Miyata, Y. 2000. Development of a multi-spectral camera system. In Proc. SPIE 3965, 331--339.

[44]

Sun, X., and Pitsianis, N. 2008. Solving non-negative linear inverse problems with the NeAREst method. Proc. SPIE 7074, 707402.

[45]

Thoury, M., Elias, M., Frigerio, J. M., and Barthou, C. 2005. Non-destructive identification of varnishes by UV fluorescence spectroscopy. In Proc. SPIE 5857, 1--11.

[46]

Vos, J. J. 1978. Colorimetric and photometric properties of a 2-deg fundamental observer. Color Res. Appl. 3, 125--128.

[47]

Wagadarikar, A., John, R., Willett, R., and Brady, D. J. 2008. Single disperser design for coded aperture snapshot spectral imaging. Appl. Opt. 47, 10, B44--B51.

[48]

Wagadarikar, A. A., Pitsianis, N. P., Sun, X., and Brady, D. J. 2009. Video rate spectral imaging using a coded aperture snapshot spectral imager. Opt. Express 17, 8, 6368--6388.

[49]

Ware, G., Chabries, D., Christiansen, R., Brady, J., and Martin, C. 2000. Multispectral analysis of ancient Maya pigments: implications for the Naj Tunich Corpus. In Proc. IEEE Geoscience and Remote Sensing Symposium, vol. 6, 2489--2491.

[50]

Wright, S., Nowak, R., and Figueiredo, M. 2009. Sparse reconstruction by separable approximation. IEEE Trans. Signal Processing 57, 7, 2479--2493.

Digital Library

[51]

Zhao, Y., Berns, R. S., Taplin, L. A., and Coddington, J. 2008. An investigation of multispectral imaging for the mapping of pigments in paintings. In Proc. SPIE 6810, 1--9.

Cited By

Cai ZLiu ZYu JZhang ZDa FJin C(2024)Reversible-Prior-Based Spectral-Spatial Transformer for Efficient Hyperspectral Image ReconstructionInternational Journal on Semantic Web & Information Systems10.4018/IJSWIS.34445720:1(1-22)Online publication date: 7-May-2024
https://dl.acm.org/doi/10.4018/IJSWIS.344457
Vasas VLowell MVilla JJamison QSiegle AKatta PBhagavathula PKevan PFulton DLosin NKepplinger DYetzbacher MSalehian SForkner RHanley D(2024)Recording animal-view videos of the natural world using a novel camera system and software packagePLOS Biology10.1371/journal.pbio.300244422:1(e3002444)Online publication date: 23-Jan-2024
https://doi.org/10.1371/journal.pbio.3002444
Zhang HYe QSun KLi YWu YXu HLuo H(2024)Anti-laser interference methods for compressive spectral imaging based on grayscale coded apertureOptics Continuum10.1364/OPTCON.5042193:1(102)Online publication date: 9-Jan-2024
https://doi.org/10.1364/OPTCON.504219
Show More Cited By

Recommendations

Compact snapshot hyperspectral imaging with diffracted rotation

Traditional snapshot hyperspectral imaging systems include various optical elements: a dispersive optical element (prism), a coded aperture, several relay lenses, and an imaging lens, resulting in an impractically large form factor. We seek an ...
Read More
Compact single-shot hyperspectral imaging using a prism

We present a novel, compact single-shot hyperspectral imaging method. It enables capturing hyperspectral images using a conventional DSLR camera equipped with just an ordinary refractive prism in front of the camera lens. Our computational imaging ...
Read More
Catadioptric hyperspectral imaging, an unmixing approach

Hyperspectral imaging systems provide dense spectral information on the scene under investigation by collecting data from a high number of contiguous bands of the electromagnetic spectrum. The low spatial resolutions of these sensors frequently give rise ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 31, Issue 4

July 2012

935 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2185520

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2012

Published in TOG Volume 31, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

112
Total Citations
View Citations
1,876
Total Downloads

Downloads (Last 12 months)74
Downloads (Last 6 weeks)7

Other Metrics

View Author Metrics

Citations

Cited By

Cai ZLiu ZYu JZhang ZDa FJin C(2024)Reversible-Prior-Based Spectral-Spatial Transformer for Efficient Hyperspectral Image ReconstructionInternational Journal on Semantic Web & Information Systems10.4018/IJSWIS.34445720:1(1-22)Online publication date: 7-May-2024
https://dl.acm.org/doi/10.4018/IJSWIS.344457
Vasas VLowell MVilla JJamison QSiegle AKatta PBhagavathula PKevan PFulton DLosin NKepplinger DYetzbacher MSalehian SForkner RHanley D(2024)Recording animal-view videos of the natural world using a novel camera system and software packagePLOS Biology10.1371/journal.pbio.300244422:1(e3002444)Online publication date: 23-Jan-2024
https://doi.org/10.1371/journal.pbio.3002444
Zhang HYe QSun KLi YWu YXu HLuo H(2024)Anti-laser interference methods for compressive spectral imaging based on grayscale coded apertureOptics Continuum10.1364/OPTCON.5042193:1(102)Online publication date: 9-Jan-2024
https://doi.org/10.1364/OPTCON.504219
Wei NSun YJiang TGao Q(2024)Direct object detection with snapshot multispectral compressed imaging in a short-wave infrared bandOptics Letters10.1364/OL.51728449:8(1941)Online publication date: 4-Apr-2024
https://doi.org/10.1364/OL.517284
Ma JZhenming YCheng Ldi jZhan NZhou Yzhao hXu K(2024)Ultra-high-speed four-dimensional hyperspectral imagingOptics Express10.1364/OE.520788Online publication date: 7-May-2024
https://doi.org/10.1364/OE.520788
Xu PLiu LZheng HYuan XXu CXue L(2024)Degradation-Aware Dynamic Fourier-Based Network for Spectral Compressive ImagingIEEE Transactions on Multimedia10.1109/TMM.2023.330445026(2838-2850)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3304450
Dong YGao DLi YShi GLiu D(2024)Degradation Estimation Recurrent Neural Network With Local and Non-Local Priors for Compressive Spectral ImagingIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.341027262(1-15)Online publication date: 2024
https://doi.org/10.1109/TGRS.2024.3410272
Huang YHossain MLiu YSun KZhang BXu C(2024)Data rectification and decoding of a microlens array-based multi-spectral light field imaging systemOptics and Lasers in Engineering10.1016/j.optlaseng.2024.108327180(108327)Online publication date: Sep-2024
https://doi.org/10.1016/j.optlaseng.2024.108327
Ji YShi JZhang YYang HZong YXu L(2023)HyFormer: Hybrid Grouping-Aggregation Transformer and Wide-Spanning CNN for Hyperspectral Image Super-ResolutionRemote Sensing10.3390/rs1517413115:17(4131)Online publication date: 23-Aug-2023
https://doi.org/10.3390/rs15174131
Na JRyu JJo YKim M(2023)Developing an HDR Imaging Method for an Ultra-thin Light-Field CameraJournal of the Korea Computer Graphics Society10.15701/kcgs.2023.29.3.1329:3(13-19)Online publication date: 25-Jul-2023
https://doi.org/10.15701/kcgs.2023.29.3.13
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents

-