Photometric stereo

Short description: 3D imaging technique

Not to be confused with Stereo photogrammetry.

Photometric stereo analyzes multiple images of an object under different lighting conditions to estimate a normal direction at each pixel.

Photometric stereo is a technique in computer vision for estimating the surface normals of objects by observing that object under different lighting conditions (photometry). It is based on the fact that the amount of light reflected by a surface is dependent on the orientation of the surface in relation to the light source and the observer.[1] By measuring the amount of light reflected into a camera, the space of possible surface orientations is limited. Given enough light sources from different angles, the surface orientation may be constrained to a single orientation or even overconstrained.

The technique was originally introduced by Woodham in 1980.[2] The special case where the data is a single image is known as shape from shading, and was analyzed by B. K. P. Horn in 1989.[3] Photometric stereo has since been generalized to many other situations, including extended light sources and non-Lambertian surface finishes. Current research aims to make the method work in the presence of projected shadows, highlights, and non-uniform lighting.

Photometric stereo is widely used in various fields, including archaeology,[4][5] cultural heritage conservation,[6] and quality control.[7] It is now integrated into widely used open-source software, such as Meshroom.[8]

## Contents

* 1 Basic method
* 2 Non-Lambertian surfaces
* 2.1 Specular reflections
* 2.2 General BRDFs and beyond
* 3 Uncalibrated photometric stereo
* 4 See also
* 5 References

## Basic method

Under Woodham's original assumptions — Lambertian reflectance, known point-like distant light sources, and uniform albedo — the problem can be solved by inverting the linear equation I=L⋅n, where I is a (known) vector of m observed intensities, n is the (unknown) surface normal, and L is a (known) 3×m matrix of normalized light directions.

This model can easily be extended to surfaces with non-uniform albedo, while keeping the problem linear.[9] Taking an albedo reflectivity of k, the formula for the reflected light intensity becomes I=k(L⋅n). If L is square (there are exactly 3 lights) and non-singular, it can be inverted, giving L−1I=kn. Since the normal vector is known to have length 1, k must be the length of the vector kn, and n is the normalised direction of that vector.

If L is not square (there are more than 3 lights), a generalisation of the inverse can be obtained using the Moore–Penrose pseudoinverse,[10] by simply multiplying both sides with LT, giving LTI=LTk(L⋅n), (LTL)−1LTI=kn, after which the normal vector and albedo can be solved as described above.

## Non-Lambertian surfaces

The classical photometric stereo problem concerns itself only with Lambertian surfaces, with perfectly diffuse reflection. This is unrealistic for many types of materials, especially metals, glass and smooth plastics, and will lead to aberrations in the resulting normal vectors.

Many methods have been developed to lift this assumption. In this section, a few of these are listed.

### Specular reflections

Historically, in computer graphics, the commonly used model to render surfaces started with Lambertian surfaces and progressed first to include simple specular reflections. Computer vision followed a similar course with photometric stereo. Specular reflections were among the first deviations from the Lambertian model. These are a few adaptations that have been developed.

* Many techniques ultimately rely on modelling the reflectance function of the surface, that is, how much light is reflected in each direction.[11] This reflectance function has to be invertible. The reflected light intensities towards the camera is measured, and the inverse reflectance function is fit onto the measured intensities, resulting in a unique solution for the normal vector.

### General BRDFs and beyond

According to the Bidirectional reflectance distribution function (BRDF) model, a surface may distribute the amount of light it receives in any outward direction. This is the most general known model for opaque surfaces. Some techniques have been developed to model (almost) general BRDFs. In practice, all of these require many light sources to obtain reliable data. These are methods in which surfaces with general BRDFs can be measured.

* Determine the explicit BRDF prior to scanning.[12] To do this, a different surface is required that has the same or a very similar BRDF, of which the actual geometry (or at least the normal vectors for many points on the surface) is already known.[13] The lights are then individually shone upon the known surface, and the amount of reflection into the camera is measured. Using this information, a look-up table can be created that maps reflected intensities for each light source to a list of possible normal vectors. This puts constraints on the possible normal vectors the surface may have, and reduces the photometric stereo problem to an interpolation between measurements. Typical known surfaces to calibrate the look-up table with are spheres for their wide variety of surface orientations.
* Restricting the BRDF to be symmetrical.[14] If the BRDF is symmetrical, the direction of the light can be restricted to a cone about the direction to the camera. Which cone this is depends on the BRDF itself, the normal vector of the surface, and the measured intensity. Given enough measured intensities and the resulting light directions, these cones can be approximated and therefore the normal vectors of the surface.

Some progress has been made towards modelling an even more general surfaces, such as Spatially Varying Bidirectional Distribution Functions (SVBRDF), Bidirectional surface scattering reflectance distribution functions (BSSRDF), and accounting for interreflections.[15][16] However, such methods are still fairly restrictive in photometric stereo. Better results have been achieved with structured light.[17]

## Uncalibrated photometric stereo

Uncalibrated Photometric Stereo is an approach in photometric stereo that aims to reconstruct the 3D shape of an object from images captured under unknown lighting conditions[18]. Unlike classical methods, which often assume controlled or known lighting setups, this approach removes these constraints, making it adaptable to diverse and real-world environments.

The advent of deep learning has revolutionized universal PS by replacing handcrafted assumptions with data-driven models[19]. Recent approaches leverage Transformer-based architectures and multi-scale encoder–decoder networks to directly estimate surface normals from input images[20].

Uncalibrated Photometric Stereo is inherently an ill-posed problem, as it attempts to recover 3D shape and lighting conditions simultaneously from images alone. This leads to fundamental ambiguities in the reconstruction process, which manifest as systematic errors in the recovered geometry, including global distortions in the object's overall shape[21], and misinterpretation of surface orientation, where concave regions may appear convex and vice versa.[22]

To address the challenges of uncalibrated photometric stereo, hybrid methods have emerged that combine multi-view stereo and photometric stereo. These approaches leverage the strengths of both techniques, including geometric reliability and resolution.[21][23]

## See also

* Photoclinometry
* Photometry
* Stereo vision
* 3D scanner

## References

1. ↑ Ying Wu. "Radiometry, BRDF and Photometric Stereo". Northwestern University. http://users.eecs.northwestern.edu/~yingwu/teaching/EECS432/Notes/lighting.pdf.
2. ↑ Woodham, R.J. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineerings 19, I, 139-144.
3. ↑ B. K. P. Horn, 1989. Obtaining shape from shading information. In B. K. P. Horn and M. J. Brooks, eds., Shape from Shading, pages 121–171. MIT Press.
4. ↑ Laurent, Antoine; Coupry, Benjamin; Mélou, Jean; Quéau, Yvain; Fritz, Carole; Durou, Jean-Denis (2024-12-14). "(R)-evolution in the implementation of photometric stereo – Application to archaeology" (in English). The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLVIII-2-W8-2024: 289–294. doi:10.5194/isprs-archives-XLVIII-2-W8-2024-289-2024. ISSN 1682-1750. Bibcode: 2024ISPAr48W8..289L. https://isprs-archives.copernicus.org/articles/XLVIII-2-W8-2024/289/2024/isprs-archives-XLVIII-2-W8-2024-289-2024.html.
5. ↑ Keep, Thomas James; Robinson, Madeline G. P.; Shoobert, Jackson; Birkett-Rees, Jessie (2025-07-29). "An Australian Overview: The Creation and Use of 3D Models in Australian Universities" (in en-US). Journal of Computer Applications in Archaeology 8 (1): 208–223. doi:10.5334/jcaa.222. ISSN 2514-8362.
6. ↑ Bornstein, Daniel; Keep, Thomas J. (2023-07-03). "New Dimensions in Conservation Imaging: Combining Photogrammetry and Photometric Stereo for 3D Documentation of Heritage Artefacts". AICCM Bulletin 44 (1–2): 148–162. doi:10.1080/10344233.2023.2288467. ISSN 1034-4233.
7. ↑ Farooq, Abdul Rehman; Smith, Melvyn Lionel; Smith, Lyndon Neal; Midha, Sagar (2005-12-01). "Dynamic photometric stereo for on line quality control of ceramic tiles". Computers in Industry. Machine Vision Special Issue 56 (8): 918–934. doi:10.1016/j.compind.2005.05.017. ISSN 0166-3615. https://www.sciencedirect.com/science/article/abs/pii/S0166361505001296.
8. ↑ "meshroom.nodes.aliceVision.PhotometricStereo.PhotometricStereo — Meshroom documentation". https://meshroom.readthedocs.io/en/stable/generated/meshroom.nodes.aliceVision.PhotometricStereo.PhotometricStereo.html.
9. ↑ S. Barsky and Maria Petrou, 2003. The 4-source photometric stereo technique for 3-dimensional surfaces in the presence of highlights and shadows. In IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, issue 10, pages 1239–1252. IEEE.
10. ↑ Chaman Singh Verma and Mon-Ju Wu. "Photometric Stereo". University of Wisconsin-Madison. http://pages.cs.wisc.edu/~csverma/CS766_09/Stereo/stereo.html.
11. ↑ Hemant D. Tagare and Rui J.P. de Figueiredo, 1991. A Theory of Photometric Stereo for a Class of Diffuse Non-Lambertian Surfaces. In IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 13, no. 2. IEEE.
12. ↑ Katsushi Ikeuchi, 1981. Determining Surface Orientations of Specular Surfaces by Using the Photometric Stereo Method. In IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-3, issue 6, pages 661-669. IEEE.
13. ↑ Aaron Hertzmann and Steven M. Seitz, 2005. Example-Based Photometric Stereo: Shape Reconstruction with General, Verying BRDFs. In IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 8. IEEE.
14. ↑ Michael Holroyd, Jason Lawrence, Greg Humphreys and Todd Zickler, 2008. A Photometric Approach for Estimating Normals and Tangents. In ACM SIGGRAPH Asia 2008 Papers, pages 133:1-133:9. ACM.
15. ↑ Shree K. Nayar, Katsushi Ikeuchi and Takeo Kanade, 1991. Shape from interreflections. In International Journal of Computer Vision, vol. 6, number 3, pages 173-195.
16. ↑ Miao Liao, Xinyu Huang and Ruigang Yang, 2011. Interreflection Removal for Photometric Stereo by Using Spectrum-dependent Albedo. In 2011 IEEE Conference on Computer Vision and Pattern Recognition, pages 689-696. IEEE.
17. ↑ Tongbo Chen, Hendrik Lensch, Christian Fuchs and H.P. Seidel, 2007. Polarization and Phase-shifting for 3D Scanning of Translucent Objects. In IEEE Conference on Computer Vision and Pattern Recognition, 2007, pages 1-8. IEEE.
18. ↑ Hayakawa, Hideki (1994-11-01). "Photometric stereo under a light source with arbitrary motion" (in EN). JOSA A 11 (11): 3079–3089. doi:10.1364/JOSAA.11.003079. ISSN 1520-8532. Bibcode: 1994JOSAA..11.3079H. https://opg.optica.org/josaa/abstract.cfm?uri=josaa-11-11-3079.
19. ↑ Ikehata, Satoshi (2018), "CNN-PS: CNN-Based Photometric Stereo for General Non-convex Surfaces", Computer Vision – ECCV 2018, Lecture Notes in Computer Science, 11219, Cham: Springer International Publishing, pp. 3–19, doi:10.1007/978-3-030-01267-0_1, ISBN 978-3-030-01266-3, https://doi.org/10.1007/978-3-030-01267-0_1, retrieved 2025-11-29
20. ↑ Hardy, Clément; Quéau, Yvain; Tschumperlé, David (2024-11-01). "Uni MS-PS: A multi-scale encoder-decoder transformer for universal photometric stereo". Computer Vision and Image Understanding 248. doi:10.1016/j.cviu.2024.104093. ISSN 1077-3142. https://www.sciencedirect.com/science/article/pii/S1077314224001747.
21. ↑ 21.0 21.1 Coupry, Benjamin; Brument, Baptiste; Laurent, Antoine; Mélou, Jean; Quéau, Yvain; Durou, Jean-Denis (February 2025). "Assessing the Quality of 3D Reconstruction in the Absence of Ground Truth: Application to a Multimodal Archaeological Dataset". 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). pp. 3457–3466. doi:10.1109/WACV61041.2025.00341. ISBN 979-8-3315-1083-1. https://hal.science/hal-04942610.
22. ↑ Coupry, Benjamin; Mélou, Jean; Laurent, Antoine; Brument, Baptiste; Gurdjos, Pierre; Quéau, Yvain; Durou, Jean-Denis (2024-07-01). "Stéréophotométrie avec estimation locale de l'éclairage - Application à la reconstruction 3D du patrimoine archéologique" (in fr). Congrès Reconnaissance des Formes, Image, Apprentissage et Perception (RFIAP 2024). https://hal.science/hal-04594418.
23. ↑ Coupry, Benjamin; Lauze, François; Laurent, Antoine; Melou, Jean; Quéau, Yvain; Durou, Jean-Denis (2025). "Self-calibrated Near-Light Photometric Stereo Using a Geometric Proxy". in Bubba, Tatiana A.; Gaburro, Romina; Gazzola, Silvia et al. (in en). Scale Space and Variational Methods in Computer Vision. Lecture Notes in Computer Science. 15668. Cham: Springer Nature Switzerland. pp. 364–376. doi:10.1007/978-3-031-92369-2_28. ISBN 978-3-031-92369-2. https://hal.science/hal-05004867.

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Photometric stereo. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Photometric_stereo&oldid=4417182"