Talk:Photometric stereo

Extension to non-line-of-sight surfaces

Dear all—I propose to this article a new section entitled "Extension to non-line-of-sight surfaces".

Specific text to be added or removed

Recovering surface orientations around the corner. A light source illuminates a wall at known locations. Some of the photons reflect off the wall, travel to the hidden object, and reflect back eventually to the photodetector. Recording the reflected photons as a function of time and wall locations allows one to reconstruct the surface orientations of the hidden object.

Photometric stereo can be extended to scenarios where the object of interest is not in line of sight from the observer. In this so-called "looking around the corner" (or non-line-of-sight-imaging) setting, the object of interest can only be observed indirectly via some diffuse (non-mirror-like) relay wall. To measure the surface orientations of the hidden object, a light source such as pulsed laser illuminates a known location on the diffuse wall and some of the light bounces off the wall, propagates to the hidden object and eventually back to the photodetector co-located with the light source. By probing multiple locations on the diffuse wall and recording the number of photons detected as a function of time, one can construct a 3D, spatio-temporal volume of photon measurements from which the surface orientations of the hidden object can be reconstructed computationally.

Due to the presence of the diffuse wall in the observation model, recovering the surface normals of the hidden involves performing a deconvolution in addition to solving a three by three system of linear equations at each pixel similarly to the basic method. The overall inverse problem can be solved using Cholesky–Wiener decomposition with computational complexity linear in the number of voxels. However, in contrast with the basic method above, the non-line-of-sight variant of photometric stereo operates on a three-dimensional volume of measurements, rendering the technique more computationally demanding. Also, in practice, the signal-to-noise ratio of the recovered surface orientations is proportional to the output power of the light source (e.g. pulsed laser), which can be a limitation considering potential safety issues associated with operating high-power lasers.

Reason for the change

Just like non-Lambertian surfaces, inverse rendering of non-line-of-sight surfaces is a problem of general interest within the graphics community. Please have a look at the reverted edit for its wider relevance.

References supporting change

MrOllie has kindly pointed out that there may be potential self-promotion in my recent edit (which he has reverted since). So I suggest that this review article be referenced instead: https://www.nature.com/articles/s42254-020-0174-8 .

References

@J6ancmvs: Happy to work this out with you. For the moment, this looks like the abstract of an experiment? I'm wondering if we could make it more "encyclopedic" by focusing on the lessons that are drawn with respect to Photometric stereo from this experiment rather than what the researchers have attempted to achieve. JBchrch talk 11:58, 30 June 2021 (UTC)

@JBchrch: Thank you JBchrch—I revised the writing a bit to make it more encyclopedic (hopefully). Also, do you think that it'd be useful to add an equation here, similar to the one under the basic method subsection? J6ancmvs (talk) 21:26, 2 July 2021 (UTC)

This still seems to be off-topic for this article to me. The focus should be fairly narrow on Photometric stereo with fairly standard cameras and lights. This business of reflecting lasers off nearby walls is not what people will be coming to this article for, and certainly should not be taking up around 1/5th of the article. - MrOllie (talk) 23:35, 2 July 2021 (UTC)

@MrOllie: Why would anyone be coming to this article if they already know what they're looking for? Anyway I leave it to you and other senior editors to decide. Inverse rendering of non-line-of-sight scenes is already quite an old topic (just not as old as the basic method above). I opine this edit is still well within scope. J6ancmvs (talk) 00:53, 3 July 2021 (UTC)

@MrOllie and J6ancmvs: What are your thoughts on adding to the article the two first sentences of the proposed paragraphs? JBchrch talk 21:06, 16 July 2021 (UTC)

@JBchrch: Hi JBchrch, MrOllie, I could shorten the whole thing down to 4–6 sentences if length is the primary concern? But I think that this article is pretty bare and should probably be fleshed out a bit in general... J6ancmvs (talk) 12:14, 17 July 2021 (UTC)

I still think this is off topic for this article. In the spirit of compromise, I could support something very short. Maybe: "With the use of special light sources, Photometric stereo can be extended to scenarios where the object of interest is not in line of sight from the observer." to follow immediately after the sentence mentioning structured light. Also, J6ancmv, you mentioned 'Inverse rendering of non-line-of-sight scenes is already quite an old topic' Could you provide an older source then? It would ease my WP:UNDUE concerns if we had a source that had been cited a lot here. - MrOllie (talk) 12:20, 17 July 2021 (UTC)

@JBchrch and MrOllie: That section is entitled "non-Lambertian surfaces" so my edit (non-line-of-sight surfaces) definitely doesn't belong there, sorry. You could Google "Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging" (2012) for older, albedo-only NLOS imaging. Perhaps you could try to get an expert to weigh in here? J6ancmvs (talk) 20:49, 17 July 2021 (UTC)

@MrOllie: Previously I entertained the possibility that my edit is off topic but now I'm convinced it is your unfamiliarity with the subject matter. Third party expert opinion may be required here. J6ancmvs (talk) 21:00, 17 July 2021 (UTC)

J6ancmvs, No, it is definitely off topic, but in deference to JBchrch's position I was willing to work toward a compromise. It is difficult to reach one if conversations are accompanied by personal attacks, though. RE: the substance of your complaint, it is a simple enough thing to change the section title to something more inclusive, such as 'Extensions of the problem' MrOllie (talk) 22:36, 17 July 2021 (UTC)

MrOllie: Saying someone is (apparently) unfamiliar with a certain topic is not a personal attack, although I agree such a remark was not necessary. Do you have a WP reference that discusses what is considered "off topic"? J6ancmvs (talk) 00:06, 18 July 2021 (UTC)

@J6ancmvs: MrOllie might have a better answer but to the best of my knowledge there is no policy or guideline on what is on-topic or off-topic. These questions are simply subject to talk page consensus, per WP:ONUS. JBchrch talk 00:55, 18 July 2021 (UTC)

@JBchrch: Thanks for pointing me at WP:ONUS. Best to wait and see how Wikipedia talk:WikiProject Computer science feels about the edit.

@J6ancmvs: I was supporting your request on the basis of WP:AGF and the fact that we most often get those from corporations rather than academics. Given your last comment and the fact that I don't know anything about this topic area, I will withdraw my support. You can ask for a third-party opinion at Wikipedia talk:WikiProject Computer science but I cannot guarantee that someone will be available. JBchrch talk 23:36, 17 July 2021 (UTC)

@JBchrch: Thanks, I have asked there.

@JBchrch: I just read up on WP:AGF—I wanted only to point out MrOllie is unfamiliar with this topic (apparent from his replies). J6ancmvs (talk) 17:24, 18 July 2021 (UTC)

I'm a member of WP:WPCS, not an expert on this topic but I share MrOllie's impression that including this material here is a stretch as photometric stereo doesn't normally concern itself with revealing hidden objects. It would probably be best to create a new Non-line-of-sight imaging article to describe these techniques. If, further down the line, editors more versed in this than MrOllie and I agree that these are more closely related than we have assessed, a WP:MERGE can be performed. ~Kvng (talk) 18:40, 21 October 2021 (UTC)