Totems: Physical Objects for Verifying Visual Integrity

• URL: http://arxiv.org/abs/2209.13032v1
• Date: Mon, 26 Sep 2022 21:19:37 GMT
• Title: Totems: Physical Objects for Verifying Visual Integrity
• Authors: Jingwei Ma, Lucy Chai, Minyoung Huh, Tongzhou Wang, Ser-Nam Lim, Phillip Isola, Antonio Torralba
• Abstract summary: We introduce a new approach to image forensics: placing physical refractive objects, which we call totems, into a scene so as to protect any photograph taken of that scene. Totems bend and redirect light rays, thus providing multiple, albeit distorted, views of the scene within a single image. A defender can use these distorted totem pixels to detect if an image has been manipulated.
• Score: 68.55682676677046
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We introduce a new approach to image forensics: placing physical refractive objects, which we call totems, into a scene so as to protect any photograph taken of that scene. Totems bend and redirect light rays, thus providing multiple, albeit distorted, views of the scene within a single image. A defender can use these distorted totem pixels to detect if an image has been manipulated. Our approach unscrambles the light rays passing through the totems by estimating their positions in the scene and using their known geometric and material properties. To verify a totem-protected image, we detect inconsistencies between the scene reconstructed from totem viewpoints and the scene's appearance from the camera viewpoint. Such an approach makes the adversarial manipulation task more difficult, as the adversary must modify both the totem and image pixels in a geometrically consistent manner without knowing the physical properties of the totem. Unlike prior learning-based approaches, our method does not require training on datasets of specific manipulations, and instead uses physical properties of the scene and camera to solve the forensics problem.

Related papers

• A Light Touch Approach to Teaching Transformers Multi-view Geometry [80.35521056416242]
  We propose a "light touch" approach to guiding visual Transformers to learn multiple-view geometry. We achieve this by using epipolar lines to guide the Transformer's cross-attention maps. Unlike previous methods, our proposal does not require any camera pose information at test-time.
  arXiv  Detail & Related papers  (2022-11-28T07:54:06Z)
• Physically Disentangled Representations [13.234029150635658]
  inverse rendering can be used to learn physically disentangled representations of scenes without supervision. We show the utility of inverse rendering in learning representations that yield improved accuracy on downstream clustering, linear classification, and segmentation tasks.
  arXiv  Detail & Related papers  (2022-04-11T17:36:40Z)
• Visiting the Invisible: Layer-by-Layer Completed Scene Decomposition [57.088328223220934]
  Existing scene understanding systems mainly focus on recognizing the visible parts of a scene, ignoring the intact appearance of physical objects in the real-world. In this work, we propose a higher-level scene understanding system to tackle both visible and invisible parts of objects and backgrounds in a given scene.
  arXiv  Detail & Related papers  (2021-04-12T11:37:23Z)
• Holistic 3D Scene Understanding from a Single Image with Implicit Representation [112.40630836979273]
  We present a new pipeline for holistic 3D scene understanding from a single image. We propose an image-based local structured implicit network to improve the object shape estimation. We also refine 3D object pose and scene layout via a novel implicit scene graph neural network.
  arXiv  Detail & Related papers  (2021-03-11T02:52:46Z)
• A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection [56.82077636126353]
  We take advantage of object-centric images to improve object detection in scene-centric images. We present a simple yet surprisingly effective framework to do so. Our approach can improve the object detection (and instance segmentation) accuracy of rare objects by 50% (and 33%) relatively.
  arXiv  Detail & Related papers  (2021-02-17T17:27:21Z)
• Face Forgery Detection by 3D Decomposition [72.22610063489248]
  We consider a face image as the production of the intervention of the underlying 3D geometry and the lighting environment. By disentangling the face image into 3D shape, common texture, identity texture, ambient light, and direct light, we find the devil lies in the direct light and the identity texture. We propose to utilize facial detail, which is the combination of direct light and identity texture, as the clue to detect the subtle forgery patterns.
  arXiv  Detail & Related papers  (2020-11-19T09:25:44Z)
• Cut-and-Paste Object Insertion by Enabling Deep Image Prior for Reshading [12.710258955529705]
  We show how to insert an object from one image to another and get realistic results in the hard case. We introduce a method that corrects shading inconsistencies of the inserted object without requiring a geometric and physical model.
  arXiv  Detail & Related papers  (2020-10-12T17:59:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.