CARL: A Framework for Equivariant Image Registration

• URL: http://arxiv.org/abs/2405.16738v2
• Date: Tue, 28 May 2024 17:44:15 GMT
• Title: CARL: A Framework for Equivariant Image Registration
• Authors: Hastings Greer, Lin Tian, Francois-Xavier Vialard, Roland Kwitt, Raul San Jose Estepar, Marc Niethammer,
• Abstract summary: Image registration estimates spatial correspondences between a pair of images. Formally, the estimator should be equivariant to a desired class of image transformations. We show how to achieve multi-step $[W,U]$ equivariance via a coordinate-attention mechanism combined with displacement-predicting refinement layers.
• Score: 17.976933318883333
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image registration estimates spatial correspondences between a pair of images. These estimates are typically obtained via numerical optimization or regression by a deep network. A desirable property of such estimators is that a correspondence estimate (e.g., the true oracle correspondence) for an image pair is maintained under deformations of the input images. Formally, the estimator should be equivariant to a desired class of image transformations. In this work, we present careful analyses of the desired equivariance properties in the context of multi-step deep registration networks. Based on these analyses we 1) introduce the notions of $[U,U]$ equivariance (network equivariance to the same deformations of the input images) and $[W,U]$ equivariance (where input images can undergo different deformations); we 2) show that in a suitable multi-step registration setup it is sufficient for overall $[W,U]$ equivariance if the first step has $[W,U]$ equivariance and all others have $[U,U]$ equivariance; we 3) show that common displacement-predicting networks only exhibit $[U,U]$ equivariance to translations instead of the more powerful $[W,U]$ equivariance; and we 4) show how to achieve multi-step $[W,U]$ equivariance via a coordinate-attention mechanism combined with displacement-predicting refinement layers (CARL). Overall, our approach obtains excellent practical registration performance on several 3D medical image registration tasks and outperforms existing unsupervised approaches for the challenging problem of abdomen registration.

Related papers

• Relaxed Equivariance via Multitask Learning [7.905957228045955]
  We introduce REMUL, a training procedure for approximating equivariance with multitask learning. We show that unconstrained models can learn approximate symmetries by minimizing an additional simple equivariance loss. Our method achieves competitive performance compared to equivariant baselines while being $10 times$ faster at inference and $2.5 times$ at training.
  arXiv  Detail & Related papers  (2024-10-23T13:50:27Z)
• Towards Better Understanding of In-Context Learning Ability from In-Context Uncertainty Quantification [7.869708570399577]
  We consider a bi-objective prediction task of predicting both the conditional expectation $mathbbE[Y|X]$ and the conditional variance Var$(Y|X)$. Theoretically, we show that the trained Transformer reaches near Bayes-optimum, suggesting the usage of the information of the training distribution.
  arXiv  Detail & Related papers  (2024-05-24T00:08:55Z)
• Approximately Piecewise E(3) Equivariant Point Networks [30.619367345806438]
  We introduce APEN: a framework for constructing approximate piecewise-$E(3)$ equivariant point networks. Our primary insight is that functions that are equivariant with respect to a finer partition will also maintain equivariance in relation to the true partition. We demonstrate the effectiveness of APEN using two data types exemplifying part-based symmetry.
  arXiv  Detail & Related papers  (2024-02-13T15:34:39Z)
• Improving Equivariance in State-of-the-Art Supervised Depth and Normal Predictors [29.562054614079788]
  We find that state-of-the-art depth and normal predictors, despite having strong performances, surprisingly do not respect equivariance. To remedy this, we propose an equivariant regularization technique, consisting of an averaging procedure and a self-consistency loss. Our approach can be applied to both CNN and Transformer architectures, does not incur extra cost during testing, and notably improves the supervised and semi-supervised learning performance.
  arXiv  Detail & Related papers  (2023-09-28T17:51:05Z)
• Transformers as Support Vector Machines [54.642793677472724]
  We establish a formal equivalence between the optimization geometry of self-attention and a hard-margin SVM problem. We characterize the implicit bias of 1-layer transformers optimized with gradient descent. We believe these findings inspire the interpretation of transformers as a hierarchy of SVMs that separates and selects optimal tokens.
  arXiv  Detail & Related papers  (2023-08-31T17:57:50Z)
• Equivariant Similarity for Vision-Language Foundation Models [134.77524524140168]
  This study focuses on the multimodal similarity function that is not only the major training objective but also the core delivery to support downstream tasks. We propose EqSim, a regularization loss that can be efficiently calculated from any two matched training pairs. Compared to the existing evaluation sets, EqBen is the first to focus on "visual-minimal change"
  arXiv  Detail & Related papers  (2023-03-25T13:22:56Z)
• The Lie Derivative for Measuring Learned Equivariance [84.29366874540217]
  We study the equivariance properties of hundreds of pretrained models, spanning CNNs, transformers, and Mixer architectures. We find that many violations of equivariance can be linked to spatial aliasing in ubiquitous network layers, such as pointwise non-linearities. For example, transformers can be more equivariant than convolutional neural networks after training.
  arXiv  Detail & Related papers  (2022-10-06T15:20:55Z)
• Equivariance Discovery by Learned Parameter-Sharing [153.41877129746223]
  We study how to discover interpretable equivariances from data. Specifically, we formulate this discovery process as an optimization problem over a model's parameter-sharing schemes. Also, we theoretically analyze the method for Gaussian data and provide a bound on the mean squared gap between the studied discovery scheme and the oracle scheme.
  arXiv  Detail & Related papers  (2022-04-07T17:59:19Z)
• PDO-eConvs: Partial Differential Operator Based Equivariant Convolutions [71.60219086238254]
  We deal with the issue from the connection between convolutions and partial differential operators (PDOs) In implementation, we discretize the system using the numerical schemes of PDOs, deriving approximately equivariant convolutions (PDO-eConvs) Experiments on rotated MNIST and natural image classification show that PDO-eConvs perform competitively yet use parameters much more efficiently.
  arXiv  Detail & Related papers  (2020-07-20T18:57:26Z)
• Relative Pose Estimation of Calibrated Cameras with Known $\mathrm{SE}(3)$ Invariants [65.2314683780204]
  We present a complete study of the relative pose estimation problem for a camera constrained by known $mathrmSE(3)$ invariants. These problems reduces the minimal number of point pairs for relative pose estimation. Experiments on synthetic and real data shows performance improvement compared to conventional relative pose estimation methods.
  arXiv  Detail & Related papers  (2020-07-15T13:55:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.