Accepted

Transformer Meets Boundary Value Inverse Problems

Ruchi Guo, Shuhao Cao, and Long Chen

ICLR, 2023

arXiv   Bibtex coming soon

ABSTRACT:

 A Transformer-based deep direct sampling method is proposed
for a class of boundary value inverse problems. A real-time
reconstruction is achieved by evaluating the learned inverse operator
between carefully designed data and the reconstructed images. An
effort is made to give a specific example to a fundamental question:
whether and how one can benefit from the theoretical structure of a
mathematical problem to develop task-oriented and structure-conforming
deep neural networks? Specifically, inspired by direct sampling
methods for inverse problems, the 1D boundary data in different
frequencies are preprocessed by a partial differential equation-based
feature map to yield 2D harmonic extensions as different input
channels. Then, by introducing learnable non-local kernels, the direct
sampling is recast to a modified attention mechanism. The proposed
method is then applied to electrical impedance tomography, a
well-known severely ill-posed nonlinear inverse problem. The new
method achieves superior accuracy over its predecessors and
contemporary operator learners, as well as shows robustness with
respect to noise. This research shall strengthen the insights that the
attention mechanism, despite being invented for natural language
processing tasks, offers great flexibility to be modified in
conformity with the a priori mathematical knowledge, which ultimately
leads to the design of more physics-compatible neural architectures.