DEPARTMENT OF COMPUTATIONAL AND DATA SCIENCES
Ph.D. Thesis Defense
=============================================================================
Speaker : Mr. Boya Sumanth Kumar
S.R. Number : 06-18-01-10-12-20-1-18435
Title : “Transformer Neural Operators for Learning Generalized Solutions of Partial Differential Equations and Data Assimilation”
Research Supervisor : Dr. Deepak Subramani
Thesis examiner : Prof. Anoop Krishnan, IIT Delhi
Date & Time : April 07, 2026 (Tuesday) at 11:00 AM
Venue : The Thesis Defense will be held on HYBRID Mode
# 303 CDS Class Room /MICROSOFT TEAMS.
Please click on the following link to join the Thesis Defense
MS Teams link
ABSTRACT
Scientific machine learning is transforming the way we solve physical dynamics governed by partial differential equations (PDEs). Solving nonlinear PDEs across multiple initial and boundary conditions requires re-running traditional numerical solvers, and existing physics-informed neural networks require costly retraining for each new condition. Furthermore, given the limited availability of observational data in meteorological and oceanic sciences, data assimilation in forward simulations is crucial. The above challenges are addressed by proposing three neural operators with architectural innovations: (1) the Physics-Informed Transformer Neural Operator (PINTO) for simulation-free PDE solving that is trained using only physics loss and generalizes to unseen initial and boundary conditions, (2) the Physics-Guided Transformer Neural Operator (PGNTO) for finding generalizable PDE solutions from simulation data in the absence of governing equations, and (3) the Attention based Coordinate Operator (ACO) for the data assimilation task of reconstructing continuous fields from limited observational data. Collectively, the above contributions push the boundaries of scientific machine learning in solving PDEs and data assimilation.
The first part of this thesis introduces a novel Physics-Informed Transformer Neural Operator (PINTO). Current neural operator approaches, though capable of learning functional mappings between infinite-dimensional spaces, suffer from two critical limitations: dependence on substantial simulation data and poor generalization to unseen conditions. PINTO addresses these fundamental challenges by introducing a novel physics-informed framework that achieves efficient generalization through simulation-free, physics-only training. The core innovation of PINTO lies in the development of iterative kernel-integral operator units that leverage cross-attention mechanisms to transform domain points into initial- and boundary-condition-aware representation vectors. This attention-based architecture enables context-aware learning that fundamentally differs from existing neural operators, allowing PINTO to learn mappings from input functions (initial/boundary conditions) to complete PDE solution spaces through a single forward pass. The architecture comprises three key stages: lifting layers that project the solution’s domain coordinates into a higher-dimensional representation space, iterative kernel integration layers for context-aware representation learning, and projection layers that map the learned representation to the solution space. We demonstrate PINTO’s superior performance across critical fluid mechanics and engineering applications, including linear advection, the nonlinear Burgers equation, and the steady and unsteady Navier-Stokes equations, spanning multiple flow scenarios. Under challenging unseen conditions, PINTO achieves relative errors of merely 20 to 33\% of those obtained by state-of-the-art physics-informed neural operator methods when compared with analytical and numerical solutions. Critically, PINTO exhibits temporal extrapolation capabilities absent in competing approaches, accurately solving the advection and Burgers equations in time steps not present during training.
In the second part of the thesis, we extend our framework to physics-guided transformer neural operators (PGNTO), trained on simulation data rather than a physics loss, to address scenarios where governing PDEs are unavailable and to eliminate the training instabilities that arise from physics loss in high-dimensional PDEs. Our PGNTO successfully solves standard PDE benchmarks, including the nonlinear Burgers and airfoil problem, and also scales to complex 3D turbulent flows, specifically predicting wake dynamics behind wind turbines under varying inlet velocities. Across all test cases, PGNTO maintains superior accuracy, with relative errors that are only one-third of those of competing neural operators.
Finally, in the third part of the thesis, we introduce the Attention-based coordinate operator (ACO) for neural data assimilation, enabling the continuous reconstruction of fields from sparse observational data. By applying Gabor filters for coordinate lifting and Fourier filters for value representation, ACO outperforms existing implicit neural representation networks on reconstruction tasks. Notably, a single ACO model generalizes across varying sparsity levels, eliminating the need for separate models tailored to different data availabilities. The performance of ACO is demonstrated using four challenging datasets: (i)sea surface height (SSH), (ii) chlorophyll concentration (CHL), (iii) Global surface temperature (GST), and (iv) sea surface temperature (SST). The ACO model is compared with other leading deep neural models for physical field reconstruction.
Overall, this thesis demonstrates that transformer-based neural operators can effectively address both physics-informed PDE solving and neural data assimilation, achieving unprecedented generalization, computational efficiency, and accuracy while requiring minimal training data. Our suite of transformer neural operators opens new avenues for real-time prediction and data assimilation in physics and engineering applications.
ALL ARE WELCOME
