Spline-PINN: Approaching PDEs without Datausing Fast, Physics-Informed Hermite-Spline CNNs

Nils Wandel,1 Michael Weinmann,2 Michael Neidlin,3 Reinhard Klein 1

1 University of Bonn2 Delft University of Technology

3 RWTH Aachen Universitywandeln@cs.uni-bonn.de, m.weinmann@tudelft.nl, neidlin@ame.rwth-aachen.de, rk@cs.uni-bonn.de

Abstract

Partial Differential Equations (PDEs) are notoriously difficultto solve. In general, closed-form solutions are not availableand numerical approximation schemes are computationallyexpensive. In this paper, we propose to approach the solu-tion of PDEs based on a novel technique that combines theadvantages of two recently emerging machine learning basedapproaches. First, physics-informed neural networks (PINNs)learn continuous solutions of PDEs and can be trained withlittle to no ground truth data. However, PINNs do not gener-alize well to unseen domains. Second, convolutional neuralnetworks provide fast inference and generalize but either re-quire large amounts of training data or a physics-constrainedloss based on finite differences that can lead to inaccuraciesand discretization artifacts.We leverage the advantages of both of these approaches byusing Hermite spline kernels in order to continuously inter-polate a grid-based state representation that can be handledby a CNN. This allows for training without any precomputedtraining data using a physics-informed loss function only andprovides fast, continuous solutions that generalize to unseendomains. We demonstrate the potential of our method at theexamples of the incompressible Navier-Stokes equation andthe damped wave equation. Our models are able to learn sev-eral intriguing phenomena such as Karman vortex streets,the Magnus effect, Doppler effect, interference patterns andwave reflections. Our quantitative assessment and an inter-active real-time demo show that we are narrowing the gapin accuracy of unsupervised ML based methods to industrialsolvers for computational fluid dynamics (CFD) while beingorders of magnitude faster.

IntroductionPartial differential equations (PDEs) are an important math-ematical concept to describe for example the motion of flu-ids, the propagation of waves, the evolution of stock mar-kets, gravity and more. However, solving partial differentialequations is a hard problem since closed-form solutions arerarely available. Thus, developing fast and accurate numeri-cal schemes in order to find approximate solutions is of greatinterest for applications such as e.g. physics engines in com-puter games, computer generated imagery (CGI) for movies,

Copyright © 2022, Association for the Advancement of ArtificialIntelligence (www.aaai.org). All rights reserved.

or computational fluid dynamics (CFD) to help engineerswith simulated wind tunnel experiments.

Recently, advances of machine learning (ML) based ap-proaches have led to promising results coping with thehigh computational costs associated with classical numeri-cal methods. Furthermore, some ML based approaches areby design differentiable. This means that they offer gradi-ents that can be used for stability analysis, optimal controlor reinforcement learning. However, ML based approachesoften do not generalize to domains not seen during training[17, 22, 35, 38, 25] or rely on large amounts of training datawhich capture the variations to be expected in foreseen sce-narios [24, 16, 18].Recent physics-constrained approachesbased on finite differences [36, 45, 40, 41] exhibit the poten-tial to mitigate these problems but might lead to inaccura-cies and discretization artifacts, especially at high Reynolds-Numbers.

Here, we propose to use a Hermite spline CNN to ob-tain continuous solutions that can be trained with a physics-informed loss only. This approach combines the advantagesof i) leveraging physics-informed neural networks (PINNs)to overcome the need for large amounts of training data[14, 26, 25] and ii) leveraging the faster inference and bet-ter generalization capabilities offered by convolutional neu-ral networks [36, 40, 41]. We demonstrate the effectivenessof our approach for the incompressible Navier-Stokes equa-tions as well as the damped wave equation. The incompress-ible Navier-Stokes equations, which are particularly hardto solve due to the non-linear advection term and thus themain focus of this paper, are investigated for Reynolds num-bers ranging from 2-10000. To assess the accuracy of ourmethod, we compute drag and lift coefficients on a CFDbenchmark domain [1] and compare the results with officialbenchmark values. For both equations, we perform gener-alization experiments, evaluate the stability over long timehorizons and present an interactive real-time demonstration.To ensure full reproducibility, our code is publicly availableon github: https://github.com/aschethor/Spline PINN.

Related WorkRecent developments indicate the potential of efficient sur-rogate models based on machine learning, and in particulardeep learning, to approximate the dynamics of partial differ-ential equations (PDEs).

PRELIMINARY PREPRINT VERSION: DO NOT CITEThe AAAI Digital Library will contain the published

version some time after the conference.

Lagrangian methods like smoothed particle hydrody-namics (SPH) [9] represent fluids in the reference frameof individual particles that follow the fluid’s velocity field.Relying on this principle, learning-based Lagrangian ap-proaches include the simulation of fluids based on regres-sion forests [18], graph neural networks [23, 19], continu-ous convolutions [39] and Smooth Particle Networks (SP-Nets) [29]. While Lagrangian methods are particularly suit-able for fluid domains that exhibit large, dynamic surfacessuch as waves and droplets, however, Eulerian methods typ-ically allow a more accurate simulation of fluid dynamicswithin a certain fluid domain (see [5]).

Eulerian methods model domain characteristics such asa velocity or pressure field on a fixed reference frame. Re-spective techniques leverage implicit neural representations,grid- and mesh-structures to describe the domain of a PDE.

Continuous Eulerian methods exploit the direct mappingof domain coordinates (i.e. positional coordinates and time)to field values (i.e. velocity, pressure, etc.) relying e.g. onimplicit neural representations to obtain smooth and ac-curate simulations [31, 10, 15, 25, 21] and handle thecurse of dimensionality faced by discrete techniques in thecase of high-dimensional PDEs [10]. Respective applica-tions include the modeling of flow through porous me-dia [44, 45, 37], fluid modeling [43, 26], turbulence mod-eling [7, 20], wave propagation [27, 33] and the modelingof molecular dynamics [30]. Training typically relies on pe-nalizing residuals of the underlying PDEs based on physics-informed loss functions for a specific domain, which pre-vents the networks from generalizing to new domain geome-tries and being used in the scope of interactive scenarios. Re-cently, Wang et al. [42] proposed to stitch pretrained DeepLearning models together to speed up the learning processfor boundary value problems. However, temporal dependen-cies are not embedded in their method yet preventing inter-active applications.

Discrete Eulerian methods such as finite difference, lat-tice Boltzmann [3, 11] or finite element methods insteadtackle the underlying PDEs on an explicit representationsuch as a discrete grid or a graph structure. Beyond earlyseminal work [12, 34], recent developments focus on ex-ploiting the potential of deep learning techniques to achievesignificant speed-ups while preserving accuracy. Learningfluid simulations has been approached with deep generativemodels for efficient interpolations between various simula-tion settings [16] and turbulent flow fields within pipe do-mains [17]. However, both of these approaches do not gener-alize to new domain geometries that have not been used dur-ing training. To speed up Eulerian fluid simulations, Tomp-son et al. [36] proposed to learn a Helmholtz projectionstep with a CNN that generalizes to new domain geometriesthat have not been used during the training. However, thisapproach requires a path tracer to deal with the advectionterm of the Navier-Stokes equations. Furthermore, as viscos-ity is not considered in Eulerian fluid models, phenomenasuch as the Magnus effect and Karman vortex streets can-not be simulated. Ensuring incompressibility of fluids hasbeen achieved with discretized vector potentials [16, 22],

however, these techniques are not capable of generalizingto domain geometries not seen during the training. The ap-proach by Geneva et al. [8] discards the pressure term in theNavier-Stokes equations which leads to the simpler Burgers’equations, for which they learned the update step based on aphysics-constrained framework. Thuerey et al. [35] learn tosolve the Reynolds-averaged Navier-Stokes equations, butthe specific approach prevents a generalization beyond air-foil flows and discards the temporal evolution of the fluidstate. Um et al. [38] proposed to learn a correction stepwhich allows approximating solutions of a high-resolutionfluid simulations in terms of a low-resolution differentiablefluid solver, but the generalization capability to domain ge-ometries that have not been used during training has notbeen shown. In contrast to the aforementioned techniques,the approach by Wandel et al. [40] and its 3D extension [41]overcome the dependence on huge amounts of training databy using a physics-constrained loss and introducing a train-ing cycle that recycles data generated by the network dur-ing training. This allows handling dynamic boundary condi-tions as required for interactions with the fluid [40, 41] and,with an improved network [41], also dealing with temporallyvarying fluid parameters such as viscosity and density dur-ing simulation. However, their physics-constrained loss canlead to discretization artifacts and requires a pressure gradi-ent regularizer for high Reynolds numbers. Further learning-based approaches also exploit graph representations in termsof graph neural networks [13, 28], graph convolutional net-works [6] as well as mesh representations [24] or sub-space representations [32, 2]. Unfortunately, graph neuralnetworks cannot make use of the highly efficient implemen-tations for convolutional operations on grids and thus usu-ally come with higher computational complexity per nodecompared to grid based CNNs.

To the best of our knowledge, the potential of SplineCNNs [4] has not yet been investigated for learning the dy-namics of PDEs and, hence, the presented physics-informedHermite Spline CNN (Spline-PINN) approach is the firstmethod that leverages both implicit and explicit characteris-tics to obtain fast, interactive, continuous surrogate modelsthat generalize and can be trained without training data usinga physics-informed loss based on the underlying PDEs.

MethodIn the main part of our paper, we first briefly discuss thebackground regarding partial differential equations with afocus on the Navier-Stokes equations and the damped waveequation, before we introduce our physics-informed Her-mite spline CNN approach (Spline-PINN), that combinesthe advantages of PINNs regarding physics-informed train-ing without training data and Spline-CNNs regarding theirgeneralization capability.

Partial Differential EquationsPartial Differential Equations describe the dependencies be-tween partial derivatives of a multivariate function inside adomain Ω and usually need to be solved for given initialconditions at the beginning of the simulation and boundaryconditions at the domain boundaries ∂Ω.

The Incompressible Navier-Stokes Equations describethe dynamics of a fluid with a pressure-field p and a velocity-field v inside a domain Ω by means of the incompressibilityand momentum equation:

∇ · v = 0 in Ω (1)

ρ ˙v = ρ(∂tv + v · ∇v) = µ∆v −∇p+ f in Ω (2)

Here, ρ represents the fluid’s density and µ its viscosity. Theexternal forces f are neglected in our experiments. Thesetwo equations have to be solved for given initial conditionsv0, p0 and boundary conditions (BCs). Here, we considerthe Dirichlet BC to set the velocity field v = vd at the do-main boundaries ∂Ω. In this work, we exploit the Helmholtzdecomposition theorem and use a vector potential a withv = ∇ × a to automatically ensure incompressibility. Fur-thermore, we restrict our considerations to 2D flows and,hence, only the z-component of a, az , is needed. The Navier-Stokes equations are considered particularly hard to solvedue to the non-linear advection term (v · ∇v) and, therefore,are the main focus of our investigations.

The Damped Wave Equation can be used to describe forexample the dynamics of a thin membrane with height-fieldz and velocity-field vz .

∂tz = vz in Ω (3)∂tvz = k∆z − δvz in Ω (4)

Here, k is the stiffness constant of the membrane and δ isa damping constant. As for the Navier-Stokes equations, wesolve these equations for given initial conditions z0, vz0 andDirichlet boundary conditions: z = zd on ∂Ω.

Although the damped wave equation contains only linearcomponents, we present this additional example to demon-strate that our method might also work for different classesof PDEs. The wave equation lays the foundation for manymore complex equations such as the electromagnetic waveequation or the Schrodinger equation. Furthermore, for highdamping constants δ, z converges towards the solution of theLaplace equation.

Spline-PINNIn this section, we introduce our Physics-Informed HermiteSpline CNN. First, we provide an overview on Hermitesplines and then show how to incorporate them into a CNNfor solving PDEs.

Hermite Splines In general, Hermite splines are piece-wise polynomials that are defined by their values and theirfirst n derivatives at given support points. To facilitate CNN-based processing, we arrange the support points on a regu-lar grid. Figure 1 shows an example of Hermite spline ker-nel functions in 1D for n = 0, 1, 2. We define the kernelfunctions hn

i (x), such that the values and first n derivativesat the support points (x = −1, 0, 1) are set to 0 exceptthe ith derivative at x = 0 which takes a value such thathni (x) ∈ [−1, 1]. In contrast to B-Spline kernels, the sup-

port of Hermite spline kernels ranges only over the directly

neighboring grid cells. This facilitates learning of high fre-quencies and allows for computationally more efficient in-terpolations with comparatively small transposed convolu-tion kernels. By linearly combining Hermite spline kernelfunctions of order n at every grid-cell we obtain continu-ous piecewise polynomials with (n + 1)-th derivatives ofbounded variation. Choosing the right spline order n is im-portant to be able to compute the physics informed loss fora given PDE as will be discussed below.

To obtain kernel functions in multiple dimensions, we usethe tensor product of multiple 1D kernel functions:

hl,m,ni,j,k (x, y, t) = hl

i(x)hmj (y)hn

k (t) (5)

In the supplementary, we show how basis flow fields can beobtained by taking the curl of hl,m

i,j (x, y). On a grid x, y, t ∈X× Y × T with discrete spline coefficients ci,j,k

x,y,t, we obtain

a continuous Hermite spline g(x, y, t) as follows:

g(x, y, t) =∑

i,j,k∈[0:l]×[0:m]×[0:n]

x,y,t∈X×Y×T

ci,j,kx,y,t

hl,m,ni,j,k (x−x, y−y, t−t)

(6)Our goal is, to find spline coefficients ci,j,k

x,y,tsuch that

g(x, y, t) matches the solution of PDEs as closely as pos-sible. The partial derivatives of g with respect to x, y, t canbe directly obtained by taking the corresponding derivativesof the spline kernel functions.

Pipeline Figure 2 depicts the pipeline of our Spline-PINN.We use a CNN (PDE Model) to map discrete Hermite-splinecoefficients and boundary conditions from a timepoint t tospline coefficients at a succeeding timepoint t + dt. By re-currently applying the PDE Model on the spline coefficients,the simulation can be unrolled in time. From these discretespline-coefficients, continuous Hermite splines can be effi-ciently obtained using transposed convolutions (see Equa-tion 6) with interpolation kernels as depicted in Figure 1.This way, training samples as well as evaluation samples canbe taken at any point in space and time.

Physics Informed Loss When training neural networks onPDEs, the information provided by the PDEs themselves canbe used to save or even completely spare out training-data.In literature, two major approaches have been established:

Physics-constrained approaches [45] compute losses ex-plicitly - usually on a grid topology using finite differencesin order to approximate the derivatives of a PDE. This ap-proach is suitable to train CNNs and allows neural net-works to learn e.g. the incompressible Navier-Stokes equa-tions without any ground truth training-data and generalizeto new domains [40]. However, relying on finite differencesmay lead to inaccuracies and strong discretization artifacts -especially at high Reynolds-numbers (see Figure 3).

In contrast, physics-informed approaches [25] computelosses implicitly - usually by taking derivatives of an im-plicit field description. This approach enables efficient train-ing of implicit neural networks and yields continuous solu-tions that can be evaluated at any point in space and time.

Figure 1: 1D Hermite spline kernels for n = 0, 1, 2 (scaled between -1 and 1). Note, that these kernel functions are in Cn andthus, the (n+ 1)-th derivatives are of bounded variation. The case n = 0 could be considered as a linear interpolation betweenspline coefficients.

Figure 2: Pipeline of PDE Model with Hermite spline inter-polation. Since the solution is continuous, training samplesand evaluation samples can be obtained at any point in spaceand time. More detailed views of the PDE Models used forthe Navier-Stokes equations and damped wave equation areprovided in the supplementary material.

However, implicit neural networks do not generalize well tonovel domains but usually require retraining of the network.

Here, we can combine the advantages of both approaches:By using a convolutional neural network that processesspline coefficients which can be considered as a discrete hid-den latent description for a continuous implicit field descrip-tion based on Hermite splines, our Spline-PINN approach iscapable to generalize to new domain geometries and yieldscontinuous solutions that avoid the detrimental effects of adiscretized loss function based on finite differences.

We aim to optimize the spline coefficients such that theintegrals of the squared residuals of the PDEs over the do-main / domain-boundaries and time steps are minimized.To compute these integrals, we uniform randomly samplepoints within the given integration domains.

For the Navier-Stokes equation, we consider a momentumloss term Lp defined as:

Lp =

∫Ω

∫ t+dt

t

||ρ(∂tv + v · ∇v)− µ∆v +∇p||2 (7)

It is important that the residuals are of bounded variation,otherwise we can not compute the integral. Lp containsthird order derivatives for az in space (see viscosity term:µ∆(∇× az)) and thus, following the argument in Figure 1,we have to choose at least l,m = 2 for the Hermite spline

kernels of az . The boundary loss term Lb is given by:

Lb =

∫∂Ω

∫ t+dt

t

||v − vd||2 (8)

Lp and Lb are then combined in the final loss term Lflowtot

according to:Lflow

tot = αLp + βLb (9)Here, we set hyperparameters α = 10 and β = 20. β waschosen higher compared to α to prevent the flow field fromleaking through boundaries. A more thorough discussion re-garding the choice of hyperparameters is provided in thesupplementary material.

For the damped wave equation, we need two loss termsinside the domain Ω:

Lz =

∫Ω

∫ t+dt

t

||ρ(∂tz − vz||2 (10)

Lv =

∫Ω

∫ t+dt

t

||ρ(vz − k∆z + δvz||2 (11)

Similar to the incompressible Navier-Stokes equation, wehave second order derivatives of z in space (see k∆z-Term)and thus have to choose at least l,m = 1 for the Hermitespline kernels of z. The boundary loss term is:

Lb =

∫∂Ω

∫ t+dt

t

||z − zd||2 (12)

Lz, Lv and Lb are then combined in the final loss term Lwavetot :

Lwavetot = αLz + βLv + γLb (13)

For the wave equation, we set the hyperparameters α =1, β = 0.1, γ = 10.

Training Procedure As presented in Figure 4, we use atraining procedure similar to [40] but replace the physics-constrained loss based on finite differences by the just intro-duced physics-informed loss on the spline coefficients. First,we initialize a training pool of randomized training domainsand spline coefficients. At the beginning, all spline coeffi-cients can be set to 0, thus no ground truth data is required.Then, we draw a random minibatch (batchsize = 50) fromthe training pool and feed it into the PDE Model, which pre-dicts the spline coefficients for the next time step. Then, we

Figure 3: Severe discretization artifacts appear using a physics-constrained loss based on a finite differences Marker and Cellgrid at Re = 10000 - even at twice the resolution compared to our Hermite spline approach.

compute a physics-informed loss inside the volume spannedby the spline coefficients of the minibatch and the predictedspline coefficients in order to optimize the weights of thePDE Model with gradient descent. To this end, we use theAdam optimizer (learning rate = 0.0001). Finally, we updatethe training pool with the just predicted spline coefficientsin order to fill the pool with more and more realistic trainingdata over the course of training. From time to time, we resetspline coefficients to 0 in the training pool in order to alsolearn the warm-up phase from 0 spline coefficients. Trainingtook 1-2 days on a NVidia GeForce RTX 2080 Ti.

Figure 4: Training cycle similar to [40]

ResultsIn the following, we provide qualitative and quantitativeevaluations of our approach for fluid and wave simulations.Additional results as well as demonstrations where the usercan interact with the PDE by dynamically changing bound-ary conditions, are contained in our supplemental.

Fluid SimulationQualitative Evaluation The dynamics of a fluid dependsstrongly on the Reynolds number, which is a dimensionlessquantity that relates the fluid density ρ, mean velocity ||v||,obstacle diameter L and viscosity µ in the following way:

Re =ρ||v||L

µ(14)

To validate our method, we reconstructed the DFG bench-mark setup [1] and scaled its size and viscosity by a factorof 100. This way, we obtained a grid size of 41 × 220 cellsfor the fluid domain and the obstacle diameter is 10 cells.

For very small Reynolds numbers, the flow field becomesbasically time-reversible which can be recognized by the

symmetry of the stream lines in Figure 5 a). At Re = 20a steady wake is forming behind the obstacle (see Figure 5b). For higher Reynolds numbers such as Re = 100, thiswake becomes unstable and a von Karman vortex street isforming (see Figure 5 c). Figure 5 d) and e) show turbulentflow fields at very high Reynolds numbers (Re = 1000 andRe = 10000). Notice the high level of details in the velocityfield learned by our method without any ground truth data.

Quantitative Evaluation To evaluate our method quanti-tatively, we computed the forces exerted by the fluid on acylinder (see Figure 6) and compared the resulting drag andlift coefficients to an implicit physics-informed neural net-work similar to [26], a MAC grid based physics-constrainedneural network [40], an industrial CFD solver (Ansys) andofficial results reported on a CFD benchmark [1]. To thebest of our knowledge, this is the first time such forces wereobtained by a fully unsupervised machine learning based ap-proach. The forces can be separated into two terms stem-ming from viscous friction (Fµ) and the pressure field (Fp):

Fµ =

∫S

µ(∇v)nds ; Fp =

∫S

−pnds (15)

Ftot = Fµ + Fp (16)

Figure 6 shows the distribution of such viscous drag andpressure forces along the cylinder surface S with surfacenormals n.

The drag-force FD is the parallel force-component of Ftot

to the flow direction (FD = Ftot,x) while the lift force FL isits orthogonal component (FL = Ftot,y). From these forces,it is possible to compute drag and lift coefficients as follows:

CD =2FD

U2meanL

, CL =2FL

U2meanL

(17)

Table 1 compares the drag and lift coefficients obtainedby our method to an implicit PINN, a finite-difference basedMAC grid approach [40], an industrial CFD solver (An-sys) and the official DFG benchmark [1] for Reynolds num-bers 2, 20 and 100. For all Reynolds numbers, the Spline-PINN approach returned significantly improved drag coeffi-cients compared to the implicit PINN and MAC-approach.For Re = 100, the implicit PINN fails to capture the dy-namics of the von Karman vortex street as it is only trainedon the domain boundaries without any additional informa-tion stemming for example from a moving die [26] or dy-namic boundary conditions [14]. In contrast, our method isable to reproduce such oscillations as can be seen in Fig-ure 7. The implicit PINN required retraining for every set-ting which took about 1 day while performing the compu-

(a) time-reversible flow(Re = 2, µ = 1, ρ = 1)

(b) laminar flow (Re = 20, µ = 0.1, ρ = 1)

(c) laminar vortex street(Re = 100, µ = 0.1, ρ = 1)

(d) turbulent flow(Re = 1000, µ = 0.1, ρ = 10)

(e) turbulent flow (Re = 10000, µ = 0.01, ρ = 10)

Figure 5: Flow and pressure fields around a cylinder obtained by our method at different Reynolds numbers. Left side: velocitymagnitude; Right side: pressure field and stream lines of velocity field. An animated real-time visualization of these experimentsis provided in the supplementary video.

Figure 6: White arrows indicate viscous and pressure forces(see Equation 15) acting on a cylinder at Re = 100 (seeFigure 5 c). While forces from viscous friction are parallelto the obstacle’s surface, pressure forces are always perpen-dicular to the surface.

tations with the MAC grid based approach or our Spline-PINN was achieved basically in real-time at 60 and 30 iter-ations per second respectively. The Ansys solver took 9 secfor Re = 2, 13 sec for Re = 20 and 37 min for Re = 100.A comparison of errors in E[|∇ · v|] and Lp as well as astability analysis can be found in the supplemental.

Figure 7: Oscillating drag and lift coefficients over time ob-tained by our Spline-PINN at Re = 100.

Wave SimulationQualitative Evaluation Figure 8 shows several experi-mental results that were obtained by our method for thedamped wave equation on a 200 × 200 domain. First, we in-vestigated interference patterns that arise when waves fromdifferent directions are superimposed. In Figure 8 a), thewaves of 4 oscillators interfere with each other. Then, in Fig-ure 8 b), we investigate whether our method is able to learnthe Doppler effect for moving oscillators. This effect is well-

Re=2 Re=20 Re=100Method CD CL CD CL CD CL

implicit PINN 25.3 0.478 3.299 0.0744 1.853 -0.02445MAC grid [40] 25.76 -0.824 4.414 -0.597 (2.655 / 2.693 / 2.725)* (-0.757 / 0.0184 / 0.86)*

Spline-PINN (ours) 29.7 -0.456 4.7 5.64e-04 (2.985 / 3.068 / 3.188)* (-0.926 / 0.179 / 1.295)*Ansys 32.035 0.774 5.57020 0.00786 (3.234 / 3.273 / 3.31)* (-1.14 / -0.059 / 1.07)*

DFG-Benchmark [1] - - 5.58 0.0106 (3.1569 / 3.1884 / 3.220 )* (-1.0206 / -0.0173 / 0.9859)*

Table 1: Drag and lift coefficients obtained by an implicit PINN, our Hermite spline approach, an industrial CFD solver (Ansys)and official results from the DFG-benchmark [1]. *:(minimum/average/maximum)-values for oscillating coefficients

known e.g. for changing the pitch of an ambulance-car thatdrives by. Finally, in Figure 8 c), we show the reflection andinterference behavior of waves hitting Dirichlet boundaries.

(a) Interference patterns forming around 4 oscillators.

(b) Doppler effect of an oscillator moving to the right.

(c) Wave reflections on domain boundaries.

Figure 8: Results of our method for the wave equation (k =10, δ = 0.1). Left side: height field z; Right side: velocityfield vz . The domain boundaries are marked in black. Forbetter performance, second-order splines were used in x andy. An animated real-time visualization of these experimentscan be found in the supplementary video.

As for the fluid simulations, all of these results were ob-tained without relying on any ground truth data. Further-more, the domain in Figure 8 c) was not contained in therandomized set of training domains indicating good gener-alization performance.

Quantitative Evaluation Table 2 compares the losses ofour method on the oscillator domain (see Figure 8 a). Wetrained two versions with two different spline orders in thespatial dimensions (l,m = 1 and l,m = 2) and observedsignificantly better performance for l,m = 2. The stabilityof our approach for the wave equation is examined in thesupplementary material.

Spline order Lz Lv Lb

l,m = 1 8.511e-02 1.127e-02 1.425e-03l,m = 2 5.294e-02 6.756e-03 1.356e-03

Table 2: Quantitative results of wave equation.

ConclusionIn this work, we approach the incompressible Navier-Stokesequation and the damped wave equation by training a con-tinuous Hermite Spline CNN using physics-informed lossfunctions only. While finite-difference based methods breakdown at high Reynolds-numbers due to discretization arti-facts, our method still returns visually appealing results atRe = 10000. Furthermore, we investigated drag and liftcoefficients on a CFD benchmark setup and observed rea-sonable accordance with officially reported values which isremarkable given the fact that our method does not rely onany ground truth data.

In the future, further boundary conditions (e.g. von Neu-mann BC) could be incorporated into the PDE model aswell. To further refine the solutions at the boundary layers,a multigrid Hermite spline CNN could be considered. Thefully differentiable nature of our method may also help inreinforcement learning scenarios, optimal control, sensitiv-ity analysis or gradient based shape optimization.

We believe that our method could have applicationsin physics engines of computer games or in computer-generated imagery as it provides fast and visually plausiblesolutions. The obtained drag and lift coefficients indicatethat in the future, unsupervised ML based methods couldreach levels of accuracy that are sufficiently close to tradi-tional industrial CFD solvers making them suitable for fastprototyping in engineering applications. We firmly believethat moving from explicit physics-constrained losses to im-plicit physics-informed losses on continuous fields based ondiscrete latent descriptions such as spline coefficients willpositively influence the performance of future ML basedPDE solvers that generalize.

References[1] 2021. The CFD Benchmarking Project.

http://www.mathematik.tu-dortmund.de/ feat-flow/en/benchmarks/cfdbenchmarking.html. (accessedat 08-Sep-2021).

[2] Ainsworth, M.; and Dong, J. 2021. Galerkin Neu-ral Networks: A Framework for Approximating Vari-ational Equations with Error Control. arXiv preprintarXiv:2105.14094.

[3] Chen, S.; and Doolen, G. D. 1998. Lattice Boltzmannmethod for fluid flows. Annual review of fluid mechan-ics, 30(1): 329–364.

[4] Fey, M.; Lenssen, J. E.; Weichert, F.; and Muller, H.2017. SplineCNN: Fast Geometric Deep Learning withContinuous B-Spline Kernels. CoRR, abs/1711.08920.

[5] Foster, N.; and Metaxas, D. 1996. Realistic Animationof Liquids. Graphical Models and Image Processing,58(5): 471 – 483.

[6] Gao, H.; Zahr, M. J.; and Wang, J. 2021. Physics-informed graph neural Galerkin networks: A unifiedframework for solving PDE-governed forward and in-verse problems. CoRR, abs/2107.12146.

[7] Geneva, N.; and Zabaras, N. 2019. Quantifyingmodel form uncertainty in Reynolds-averaged turbu-lence models with Bayesian deep neural networks.Journal of Computational Physics, 383: 125 – 147.

[8] Geneva, N.; and Zabaras, N. 2020. Modeling the dy-namics of PDE systems with physics-constrained deepauto-regressive networks. Journal of ComputationalPhysics, 403: 109056.

[9] Gingold, R. A.; and Monaghan, J. J. 1977. Smoothedparticle hydrodynamics: theory and application to non-spherical stars. Monthly notices of the royal astronom-ical society, 181(3): 375–389.

[10] Grohs, P.; Hornung, F.; Jentzen, A.; and Von Wurstem-berger, P. 2018. A proof that artificial neural networksovercome the curse of dimensionality in the numeri-cal approximation of Black-Scholes partial differentialequations. arXiv preprint arXiv:1809.02362.

[11] Guo, Z. 2021. Well-balanced lattice Boltzmann modelfor two-phase systems. Physics of Fluids, 33(3):031709.

[12] Harlow, F. H.; and Welch, J. E. 1965. Numerical calcu-lation of time-dependent viscous incompressible flowof fluid with free surface. The physics of fluids, 8(12):2182–2189.

[13] Harsch, L.; and Riedelbauch, S. 2021. Direct Predic-tion of Steady-State Flow Fields in Meshed Domainwith Graph Networks. arXiv:2105.02575.

[14] Jin, X.; Cai, S.; Li, H.; and Karniadakis, G. E. 2021.NSFnets (Navier-Stokes flow nets): Physics-informedneural networks for the incompressible Navier-Stokesequations. Journal of Computational Physics, 426:109951.

[15] Khoo, Y.; Lu, J.; and Ying, L. 2019. Solving for high-dimensional committor functions using artificial neu-ral networks. Research in the Mathematical Sciences,6(1): 1.

[16] Kim, B.; Azevedo, V. C.; Thuerey, N.; Kim, T.; Gross,M.; and Solenthaler, B. 2019. Deep fluids: A gener-ative network for parameterized fluid simulations. InComputer Graphics Forum, volume 38, 59–70. WileyOnline Library.

[17] Kim, J.; and Lee, C. 2020. Deep unsupervised learningof turbulence for inflow generation at various Reynoldsnumbers. Journal of Computational Physics, 406:109216.

[18] Ladicky, L.; Jeong, S.; Solenthaler, B.; Pollefeys, M.;and Gross, M. 2015. Data-Driven Fluid SimulationsUsing Regression Forests. ACM Trans. Graph., 34(6).

[19] Li, Y.; Wu, J.; Tedrake, R.; Tenenbaum, J. B.; and Tor-ralba, A. 2019. Learning Particle Dynamics for Manip-ulating Rigid Bodies, Deformable Objects, and Fluids.In ICLR.

[20] Ling, J.; Kurzawski, A.; and Templeton, J. 2016.Reynolds averaged turbulence modelling using deepneural networks with embedded invariance. Journalof Fluid Mechanics, 807: 155–166.

[21] Lu, L.; Meng, X.; Mao, Z.; and Karniadakis, G. E.2021. DeepXDE: A Deep Learning Library for Solv-ing Differential Equations. SIAM Review, 63(1):208–228.

[22] Mohan, A. T.; Lubbers, N.; Livescu, D.; and Chertkov,M. 2020. Embedding Hard Physical Constraints inNeural Network Coarse-Graining of 3D Turbulence.arXiv:2002.00021.

[23] Mrowca, D.; Zhuang, C.; Wang, E.; Haber, N.; Fei-Fei, L.; Tenenbaum, J. B.; and Yamins, D. L. K. 2018.Flexible Neural Representation for Physics Prediction.In Proceedings of the 32nd International Conferenceon Neural Information Processing Systems, NIPS’18,8813–8824. Red Hook, NY, USA: Curran AssociatesInc.

[24] Pfaff, T.; Fortunato, M.; Sanchez-Gonzalez, A.; andBattaglia, P. W. 2021. Learning Mesh-Based Simula-tion with Graph Networks. ICLR 2021:2010.03409.

[25] Raissi, M.; Perdikaris, P.; and Karniadakis, G. E. 2019.Physics-informed neural networks: A deep learningframework for solving forward and inverse problemsinvolving nonlinear partial differential equations. Jour-nal of Computational Physics, 378: 686 – 707.

[26] Raissi, M.; Yazdani, A.; and Karniadakis, G. E. 2018.Hidden Fluid Mechanics: A Navier-Stokes InformedDeep Learning Framework for Assimilating Flow Vi-sualization Data. arXiv preprint arXiv:1808.04327.

[27] Rasht-Behesht, M.; Huber, C.; Shukla, K.; and Karni-adakis, G. E. 2021. Physics-informed Neural Networks(PINNs) for Wave Propagation and Full Waveform In-versions. arXiv:2108.12035.

[28] Sanchez-Gonzalez, A.; Godwin, J.; Pfaff, T.; Ying, R.;Leskovec, J.; and Battaglia, P. W. 2020. Learningto Simulate Complex Physics with Graph Networks.arXiv:2002.09405.

[29] Schenck, C.; and Fox, D. 2018. SPNets: DifferentiableFluid Dynamics for Deep Neural Networks. In Con-ference on Robot Learning, 317–335.

[30] Schoberl, M.; Zabaras, N.; and Koutsourelakis, P.-S.2019. Predictive collective variable discovery withdeep Bayesian models. The Journal of ChemicalPhysics, 150(2): 024109.

[31] Sirignano, J.; and Spiliopoulos, K. 2018. DGM: Adeep learning algorithm for solving partial differen-tial equations. Journal of Computational Physics, 375:1339 – 1364.

[32] Sirignano, J.; and Spiliopoulos, K. 2018. DGM: Adeep learning algorithm for solving partial differen-tial equations. Journal of computational physics, 375:1339–1364.

[33] Sitzmann, V.; Martel, J.; Bergman, A.; Lindell, D.; andWetzstein, G. 2020. Implicit neural representationswith periodic activation functions. Advances in Neu-ral Information Processing Systems, 33.

[34] Stam, J. 1999. Stable fluids. In Proceedings of the 26thannual conference on Computer graphics and interac-tive techniques, 121–128.

[35] Thuerey, N.; Weißenow, K.; Prantl, L.; and Hu, X.2019. Deep Learning Methods for Reynolds-AveragedNavier–Stokes Simulations of Airfoil Flows. AIAAJournal, 1–12.

[36] Tompson, J.; Schlachter, K.; Sprechmann, P.; and Per-lin, K. 2017. Accelerating eulerian fluid simulationwith convolutional networks. In Proceedings of the34th International Conference on Machine Learning-Volume 70, 3424–3433. JMLR. org.

[37] Tripathy, R. K.; and Bilionis, I. 2018. Deep UQ: Learn-ing deep neural network surrogate models for high di-mensional uncertainty quantification. Journal of Com-putational Physics, 375: 565 – 588.

[38] Um, K.; Fei, R.; Holl, P.; Brand, R.; and Thuerey, N.2020. Solver-in-the-Loop: Learning from Differen-tiable Physics to Interact with Iterative PDE-Solvers.arXiv:2007.00016.

[39] Ummenhofer, B.; Prantl, L.; Thuerey, N.; and Koltun,V. 2020. Lagrangian Fluid Simulation with Continu-ous Convolutions. In 8th International Conference onLearning Representations, ICLR 2020, Addis Ababa,Ethiopia, April 26-30, 2020. OpenReview.net.

[40] Wandel, N.; Weinmann, M.; and Klein, R. 2021.Learning Incompressible Fluid Dynamics fromScratch - Towards Fast, Differentiable Fluid Modelsthat Generalize. 9th International Conference onLearning Representations, ICLR 2021.

[41] Wandel, N.; Weinmann, M.; and Klein, R. 2021.Teaching the Incompressible Navier-Stokes Equationsto Fast Neural Surrogate Models in 3D. Physics of Flu-ids (AIP).

[42] Wang, H.; Planas, R.; Chandramowlishwaran, A.; andBostanabad, R. 2021. Train Once and Use Forever:Solving Boundary Value Problems in Unseen Do-mains with Pre-trained Deep Learning Models. CoRR,abs/2104.10873.

[43] Yang, C.; Yang, X.; and Xiao, X. 2016. Data-drivenprojection method in fluid simulation. Computer Ani-mation and Virtual Worlds, 27(3-4): 415–424.

[44] Zhu, Y.; and Zabaras, N. 2018. Bayesian deep convo-lutional encoder–decoder networks for surrogate mod-eling and uncertainty quantification. Journal of Com-putational Physics, 366: 415 – 447.

[45] Zhu, Y.; Zabaras, N.; Koutsourelakis, P.-S.; andPerdikaris, P. 2019. Physics-constrained deep learn-ing for high-dimensional surrogate modeling and un-certainty quantification without labeled data. Journalof Computational Physics, 394: 56 – 81.