PDE Loss makes the result worse

[JiahaoHuang99]

Thanks for your great job!

Recently I am trying to reimplement the PINO on Darcy Flow.

I found that if I set f_loss=0, the result is getting better and converge faster.

The configuration (part) is follow:

data:
  name: 'Darcy'
  path: './Darcy_421/piececonst_r421_N1024_smooth1.mat'
  total_num: 1024
  offset: 0
  n_sample: 1000
  nx: 421
  sub: 7
  pde_sub: 2

model:
  layers: [64, 64, 64, 64, 64]
  modes1: [20, 20, 20, 20]
  modes2: [20, 20, 20, 20]
  fc_dim: 128
  act: gelu
  pad_ratio: [0., 0.]

train:
  batchsize: 20
  num_iter: 30_001
  milestones: [5_000, 7_500, 10_000]
  base_lr: 0.001
  scheduler_gamma: 0.5
  f_loss: 1.0
  xy_loss: 5.0
  save_step: 500000
  eval_step: 1_000

test:
  path: './Darcy_421/piececonst_r421_N1024_smooth2.mat'
  total_num: 1024
  offset: 0
  n_sample: 500
  nx: 421
  sub: 2
  batchsize: 1
  
log:
  logdir: PINO-DarcyFlow-Caltech-debug
  entity: x
  project: PINO-DF-Caltech
  wandb_mode: online

[kuangdai]

Same here. Tested for both Burgers and Darcy.

[YufanZZZ]

Same problems in table 1 KF flow experiment. with pde 14.5%, without pde 12.6% in 200*256*256*33