reproducibility of paper

[phtephanx]

Hi Felix,
comparing the architectural details of Structure by Architecture: Disentangled
Representations without Regularization with the base configuration, there's no yaml configuration yet which reproduces the paper, is there? The latter creates an MLP w/o any convolutional blocks, also the activation function is ELU instead of Mish.

[felixludos]

The demo config does not reproduce the paper results because it is primarily meant for users to check whether their environments are setup correctly. However, by modifying some of the other commands in the readme, you should be able to reproduce the results. For example, to train an SAE-12 model on 3D-shapes, you just have to replace the dataset: fig train hybrid a/e/conv12 a/d/strc/12 m/ae d/3ds (assuming you already have the dataset downloaded/formatted as discussed in the readme). Generally, all the dataset-specific information is in "config/d/" and can be referenced in a command with "d/".
Also, the nonlinearity can specified using "--model.nonlin mish"

[phtephanx]

I executed both

fig train hybrid a/e/conv12 a/d/strc/12 m/ae d/3ds --model.nonlin mish --download

(default learning rate is 1e-4) and

fig train hybrid a/e/conv12 a/d/strc/12 m/ae d/3ds --model.nonlin mish --model.optim.lr 0.0005 --download

see notebook.

The latter should be the exact training setup of the paper.

The training loss does not seem to converge as expected.
On the other hand one cannot inspect the fid to verify this because there seem to be a bug:

failed to compute the Rec FID
Traceback (most recent call last):
  File "/usr/local/bin/fig", line 33, in <module>
    sys.exit(load_entry_point('omnifig', 'console_scripts', 'fig')())
  File "/usr/local/lib/python3.7/dist-packages/omnifig/top.py", line 42, in entry
    main(*argv, script_name=script_name)
  File "/usr/local/lib/python3.7/dist-packages/omnifig/top.py", line 62, in main
    out = project.run(config=config)
  File "/usr/local/lib/python3.7/dist-packages/omnifig/organization.py", line 324, in run
    return mode.run(config.sub('_meta'), config)
  File "/usr/local/lib/python3.7/dist-packages/omnifig/modes.py", line 47, in run
    return script_fn(config)
  File "/usr/local/lib/python3.7/dist-packages/omnilearn/op/training.py", line 395, in iterative_training
    output = run.evaluate()
  File "/usr/local/lib/python3.7/dist-packages/omnilearn/op/runs.py", line 713, in evaluate
    eval_out = model.evaluate(self, config=config)
  File "/usr/local/lib/python3.7/dist-packages/omnilearn/op/framework.py", line 193, in evaluate
    return self._evaluate(info, config, out=out)
  File "/home/learn_rep/src/methods.py", line 221, in _evaluate
    out = super()._evaluate(info, config, out=out)
  File "/home/learn_rep/src/methods.py", line 113, in _evaluate
    self._compute_fid(fid, generate_fn=_rec_gen, name='rec', out=out)
  File "/home/learn_rep/src/methods.py", line 87, in _compute_fid
    dist = out[key]
KeyError: 'rec_fid'