STEM Denoising + 2× Super-Resolution

33.17 dB PSNR / 0.9650 SSIM on 300 held-out patches, against 26.51 dB / 0.7996 for bicubic upsampling — a gain of +6.66 dB and +0.165 SSIM. The model beats the baseline on 300 of 300 patches.

The auxiliary head did not help. It cost 0.33 dB. Both models were scored on the identical split, so the comparison can be paired, and paired it is unambiguous: mean difference 0.333 dB, SE 0.011, t = 31, plain model ahead on 97% of individual patches. Small, but real rather than noise. The likely cause is capacity competition, not anything subtle — the point_map loss pulls the shared trunk toward sparse delta-like peaks, which is the opposite of what the smooth target needs. Use the plain model.

The qualitative figure is the honest read on what the dB figure means. In the median and best rows the model recovers individual atom columns that are genuinely ambiguous in the input. In the worst row — a very low-dose patch where bicubic manages 16.6 dB — it produces a plausible lattice, but the error map shows systematic errors at column positions, so the reconstruction should not be trusted for measurement there.

Atom-column localisation

Since the auxiliary model predicts an atom map, it can be scored the way a microscopist would: detect peaks, match them to the true point_map within 3 px at 128×128 scale, and count.

Sub-pixel accuracy on the columns it finds, and it rarely invents one, but it misses 20% — and the misses concentrate in the low-dose patches, visible as the sparse bottom row below. That recall figure, not the 33 dB, is the number to quote if the downstream use is measuring lattice spacings.

input and target are not the same resolution, so this is not a pure denoising task.

1500 triplets, filenames matching across all three folders. They come from three source simulations, 500 patches each: 20_ADF_45_200, 25_ADF_45_200, 30_ADF_45_200 (the leading number is presumably thickness or dose).

Two checks worth recording:

• The fields of view are aligned. Correlation between input and the 2×-downsampled target is 0.78–0.95 across sampled patches, so the model has to interpolate, not register.
• The patches do not overlap. Correlation between neighbouring patch indices from the same source is −0.14 to 0.30, i.e. no shared pixels. A random train/test split therefore does not leak.

An EDSR-style residual CNN (0.6M parameters, 0.8M with the auxiliary head) that predicts a correction on top of a plain 2× upsample rather than the image itself.

The global skip matters more than the block count. With only 1200 training patches, letting the network learn the low-frequency image from scratch wastes capacity; giving it the upsample for free means every parameter goes into deblurring and denoising.

The auxiliary head. A second 3×3 convolution branches off the same upsampled features and is supervised by point_map, with total loss L1(image) + 0.5 * L1(point_map). The idea is that forcing the trunk to localise atom columns should sharpen them in the restored image. Only the image head is used at evaluation; the atom map is a by-product. A model without this head was trained identically so the effect is measurable rather than assumed.

Training recipe

One implementation note: the skip inside the network is bilinear, not bicubic, because PyTorch 2.2 has no upsample_bicubic2d kernel for MPS. The bicubic baseline is still computed properly, on CPU.

One split: 1200 train / 300 test, stratified so each of the three source simulations contributes proportionally to both sides. sklearn.model_selection.train_test_split, seed 42. The two models share the identical split, so their numbers are directly comparable.

Nothing was selected on the test set. The epoch budget was fixed in advance and the last epoch is the one reported. Test PSNR is logged every 5 epochs, but only as a convergence trace; no early stopping, no checkpoint picking, no hyperparameter search against it.

Metric conventions, which matter because PSNR and SSIM are both reported with several incompatible defaults in the literature:

• Images are scaled to [0, 1] and predictions clamped to that range; data_range = 1.0.
• PSNR and SSIM are computed per patch and then averaged over the 300, rather than on a concatenated array. The standard deviation quoted is across patches.
• SSIM uses Gaussian weighting, sigma 1.5, use_sample_covariance=False — the Wang et al. (2004) formulation, not scikit-image's default 7×7 uniform window, which reads ~0.01–0.02 higher.
• The baseline is bicubic 2× upsampling of the input, scored against the same targets. Every gain quoted is relative to that, not to the raw 64×64 input, which cannot be scored against a 128×128 target at all.

Four things these numbers do not establish.

1. This is simulated data, and the targets are unusually clean. The target images are smooth and nearly noise-free, so the mapping is closer to “denoise and sharpen toward a known blur kernel” than to the harder experimental case. Expect a drop on the real micrographs in test/.

2. Patches from all three source simulations appear in both train and test. The split is random and stratified, which is the number normally reported, but it measures interpolation within known conditions — not generalisation to a new specimen, thickness or dose. A leave-one-source-out run would measure that, and it would almost certainly score lower.

3. PSNR and SSIM reward smoothness. Both metrics tolerate slightly over-smoothed output, which for atomic-resolution imaging is exactly the failure mode that matters — a blurred atom column still scores well but is useless for measuring a lattice spacing. This is why the atom-localisation numbers are worth more than the dB figure.

4. One split, one seed. With 300 test patches the standard error on mean PSNR is roughly 0.17 dB, so an unpaired comparison of the two models could not resolve a gap of a few tenths of a dB. The paired test in the results section does resolve it, which is why it was used — but it still speaks only for this split and this seed. Run several seeds before generalising the auxiliary-head finding.

The model was run on all 15 experimental images in test/. It scores far worse than bicubic on every one of them — mean 20.78 dB vs 29.61 dB — but the metric is measuring the wrong thing, and the pictures tell a different story from the numbers.

There is no ground truth for these micrographs, so PSNR and SSIM cannot be computed the way they were on the simulation. The protocol used instead: take the real image as the reference R, downsample it 2× to make the model's input, and score the restoration against R. This mirrors the training relationship exactly, and bicubic 2× is scored identically.

Why the metric is misleading here. R is a noisy experimental image. The model was trained to remove noise, so every grain of noise it correctly suppresses counts against it, while bicubic — which cannot denoise but also cannot over-smooth — stays closer to the noisy reference. The effect is largest on the images that were already denoised by other means before being saved: Page1_..._result, the two Radial Wiener images and the iDPC images, where bicubic reaches 33–48 dB because downsample-then-upsample is nearly lossless on a smooth image. On the genuinely noisy raw micrographs the gap is much smaller, 1.1–1.6 dB.

What the pictures actually show, which is the part worth acting on:

• Page1, Page3, Page7 — crystalline, lattice spacing within 1.0–1.5× of the training scale. The model output is visibly cleaner than either the reference or bicubic, with atom columns resolved that are ambiguous in the raw data. This is the model working as intended, and the negative dB is an artefact of scoring against a noisy reference.
• Page5 and Page8 — the model paints a regular lattice texture into amorphous regions and into the vacuum, where the reference has no periodic structure at all. This is hallucination, and it is the failure mode that matters. Both images have lattice spacings 3.8–4.0× coarser than anything in training, so the network is fitting its learned periodicity onto content it has never seen.
• Page4 — very low dose. The output is plausible but the SSIM of 0.12–0.23 says the fine structure is not being recovered reliably.

Everything lives in the workshop-0922/ project folder.

About 13 minutes per run on an M-series Mac (MPS, fp32). No CUDA needed, no data preprocessing step — train.py reads the PNGs directly.

Worth doing next, roughly in order of what each would teach you

1. Leave-one-source-out. Train on two of the three simulations, test on the third. This is the number that predicts behaviour on a new specimen, and it is the honest headline if the model is ever pointed at unseen data. About 40 minutes for all three folds.

2. Fix the scale mismatch on test/. Resample each micrograph to the training lattice scale before inference, then re-read the crops in Fig. 4. This tells you immediately whether the simulation-to-experiment gap is small or fatal. Ten minutes.

3. Add train_data2. It has input and target but no point_map, so it can train the image head. Doubling the data is usually worth more than any architecture change at this scale.

4. Then, and only then, a bigger model. 8 residual blocks at 64 channels is deliberately small. Scaling up is the obvious lever, but it is the least informative one until the generalisation question above is settled.