Deconstructing the NeRF Kernel
The single trick that made NeRF work — Fourier feature encoding of input coordinates. +10.64 dB PSNR over a vanilla MLP on a 2D image regression task.
Part 1
Why MLPs Need Fourier Features
Spectral bias of ReLU MLPs, how Fourier features fix it, why exponentially-spaced frequencies, the connection to NeRF's volume rendering.
Part 2
Forty Lines of PyTorch
FourierFeatures (zero trainable params) and CoordMLP with toggleable encoder.
View Code on GitHub
Part 3
+10.64 dB From One Trick
Same MLP, same training, same data. Without encoding: PSNR 11.13 dB (blurry mess). With Fourier features: 21.77 dB (sharp reconstruction).