Coming Soon
My previous blogpost gives a very clear visualization of how the latents of simple ReLU networks look like, how to interpret them, and a good description of optimization pressures that force them into or away from local optima. I feel well equipped to investigate the problems of feature splitting and feature absorption. Please reach out to me at ryan.rtjj@gmail.com if you’d like to collaborate.