Neurocode - Latent-Space Program Synthesis via Diffusion Models and Bespoke Compilation
As someone who uses LLMs to generate code, I am used to seeing a Chain of Thought process as the LLM works through a coding challenge. These chains of thought are expressed in natural language, which means that the LLMs are constantly translating and committing to concrete expressions of a more latent and liminal process.
Coconut (Chain of Continuous Thought) is an approach which does not require LLMs to keep on resolving intermediate steps into natural language.
I wondered if this could be applied to code generation, and then I wondered if we could stay in latent space beyond code generation, reframing what we think of as an executable. I was interested in:
