The Future of Programming
I wanted to share it to get your thoughts and see if you foresee the same implications. Beyond Co-Pilots and Snippet Generation We're all seeing how AI can generate code, help debug, or explain snippets. But what if we take this much, much further?
Powerful, Multilingual Base Models: We already have models like Qwen, Llama, Gemini, etc., which are proficient in programming and understand multiple languages. These are our starting point. The Real Leap: Deep Training on Our Specific Code: This is the game-changer. It's not just about using a generic pre-trained model with limited context. I'm talking about the ability to train (or perform advanced fine-tuning on) one of these models with our entire proprietary codebase: hundreds of megabytes or even gigabytes of our software, our patterns, our internal APIs, our business logic.
The 'Program' Evolves into a Specification: Instead of writing thousands or millions of lines of imperative code as we do today, our primary "programming work" would involve creating and maintaining a high-level specification. This could be a highly structured JSON file, YAML, or a new declarative language designed for this purpose. This file would describe what the software should do, its modules, interactions, and objectives. 'Compiling' Becomes 'Training': The "compilation process" would take our specification (let's call it "program.json"). It would use the base model (which might already be pre-trained with our code or would be trained at that moment using our code as the primary corpus). The result of this "compilation" wouldn't be a traditional executable binary, but a highly specialized and optimized AI model that is the functional application.
Hardware Will Make It Viable: I know that right now, training large models is expensive and slow. But let's think long-term: GPUs 100x more powerful than today's, with Terabytes of VRAM, would make this "training-compilation" process for an entire project feasible in a weekend, or even hours. The current "horror of training" would become a manageable process, similar to a large compilation today.
Why Would This Be Absolutely Revolutionary? Exponential Development and Evolution Speed: Need a new feature or a major change? Modify the high-level specification and "recompile" (retrain the model). Automatic and Continuous Refactoring: The hell of massive manual refactoring could disappear. If you change the specification or update the base model with new best practices, the "code" (the resulting model) is automatically "refactored" during retraining to align. The 'Language' is the Model, the 'Program' is the Training Data: The paradigm shifts completely. The true "programming language" lies in the capabilities of the base model and how it can interpret our specifications and learn from our code. The "software" we directly write becomes those specifications and the preparation of data (our existing code) for training. The Programmer's Role: Evolution or Extinction (Towards AI Analyst/Architect): Line-by-line coding would drastically decrease. The programmer would evolve into an AI systems analyst, an architect of these specifications, a "trainer" guiding the model's learning, and a validator of the generated models. We define the what and how at a much higher level of abstraction. Custom-Tailored, Ultra-Optimized Software: Each application would be an AI model specifically fine-tuned for its purpose, potentially far more efficient and adapted than modular software assembled piece by piece today. I know this is years away, and there are many challenges (interpretability of the final model, debugging, security, etc.), but the direction seems clear. We're already seeing the early signs with models like Qwen and the increasing capabilities of fine-tuning.
That shouldn't be needed. The current "promise" is that AI should reason like a human, so in theory (or at least in the original definition of AGI) it should be literally the same as if giving instructions to a human engineer.
The problem right now is that the models display higher than average expertise but only in specific and narrow ways. In my opinion we still have narrow AI with LLMs, it's just that it's narrow in language and context processing, which makes it seem like it's doing actual reasoning. If it's doing any reasoning it is only indirectly by some coincidence that transformers are capturing some higher order structure of the world. What we need is an AI that thinks and reasons like a human so that it can easily take a task from beginning to end without needing any assistance at all.
but spec to code with an LLM takes something like six orders of magnitude more work than a traditional compiler, solving two of those OOMs with faster GPUs just doesn't get you there