FUSE: A New Metric for Evaluating Machine Translation in Indigenous Languages
November 11, 2025How AI Agents and Large Language Models Work Together Like a Super Team
November 11, 2025Practical Steps to Implement Self-Reporting Tests
For teams adopting this approach, start small. Choose a well-defined, well-specified subsystem to translate or reimplement. GNU ed was perfect because it is small but complex. Identify the reference implementation (the original GNU ed) and the desired replacement (the Rust version). Build a test harness that runs both with identical inputs and compares outputs. Start with the most basic commands and gradually expand. For each new feature, write tests first. When tests fail, note what is missing and add it to the test suite. Never mark a test as passed until it actually passes. This sounds obvious but is the most common mistake. If tests can pass when the system is broken, they are worse than useless.
For LLM collaboration, the workflow becomes: For each feature, write a test that fails. Then, task the LLM with “implement feature X such that it passes this test”. Since the test defines correct behavior, the LLM can use it as a specification. When the test passes, you know the feature is done. This is the reverse of traditional development where you write code then tests. Here, tests come first and are used to generate the code. This also works for legacy code: the tests are the living documentation of what the system should do, and the LLM can extend it by following the test patterns.
- Test failures become step-by-step instructions for the LLM
- Each test failure includes expected output, actual output, and the relevant source section
- The LLM never guesses what correct means – the tests show it
- This workflow enables one developer to oversee dozens of LLM-assisted projects with confidence
After months of development, the final result was a Rust implementation of GNU ed that is 100% compatible. It passes every test the original does. But more importantly, the test suite and testing framework can now be used with any other legacy Unix tool. The method is general. And the LLM-assisted development process is so efficient that the same developer can oversee dozens of such projects simultaneously, as each one is guided to correctness by its tests. This is the future of software engineering with AI: not writing code, but building the systems that ensure code is correct.
