Imagine if AI agents could learn from their past successes and reuse effective strategies, just like we form habits. This idea of giving AI agents a procedural memory is sparking exciting discussions about efficiency and reliability in AI systems.
Evolutionary algorithms discover a novel attention mechanism that outperforms standard transformers by 4% on WikiText-2, using sparsemax and output gating instead of softmax.