I’m
Welcome to my blog! I'm a seasoned Software Engineer with over 20 years of experience in the tech industry. My passion lies in writing about the latest advancements in technology and sharing my insights through public speaking. I envision a world where technology is accessible to everyone and understood as deeply as by those who create it. Join me on this journey as we explore the fascinating world of technology together.
Popular Tags
Most searched keywords
Recent posts
Don't miss the latest trends
La distillation de modèles de langage
Comment un petit modèle apprend à faire un travail aussi bien qu’un grand, ce que cela coûte, et quand c’est une mauvaise idée. Guide conceptuel et opérationnel. Dernière revue : août 2026. Ce que couvre cet article La distillation est la technique derrière la plupart des petits modèles qui dépassent leur catégorie, et derrière une…
Model distillation in large language models
How a small model learns to do one job as well as a large one, what it costs, and when it is the wrong idea. Conceptual and procedural guide. Last reviewed August 2026. What this covers Distillation is the technique behind most small models that punch above their weight, and behind a large share of…
Context compaction is not a compression problem
Three days ago I argued that the fix for long-context failure is to send the model less. That was right, and it was incomplete. It tells you to shrink the context. It does not tell you what to keep. The default answer to that second question has been recursive summarization. Every few turns, compress the…
Lost in the Middle : Pourquoi les modèles de langage oublient et comment y remédier
Vous avez récupéré les bons documents. Vous les avez tous mis dans le prompt. Le modèle est quand même passé à côté de la réponse. Cet article explique le mécanisme derrière cet échec, et ce qu’il faut faire. En résumé : un transformeur ne traite pas toutes les positions de son entrée de la même…
Lost in the Middle: Why Language Models Forget and How to Fix It
You retrieved the right documents. You put them all in the prompt. The model still missed the answer. This article explains the mechanism behind that failure “Lost in the Middle” and what to do about it. The short version: transformers do not treat all input positions equally. Recall follows a U-shaped curve. Information at the…