new article

1 files changed, 0 insertions, 97 deletions

diff --git a/content/posts/an-update-on-local-models.md b/content/posts/an-update-on-local-models.md
deleted file mode 100644
index a916bb5..0000000
--- a/content/posts/an-update-on-local-models.md
+++ /dev/null
@@ -1,97 +0,0 @@
----
-title: ""
-draft: true
-date: 2026-08-22
-lastmod:
-description:
-categories:
-  - LLM
-  - Coding Agents
----
-
-## Qwen 3.8 27B
-
-This week was interesting for local models, because we had the
-open weights release of Qwen 3.8 27B. It was much expected, because
-Qwen 3.6 held the crown for a long time in open models of a size
-that is runnable on almost consumer hardware.
-
-The [Artificial Analysis](https://artificialanalysis.ai/models/qwen3-8-27b)
-score is pretty damn good! 52 with xhigh is in the ballpark of 
-GPT-5.6 Luna (max) and DeepSeek V4 Flash 0731, which were my go
-to models for august, given their relatively low price.
-To think that I could run as a powerful model locally feels
-insane.
-For reference, it sits between a Claude Opus 4.6 and a
-Claude Opus 4.7, and that was the best what we had in February
-in Claude Code.
-
-## Artificial Analysis pareto frontier
-
-https://artificialanalysis.ai is my reference for model comparison.
-It actually compiles scores from many benchmarks. I know there are
-many pitfalls to these benchmarks, but hey, we have to reduce
-the world's complexity, right?
-I found my prefered charts, and that's when you remove all models.
-Then AAI highlights the pareto frontier, with the best option
-at each tradeoff. Go see for yourself!
-
-
-[Int vs Cost/Time per task](https://artificialanalysis.ai/models#intelligence-comparisons) - What frontier model to use in
-what case. Here we can see Luna is pretty neat for cost, and
-that's what I used in august.
-
-[Int vs Active/Total params](https://artificialanalysis.ai/models#model-size) - This helps me understand what I could
-run on my hardware. For this one, you need to filter on small
-(and maybe tiny) models. This show Qwen 3.8 27B as much better
-than the previous Gemma 4 26B A4B. And I can see a new contender
-[Ling 3.0 Tiny](https://artificialanalysis.ai/models/ling-3-0-tiny)
-that looks promising.
-
-## Running modern models
-
-I wonder if local inference got better since https://blog.kagamino.dev/posts/local-llm-inference-offline-scenario/.
-Back then, I had decent inference performance with Gemma 4 E2B
-(AAI 15), but it felt too dumb to do meaningful work.
-I wanted to test Qwen 3.5 9B, but this time it was too slow,
-on my Desktop as well as on my Mac.
-
-As I ran out of my OpenCode Go subscription for the last ten days
-of august, it's time to check if I can work locally!
-Maybe there were new smaller and smarter models, like Qwen
-or Ling ? Or better inference, like oMLX on Mac?
-
-Last time was a one shot scenario, so this time I want to test
-a coding agentic workflow with Pi - it has a smaller system prompt.
-I'll go by feel, because I don't have clear metrics to target.
-
-Sadly, Qwen 3.8 27B is too large, even with oMLX. That's also
-because I need a larger context of 100k tokens. And I couln't
-run Ling 3.0 Tiny, because of unsupported architecture. So much
-for modern models!
-
-## New contenders
-
-But the AAI pareto frontier gives me new ideas of old models
-to run.
-[Qwen 3.5 4B](https://artificialanalysis.ai/models/qwen3-5-4b)
-is still on this frontier with a AAI 20, and a smaller size
-than the
-[Gemma 4 E2B](https://artificialanalysis.ai/models/gemma-4-e2b/)
-(AAI 10) I tested last time.
-
-With oMLX and a 8bits version, it runs using around 10GB
-unified memory. The first prompt is always a bit slow, but then
-the KV cache kicks in and it feels snappy enough.
-But after some time it ended up in a locked grep loop. I think
-this is usable for targeted changes.
-
-I had an attempt on my desktop, which is better a prefill (input)
-than the Mac. But this time it didn't feel really smart.
-
-In the end, I'm a bit disappointed I couldn't run the newer models,
-even though Ling 3.0 Tiny was supposed to fit. But maybe i can
-run it later when the tooling has caught up.
-I think Qwen 3.5 4B may have a spot for offline use, as I can run
-it at 50 tok/s on Mac. I'd like to investigate using it
-as an autocomplete model for IDE.