They swapped the tokenizer which either means a new pretrain, or token/weights surgery. The latter one seems more likely both because
- economics: i'd wager a bet that Opus 4.7 is just distilled Mythos Preview
- performance: surgery like this would explain the spiky performance and weird issues
I think you can teach some skills through games. Coding in a REPL loop is great for learning certain types of problem solving since the feedback loop is so tight.
Chess is another good one, but the feedback loop is not nearly as tight.
the feedback loop point is exactly it. flight simulators work because every decision gets immediate feedback. most real life skill practice has the worst feedback loop of all, sometimes you don’t find out you negotiated badly until years later. that’s what we’re trying to compress
Fwiw, with its predecessor's Qwen3.5-35B-A3B-Q6_K.gguf, on a laptop's 6 GB VRAM and 32 GB RAM, with default llama.cpp settings, I get 20 t/s generation.
Have you tried running llama.cpp with Unified Memory Access[1] so your iGPU can seamlessly grab some of the RAM? The environment variable is prefixed with CUDA but this is not CUDA specific. It made a pretty significant difference (> 40% tg/s) on my Ryzen 7840U laptop.
Your link seems to be describing a runtime environment variable, it doesn't need a separate build from source. I'm not sure though (1) why this info is in build.md which should be specific to the building process, rather than some separate documentation; and (2) if this really isn't CUDA-specific, why the canonical GGML variable name isn't GGML_ENABLE_UNIFIED_MEMORY , with the _CUDA_ variant treated as a legacy alias. AIUI, both of these should be addressed with pull requests for llama.cpp and/or the ggml library itself.
Hmm. Perhaps there's a niche for a "The Missing Guide to llama.cpp"? Getting started, I did things like wrapping llama-cli in a pty... and only later noticing a --simple-io argument. I wonder if "living documents" are a thing yet, where LLMs keep an eye on repo and fora, and update a doc autonomously.
I hadn't tried that, thanks! I found simply defining GGML_CUDA_ENABLE_UNIFIED_MEMORY, whether 1, 0, or "", was a 10x hit to 2 t/s. Perhaps because the laptop's RAM is already so over-committed there. But with the much smaller 4B Qwen3.5-4B-Q8_0.gguf, it doubled performance from 20 to 40+ t/s! Tnx! (an old Quadro RTX 3000 rather than an iGPU)
That is pretty solid, I have a 2070 with 8GB VRAM and 64GB RAM, but I haven't run too much. I regret not getting a 3090 back when I built this machine.
Nod. Mine was VR dev leftovers. Fwiw, running 6ish prompts in parallel, roughly doubles my aggregate t/s (but requires cooling kludgery). If one's goal is not local, but rather real-time or consistent or transparent or scalable, there's AWS.
I am waiting for the 2x usage window to close to try it out today.
If they are charging 2x usage during the most important part of the day, doesn't this give OpenAI a slight advantage as people might naturally use Codex during this period?
They were more than happy to write me testimonials.
reply