I love these optimization tales. Memory throughput bottlenecks (extremely common, perhaps moreso than they seem) are my favorite to tackle - there are frequently some juicy optimizations that can apply there.
Do model weights have any spatial locality that can be exploited? If so, there are some more general pre-compression techniques that might be interesting to try, e.g. bitshuffle is one I've worked with (https://github.com/kiyo-masui/bitshuffle).
Another fun fact: in some scenarios (depends a lot on CPU and memory characteristics), gzip+memcpy+gunzip can be faster end-to-end than just memcpy. I forget where I first heard this but my familiarity comes from the blosc compression library.
Like pretty much everything else about human beings, I expect most of us exist at a point on a spectrum rather than belonging to this type or not.
I think the engineering mindset comes from a place of curiosity. I don't agree that there is anything fundamentally different about the person who chooses to be an engineer versus not, I think everyone just has a different mix of characteristics and preferences.
For me, the act of creation is the compelling part. Sometimes that's writing code, other times it's seeing the fully-formed thing come into being. For the latter, LLMs can certainly help me do more, or at least take care of some of the menial stuff like writing test cases while I do more interesting things like think about strategy/architecture.
Just make sure you're reviewing your LLM's code as if you were a colleague's!
Some random thoughts, since I've had a similar train of thought for a while now.
On one hand I also lament the amount of hardware-potential wastage that occurs with deep stacks of abstractions. On the other hand, I've evolved my perspective into feeling that the medium doesn't really matter as much as the result... and most software is about achieving a result. I still take personal joy in writing what I think is well-crafted code, and I also accept that that may become more niche as time goes on.
To me this shift from software-as-craft to software-as-bulk-product has some similarities to the "pets vs cattle" mindset change when thinking about server / process orchestration and provisioning.
Then also on the dismay of JS becoming even more entrenched as the lingua franca. There's every possibility that in a software-as-bulk-product world, LLM-driven development could land on a safer language due to efficiency gains from e.g. static type checking. Economically I wonder if an adoption of a different lingua franca could manifest by way of increasing LLM development speed / throughput.
> LLM-driven development could land on a safer language
Why does an LLM need to produce human readable code at all? Especially in a language optimized around preventing humans from making human mistakes. For now, sure, we're in the transitional period, but in the long run? Why?
"It has been lost in AI money-grabbing frenzy but a few years ago we were talking a lot about AIs being “legible”, that they could explain their actions in human-comprehensible terms. “Running code we can examine” is the highest grade of legibility any AI system has produced to date. We should not give that away.
"We will, of course. The Number Must Go Up. We aren’t very good at this sort of thinking.
Because the traits that make code easy for LLMs to work on are the same that make it ideal for humans: predictable patterns, clearly named functions and variables, one canonical way to accomplish a task, logical separation of concerns, clear separation of layers of abstraction, etc. Ultimately human readability costs very little.
In a sense they do use their own language; they program in tokenized source, not ASCII source. And maybe that's just a form of syntactic sugar, like replacing >= with ≥ but x100. Or... maybe it's more than that? The tokenization and the models coevolve, from my understanding.
If we do enough passes of synthetic or goal-based training of source code generation, where the models are trained to successfully implement things instead of imitating success, then we may see new programming paradigms emerge that were not present in any training data. The "new language" would probably not be a programming language (because we train on generating source FOR a language, not giving it the freedom to generate languages), but could be new patterns within languages.
> For now, sure, we're in the transitional period, but in the long run? Why?
Assuming that after the transitional period it will still be humans working with ai tools to build things where humans actually add value to the process. Will the human+ai where the ai can explain what the ai built in detail and the human leverages that to build something better, be more productive that the human+ai where the human does not leverage those details?
That 'explanation' will be/can act as the human readable code or the equivalent. It does not need to be any coding language we know today however. The languages we have today are already abstractions and generalizations over architectures, OSs, etc and that 'explanation' will be different but in the same vein.
Well, IMO there's not much reason for an LLM to be trained to produce machine language, nor a functional binary blob appearing fully-formed from its head.
If you take your question and look into the future, you might consider the existence of an LLM specifically trained to take high-level language inputs and produce machine code. Well, we already have that technology: we call it a compiler. Compilers exist, are (frequently) deterministic, and are generally exceedingly good at their job. Leaving this behind in favor of a complete English -> binary blob black box doesn't make much sense to me, logically or economically.
I also think there is utility in humans being able to read the generated output. At the end of the day, we're the conscious ones here, we're the ones operating in meatspace, and we're driving the goals, outputs, etc. Reading and understanding the building blocks of what's driving our lives feels like a good thing to me. (I don't have many well-articulated thoughts about the concept of singularity, so I leave that to others to contemplate.)
It's been a decade+ since I used to catch the Green line at Park St, but at that time it was the noisiest, squealiest station that I regularly used. Not surprising to learn that parts of that station are left over from the 1890s.
IMHO there's no gatekeeper of what the "real" web is or should be. It grew organically - regular people building things they liked or needed. It's certainly more of a life necessity than it used to be, but that happened organically too.
I know there are strongly held opinions about this, but I for one see no reason why the "application web" can't peacefully coexist, and interlink with, the document web. In my opinion it therefore makes sense to allow for different models for the application web, ones that do not revolve around a document.
On the other hand, if we're just bashing on javascript being the lingua franca of the web, that's a train I'll happily board!
I've got a pretty large flutter app in production, using just the flutter-provided building blocks like ValueNotifier, ListenableBuilder, InheritedWidget, etc. It has scaled quite well for me.
The main issue IMHO with many of these boilerplate-reducing packages is that they feel like one-way trips. Most of them require a change to widget inheritance, and they all have slightly different syntax and semantics for wiring things up to state changes. This means if you get a few years into a project, migrating away from the package you chose at the beginning will probably be very difficult.
So while the quick example in the readme of this package looks simple and understandable, locking in to a third-party library makes me nervous, especially if the main benefit is just fewer keystrokes. Does anyone have experience or informed opinion here that would be willing to chime in?
I believe that no matter what framework you use, you’ll encounter similar challenges. Every framework introduces its own complexity by nature — the key difference lies in whether its abstraction can reduce the complexity of building sophisticated applications to a manageable level.
To me, the strength of a framework isn’t just about reducing how much code you write — it’s about providing a way of thinking that helps developers build and maintain applications more easily.
For this project, my goal is to fully leverage the composability of Vue’s Composition API and provide automatic dependency tracking to help developers write cleaner and more reactive Flutter applications.
I still haven't found a way to coax MathML into looking the way I want it... Even using the same fonts (like Computer Modern or its descendants) there's still something not quite the same as LaTeX-drawn math. It's a nitpick but noticeable for me.
Been a happy customer of Fastmail for ~10 years now. Nice web and mobile interfaces. I've never been impacted by downtime in that time (to my knowledge).
Thank you very much for all this feedback! I really appreciate your consideration.
For your note on different states for screens. Can you elaborate or give an example? I'm quite interested in this mode that you're describing.
The ellipse issue sounds like snapping getting in the way. It's on my list to come up with a better heuristic of when to apply snapping or not. One example of an idea I have here is to incorporate some hysteresis effect - essentially examining the local history of mouse movements when deciding to apply snapping or not. Right now each mouse movement is treated independently.
Also, were you in dark mode? It's definitely not intended that the whole screen should change color - a popup should dim the content behind. The whole screen changing color sounds like a bug and I'll investigate.
>For your note on different states for screens. Can you elaborate or give an example? I'm quite interested in this mode that you're describing.
I meant the very obvious feature (you might already have it) of objects with properties and a unique id, then a state that indexes snapshots of those objects with different properties. With the ability to diff between states by id of the object. So State1:[Square{id:0, x:0, y:0}], State2:[Square{id:0, x:screen.width, y: screen.width}], where going from state1 to state2 would tween the square from top-left to bottom-right.
I am assuming you already have something like that (you already have the objects, the properties, and some states like undo/redo) and it's about exposing the ui around it with labeled states at the user-level.
> Also, were you in dark mode? It's definitely not intended that the whole screen should change color - a popup should dim the content behind. The whole screen changing color sounds like a bug and I'll investigate.
That's what it was! I tried with light mode in firefox and it worked much better.
Ah, ok - thank you for elaborating on that. I don't actually have that type of interactivity at the moment, though you're right in saying all the information is there in order to make it happen. I also like your idea of changing between the states via some collapsible menu. In theory all of this could be achieved just by building on what's already here. Do you mind sharing what sorts of things you might use this for?
Glad to hear dark mode was the other issue :-). I'll investigate further, it's probably just a matter of a flag not being propagated properly during the scene change.
This is a great point and not one that I had considered. I suspect I am the same way - scanning back and forth while studying a diagram.
It's possible this tool may be more appropriate for e.g. presentation of info to a mixed audience. Technical people may have questions that can be answered with detail that non-technical people would find distracting, if it were visible on the main diagram. And "presentation" here could be a literal presentation in a conference room, or documenting something on a site that people visit on their own.
You could make the argument in that example that you should create two diagrams, one for each audience. I think that is a valid argument - but I can also imagine maintaining two separate diagrams that present similar information could be irritating.
Do model weights have any spatial locality that can be exploited? If so, there are some more general pre-compression techniques that might be interesting to try, e.g. bitshuffle is one I've worked with (https://github.com/kiyo-masui/bitshuffle).
Another fun fact: in some scenarios (depends a lot on CPU and memory characteristics), gzip+memcpy+gunzip can be faster end-to-end than just memcpy. I forget where I first heard this but my familiarity comes from the blosc compression library.
reply