Exactly, and we all know those are pronounced mooze, nooze, and cabooze.

I think you mean mose, nose, cabose.

So it’s not a binary thing, there’s context and nuance?

Yes. The oauth ID is indisputable. It it seems to be context.ai. But suppose it was a fake context.ai that the employee was tricked into using. Or… or…

Better to report 100% known things quickly. People can figure it out with near zero effort, and it reduces one tiny bit of potential liability in the ops shitstorm they’re going through.

It’s hazardous to blend fixed and variable costs.

Sure. But if that fully depreciates, $1100/year GPU produces $20k of economic benefit, would you decommission it as long as there is demand?

If my data center sells a pflop at $5 because of our electricity use and the data center a state over with newer GPUs sells it at $2.50/pflop, it doesn't matter how much economic benefit it generates, my customers are all going to the data center a state over.

I want to see math on how a single GPU will pull down that much revenue, because that seems like a dubious outcome.

Fair, I was hand waving to make a point. “If it generates more than $1100 + (resale price * WACC) + opportunity cost from physical space/etc” would have been more accurate.

But the point is — you don’t decommission profit generators just because a competitor has a lower cost structure. You run things until it is more profitable for you to decommission them.

That all depends on if you're running your own hardware (unlikely) or renting.

I just don’t see it. Both professionally and personally I’m producing so much more now. Back burner projects that weren’t worth months of my time are easily worth a few hours and $20 or whatever.

Why would I pull back?

You’re probably already experienced at your job and using AI to enhance that, or at least using that experience to keep the AI results clean. That’s something you or a company would want to pay for but it has to be a lot more than today’s prices to make it profitable. Companies want to get more out of you, or get a better price/performance ratio (an AI that delivers cheaper than the equivalent human).

But current gen AIs are like eternal juniors, never quite ready to operate independently, never learning to become the expert that you are, they are practically frozen in time to the capabilities gained during training. Yet these LLMs replaced the first few rungs of the ladder so human juniors have a canyon to jump if they want the same progression you had. I’m seeing inexperienced people just using AI like a magic 8 ball. “The AI said whatever”. [0] LLMs are smart and cheap enough to undercut human juniors, especially in the hands of a senior. But they’re too dumb to ever become a senior. Where’s the big money in that? What company wants to pay for the “eternal juniors” workforce and whatever they save on payroll goes to procuring external seniors which they’re no longer producing internally?

So I’m not too sure a generation of people who have to compete against the LLMs from day 1 will really be producing “so much more” of value later on. Maybe a select few will. Without a big jump in model quality we might see “always junior” LLMs without seniors to enhance. This is not sustainable.

And you enhancing your carpentry skills for your free time isn’t what pays for the datacenters and some CEO’s fat paycheck.

[0] I hire trainees/interns every year, and pore through hundreds of CVs and interviews for this. The quality of a significant portion of them has gone way down in the past years, coinciding with LLMs gaining popularity.

You're forgetting that the 20$ are not a sustainable price point. Would your backburner personal app thingy be worth 200$?

This is thoroughly debunked at this point. The frontier labs are profitable on the tokens they serve. They are negative when you bake in the training costs for the next generation.

Yes, I have often paid more than that to have someone else develop a personal side project.

Compute capacity for the same workload always gets cheaper over time.

No piece of compute capacity (in the form of equipment) I have bought this year has been in any way cheaper than last year.

So what. Fluctuations over a year or two are meaningless. Do you really believe that the constant-dollar price of an LLM token will be higher in 20 years?

I can see a world where energy costs rise at a rate faster than overall inflation, or are a leading indicator. In that scenario then yes I could see LLM token costs going up.

Lol are people like you going to be enough to support the large revenues? Nope.

A firm that see's rising operating expenses but no not enough increase in revenue will start to cut back on spending on LLMs and become very frugal (e.g. rationing).

Before they cut back on human programmers?

I’m not sure tax depreciation rates are the best measure here. Those GPUs will be used for much longer than 6 years, and the returns from the businesses will be an order of magnitude longer.

The jury is still out on this. Those tax based deprecation schedules are largely a relic of traditional data centers, where workloads are fairly moderate compared to AI use cases. Additionally, power and rack space constraints can complicate things quite a bit. If next gen chips are significantly more efficient and you are currently constrained by power availability, you might pull your old servers and replace them with the newer ones regardless of how much useful life you have left.

Azure ran K-80/P-100 fleets a bit longer for 8-9 years . Google does 9 years for TPUs .

In the current generation There are plenty of questions around

- viability of training to inference cascades (the key to extended life) given custom ASICs hitting production like cerebras did early this year.

- energy efficiency of older chips in tight energy environments , just new grid capacity constraints favor running newer efficient chips ignoring perhaps short term(< 1 year) price shock due to war.

- higher MBTF , compared to older GPUs modern nodes are 8 GPU clusters built on 2/3 nm processors depending on HBM memory, the tolerances are much lower especially for training.

- new DCs being spun up are being by up less than ideal conditions due to permitting, part supply and other constraints which will impact operating environment.

Not withstanding, all these issues and even taking a generous 10 year useful life . The expenses dwarf every mega project before it .

> Those GPUs will be used for much longer than 6 years

Will it be worth the cost of electricity to run them if the flops/watt of newer chips is lower?

If demand is less than supply, definitely not.

If every latest-gen is booked solid and there is still unmet demand, why would you decommission?

Like, they break? Or it just becomes more profitable for the data center to replace them?

It will become more expensive to fix than replace. Also more energy intensive than newer generation to operate. MBTF is significant the older the fleet gets higher the failure rates .

A typical node today is 8 GPU node today , you have to keep replacing failed GPUs by cannibalizing parts from other GPUs as nobody is selling new GPUs of that model anymore at higher frequencies.

In addition to outright failure there are higher error rates in computation in graphics it tends to be flickers or screen artifacts and so on.

Azure operated K-80s and P-100s for 9 and 7 years respectively but they were running at 2 GPU nodes and of course were much simpler compared to today’s HBM behomouths on 2/5 nm processor nodes . Google operates their custom ASIC TPUs for about 8-9 years .

With custom inference ASICs like cerebras hitting production the cascading of training NVIDIA chips to inference to get the 5-6 year useful life is also not clear.

PowerPoint is the poster child for the class of applications that AI totally obsoletes:

* A large application whose outputs are independent of the all (people still print slides; when presenting nobody knows or cares what app was used) * Complicated and requires users to learn lots of skills unrelated to the work they’re doing (compare to Excel, where the model and calculations require and reflect domain knowledge about the data) * Practically zero value add in document / info management (compare to word where large documents benefit from structure and organization)

We’re pretty close to presentations just being image files without layers and objects and smartart and all that.

AI will come for all productivity tools, but PowerPoint will be the canary that gets snuffed first, and soon.

I recall reading comments like this when PowerPoint was invented as it would kill all graphic design jobs. The absolute reverse happened. It created an entirely new industry. There is no AI today, or in the near future, that can combine human emotive story telling with impactful design, animated flow and interactivity. Yes it can create flattened boring 'documents' with no passion or depth. I for one would never want to be asked to stand in front of an audience and actually have to present a deck created by Copilot or Claude. Feast your eyes on this PowerPoint creation made by very talented real human designers and ask yourself the question "How long before AI can do this?" https://www.brightcarbon.com/portfolio/intersystems-partners...

Wait, how does PowerPoint do emotive storytelling in a way that a human driving an AI tool could not?

It sounds like you’re confusing my argument that AI can replace PowerPoint tools like gradient, layers, fonts, etc, with an argument I did not make that AI will take humans out of the equation.

Do we know why it works for humans?

Models are trained on human outputs. It’s not super surprising to me that inputs following encouraging patterns product better results outputs; much of the training material reflects that.

> Do we know why it works for humans?

Try to figure it out. You can do it.

If I had to wager a lazy, armchair guess, I think it forces it to think harder/longer

The answer is probably more straightforward than we think, e.g. “the user thinks I can do this so I better make sure I didn’t miss anything”

Been thinking about this a lot recently.

I think we need a way to verify the specs. A combo of formal logic and adversarial thinking (probably from LLMs) that will produce an exhaustive list of everything the program will do, and everything it won’t do, and everything that is underspecified.

Still not quite sure what it looks like, but if you stipulate that program generation will be provable, it pushes the correctness challenge up to the spec (and once we solve that, it’ll be pushed up to the requirements…)

What’s important is to prove useful, high-level properties derived from the specs. The specs of program behavior are just the price of admission.

I agree. It’s kind of like secure boot, in reverse: the high level stuff has to be complete and correct enough that the next level down has a chance to be complete and correct.