this seems like a pattern seen across industries when it comes to AI

even more consolidation and lock in

you can already "compile" TS binaries with deno, but it'll include the runtime in it and etc. so it'll take some disk space but I think these days it's less of a concern than before

Totally, it's inconsequential for our use cases.

I think the binaries wind up being somewhere around 70mb. That's insane, but these are disposable tools and the cost is negligible in practice.

you're right. we should just not use any interpreted/script languages because they're not as fast as compiled ones.

why does a CLI tool that just wraps APIs need this native performance?

that 1M context thing, I wonder if it's just some abstraction thing where it compresses/sums up parts of the context so it fits into a smaller context window?

You don’t normally compress the system prompts, though I guess maybe it treats its own summary with more authority. This article [0] talks about the problem very well.

Though I feel it’s most likely because models tend to degrade on large context (which can be seen experimentally). My guess is that they aren’t RLed on large context as much, but that’s just a guess.

[0]: https://openai.com/index/instruction-hierarchy-challenge/

as the context fills up, the model will generate based on that context, incl. whatever illegal stuff you've said, i.e. it'll mimic that, instead of whatever safety prompt they have at the top

they could make it more "safe" but it'd be much more invasive and would likely have to scan much more tokens also, and it'd cause false positives (probably the biggest reason it's not implemented)

these LLMs will never be able to mitigate this unless they literally scan everything all the time and nobody is gonna want that.

besides, open source models exist now

I remember him tweeting about how he can "feel the AGI" when speaking to GPT

Another meaningless, extremely cringeworthy, tweet, hailed as a messianic message by many at the time.

Yeah, it's hard to say if he's doing marketing because that's his job or if he's really swallowed the whole pill

Is it really hard to figure out that the owner of a company, who personally stands to make 100s of billions, would be doing marketing when talking about said company? Do they not teach critical thinking anymore in schools, did it go away with phonics too? Why would you ever ignore the MASSIVE conflict of interest here, it's just really foolish but it's endemic not just in tech journalism or journalism in general where people just take the words of others and not apply any critical analysis to them.

It's all access journalism now, waste of time.

> Is it really hard to figure out that the owner of a company, who personally stands to make 100s of billions, would be doing marketing when talking about said company

The question isn't about what action he's taking, it's about what motivates him under the surface. Obviously what he is doing is marketing. What I'm curious about is whether he truly believes his own marketing or if he is just doing it because its his job

People that are good liars are good at it because they are lying to themselves at the same time. Even if they can initially compartmentalize I believe after a while it gets them too.

it won't be creative because it's a transformer, it's like a big query engine.

it's a tool like everything else we've gotten before, but admittedly a much more major one

but "creativity" must come from either it's training data (already widely known) or from the prompts (i.e. mostly human sources)

how is it "doomed"?

The cost far outweighs the profits.

i am already on api tokens for the chinese open source models and no subscriptions. these are all available in the original form open source and priced above the inference cost. i think this is the long term option.

zero degradation in speed or quality seen.

So you see better performance with the API plans than the subscriptions?

there are consumers who legitimately drool over RGB