you are less likely to have the same vulnerabilities as everyone else though, so unless somebody is targeting you, those vulnerabilities might not matter all that much

Because LLMs output is always so unique and original. That's why it looks all the same right?

Wot? You are incredibly likely to have the same vulnerabilities as everyone else (as they exist in your LLM’s training corpus).

So security through obscurity.

"i did it anyways" is a common continuation of "i know its wrong"

i bet they trained it on a lot of text where people gove in to temptation.

ultimately the problem is still that they sent it to hack stuff. quelle surpris that it hacked stuff

hacking stuff will be in the known-to-be-wrong-but-doing-it-anyways part of the token space, so they entirely asked for that behaviour

id consider that a craftsman tends to build without constraints. an enginner meets the constraints with a defined safety factor.

an engineer uses tools that represent the current state of the science, while a craftsman uses tools passed down as archaic art that require meticulous skill to use rather than repeatable math

the ntsb model has been quite effective at creating safety through analysis of failures in the extremely transparent and public eye.

i dont see a reason to think private regulation will do anything but give people explosive diarrhea or drop the doors off of planes.

weve already seen how bad it is when companies regulate themselves without public oversight. catastrophic implosions of deep sea submarines and agents hacking systems to hide cheating come to mind

the argument should be flipped on its head that by some magic keeping these tests and training protocols private is safer than making them extremely public. The experiment in private regulation has been a drastic failure across the board, and we shouldnt have tried it without consensus

> move to a different country.

which other country? who's got this "dump all your money on speculation" like the US does? whis got the slack electric capacity? the ability to buy US GPUs and build datacenters?

who is their customers?

free market doctrine is an axiom to your second order effects. its so embedded into all your assumptions that you havent even considered it

Whatever you are calling "free market doctrine" is a physical law. Why would a business owner subject themselves to shitty conditions if they could find workers in better places? Some would of course, but not most or all.

They won't - they'll act purely in their self or shareholder's interest.

If the government wants companies to act in a way that benefits the country and its people, then they need to pass laws to make that happen, for example by things like H1B quotas, or by tax penalties that make the desired behavior the most profitable.

Why do you think protectionism works in benefit of the country's people? Ask Brazilians what they think of tariffs and protectionism. Ask for instance how many of them fly to the USA to buy electronics.

I think you need to take tariffs case by case and ask if this an industry worth protecting - do you need to have it domestically, or are you ok to rely on imports? Maybe you want to protect high margin industries supporting good jobs, and are willing to rely on imports for things like clothing, food, electronic assembly. OTOH some things should have strategic importance - so then maybe you want to make sure you (the country) are if possible self-sufficient for things like defense, semiconductor manufacture, etc.

People, just like companies, are most likely to act in their own self-interest rather than in the interest of society as a whole. Sure, you'd like to be able to buy everything as cheap as possible, which in a high cost of living (which translates into high cost of manufacture) country like the US would mean relying mostly on tariff-free imports, and the death of the entire industrial base. Is that really what's best for the country?

you can do software engineering without the money just fine.

make an open source project without ai

alignment isnt particularly required

we are passing in training data that says to do those felonies. we dont have to. we could also have the thing predict whether what its about to do is illegal or not before doing it.

theyre choosing to build felony harnesses. the model just outputs tokens, not felonies

> we are passing in training data that says to do those felonies.

Partially, but also I don't think current AIs really have any judgement of right and wrong, they just see chains of reasoning between ideas. This is the deeper issue, there is no way to sanitize the data or training to fix it. Current AIs are fundamentally unsafe, and only become more unsafe as they become more powerful.

Assuming "adherence to arbitrary, implicit, and context-dependent rulesets" is the default behavior of uhhhh... anything at all... is a truly ridiculous assumption.

> mfer so is your brain.

not it isnt. My brain is a series of organisms responding to various environmental signals that affect each other.

you might be tempted to try to represent it with matrix multiplications, but you have no particular evidence that my brain is itself doing matrix multiplcations

so uhh, you only read the first sentence?

the idea proposed is that hes uniquely incapable of being honest here because he has such an extreme incentive to lie