This reduction to the halting problem looks too handwawy to me. I don't see as a given that the possibility of the system taking into account the attack follows from the existence of the attack.
Formally, any non-trivial semantic property of a Turing machine is undecidable.
Semantic here (roughly) means "behavioral" questions of the turing machine. E.g. if you only look at the "language" it defines (viewing it as a black box), then it is undecidable to answer any question about that language (including things like if it terminates on all inputs).
Practically though that isn't a complete no-go result. You can do various things, like
1. weaken the target you're looking for. if you're ok with admitting false positives or false negatives, Rice's theorem no longer applies, or
2. rephrase your question in terms of "syntatic properties", e.g. questions about how the code is implemented. Rust's borrow checker does this via lifetime annotations, for example.
Rice's theorem is a close corollary, but I did mean the halting problem. Pointing to the halting problem was a bit of a throwaway quip because the "general shape" of it is an easy smell test for whether something is likely to be possible:
If you have access to run a transform on data, you can use it to train a model that acts as a detector of whether that transform has been applied to the data.
When you have a detector for a given property, you can use that detector to alter behaviour to exclude that property.
And that is the abstract core of why the halting problem is unsolvable.
In this case, if you have access to a mechanism for poisoning data, you can use that to train a detector. Once you have a detector, you can either exclude poisoned data, or use it for adversarial training.
Either way: The existence of the poisoning mechanism can be directly used to derive the tools to create its own antidote.
And that's back to the core of the halting problem.
> If you have access to run a transform on data, you can use it to train a model that acts as a detector of whether that transform has been applied to the data.
This still seems too handwavy. For example, you're implying that the transform is one that can be learned by gradient descent. As a trivial counterexample, you can't train a model to detect valid (text, SHA) pairs.
This particular one doesn't seem to be a problem for your argument, but I still think your argument generally does not hold.
You can't train a model to detect arbitrary text to SHA, so yes, there are edge cases - this doesn't fully generalise. You can, however, trivially train a model to detect natural language to SHA. More specifically, for poisoning to work, the output needs to have qualities distinctly different in the poisoned or unpoisoned case, pretty much by definition, or we wouldn't notice the effect, and the reason the general case doesn't work for SHA is that you can feed in text that is likely statistically indistinguishable (you can feed SHA in as text) from the output, or "close enough".
This is why I pointed out that the only way poisoning has a chance of working other than over very short timelines is if the tools to do so remains private and inaccessible to the public.
It's a bit of a leap, but the halting problem can be generalized to:
It is impossible in the general case to produce a detector function f(x), that will decide if program x behaves according to rule y if x can include f(x) as part of the itself.
The reason is that if a program x can make use of the detector, it can effectively do if f(x) { do the opposite of what f(x) predicts}
The leap from that to poisoning might be a bit unintuitive, but it boils down to the poisoner having a mechanism that would alter model behaviour.
If you have access to that mechanism, you can produce a detector by using the mechanism to induce the unwanted behaviour, and train a model on that.
Once you have a detector, you can behave differently based on the signal from the detector, and by extension avoid the effects of the original mechanism.
It doesn't need to be easy, but unlike with encryption it also doesn't need to be particularly precise. E.g. it's okay to exclude non-poisoned training data because you didn't manage to create a precise enough detector, as long as you don't exclude too much.
Basically any poisoning attack is also fundamentally limited because it needs to be non-invasive enough for humans not to be adversely affected, and that limits the problem space severely - the poisoning mechanism basically becomes reduced to a training mechanism to train out places where the models act different to humans.
This isn't my field, but I see a huge gap between what in the abstract they say it would be feasible for us and what we're currently capable of. I mean, we're able to send space probes around, but self-replicating space probes and Dyson spheres feel on another level. Am I the only one?
Life itself could arguably be a Von Neumann probe. It's so good at spreading that it's a problem, when we send probes to other bodies in our own solar system we often sanitize them because we worry life will hitch a ride and start colonizing.
Life on a planet is a lot like a continuous fire, fires often send out embers that start new fires elsewhere.
You send out little packets of life to new places, wait single-digit billions of years (a blink of an eye for the universe, really), boom: new intelligent species with potential to shoot more seeds out into the universe.
Do we have anything that self-replicates physically?
Software, sure. I know 3D printer folks will sometimes 3D print parts for new machines. But nothing that fully replicates itself, right? Especially autonomously.
I guess the OP meant something that is more flatous for human vanity to feel like a demiurge when at most it would be a tool of cosmos in its experiment.
Additionally, you also have to consider that a self-replicating space probe should be able to find, retrieve, and process the raw materials needed to build new probes on its own. A 3D printer can print some of its own parts, but with externally-provided material that it isn't able to produce on its own.
What we're capable of != what we're doing / not doing because of political will. We are technically capable of reaching significant fractions of c with tech from the 1960s. We'll never do that because there's no will to do it, but the tech is there.
Same for self replicating stuff. We could build self-contained factories that build stuff from raw minerals, but we'll likely not do it until there's a will for it. Or need for it.
Political will is part of the Fermi paradox. So are technical reliability and cultural stability.
The idea that you can just build a thing and send out a swarm and (slow) boom - you've colonised the galaxy, and all the adjacent galaxies - is hopelessly naive. To the point I'd call it stupid and silly.
Let's say you have a replicator thing that works. You send them out in swarms.
And then what? Some die, some miss, some are destroyed by accidents.
Some work.
But "a replicator landed and made some more" is not colonisation. Colonisation implies there's some kind of to-and-fro traffic, maybe trade, some kind of information exchange at a minimum.
And that implies the source civilisation has political, technological, and cultural stability, which can survive an incredibly slow diaspora.
Colonisation worked on Earth because it didn't take long to cross the Atlantic by sailing ship. Successful colonisers landed where humans already existed and trade was easy.
It doesn't work on interstellar, never mind intergalactic time scales, because nothing stays stable for that long. Not hardware, software, politics, or culture.
Nor, on slightly longer periods, biology. On much longer periods, geology, and eventually astrophysics, because stars change, and planetary systems aren't unconditionally stable.
So a colonising wave from a unified culture is an incredibly unlikely thing, not at all an obvious necessity.
This would mean bootstrapping current advanced manufacturing technologies to a new planet. You would need so many different tools to do that that I seriously doubt we are currently capable of making it compact enough to be sent into deep space with current technologies. We're currently sending at most small capsules into deep space, my gut tells me that for self-contained factories we would need to send something in the size order of a skyscraper.
Yea, the paper discusses a probe with a mass between 50g and 500kg using a diamondoid data storage medium that holds ~6,250 exabytes per gram. Plenty of room for any blueprints you want to include, up to and including a planet full of humans. If not actually today’s tech, it is but a few years into the future. I’m sure my next computer will have a few hundred grams of that diamondoid storage.
Blueprints are the last of my concerns. What I think will be hard to do is to implement a full supply chain into a single space-travelling factory, including sourcing and refining of raw materials. But, regarding the blueprints, it now occurs to me that our "recipes" are made to work on our planet. Another one may lack some "ingredients" or have atmospheric conditions that could mess with the chemical reactions we use here. So we would need an advanced AI able to adapt production to the environment it finds.
Sourcing and refining materials are just blueprints. Adaptations for other environments are just different blueprints.
But honestly most of the work would be done in vacuum. Skip the planets, build the daughter probes out of asteroids. Most systems should have plenty of easily accessible material even if they don’t have a prominent asteroid belt, even if the probe has to scavenge the system’s oort cloud.
I don't see it. Hobby projects can use a VPN tunnel to make a data center from local equipment. Real projects that choose colocation have uptime requirements that simply can't be met by random consumer hardware. The venn diagrams don't intersect.
There's no middle ground where you try to run a real business on old laptops. That's insane. You either keep things small/hobby and stay simple, or graduate to production-grade equipment once you have real requirements.
The middle ground, taking on production colocation problems plus the unreliability of random hardware, sounds like the worst of both worlds. There are both simpler and more robust options.
Initially I believe Google was known for getting unreliable hardware with good software to manage it (a single laptop probably won't cut it, but a bunch of laptops scattered around the globe could be interesting -- when you grow things fail all the time anyways).
They aren't targeting no one (and looks like they aren't at all).
Just do the math: for a measly €2000 a month, a salary of a cashier in Amsterdam, you already need to have 285 clients - and this is without taxes and revenue.
My understanding is that Austria was the Germans who ended up running a multi ethnic empire ruling over Slavs also, while Germany was the Germans who didn’t do that.
Then once the Austrian empire fell apart only the German part was left in “Austria” and it basically has no reason to be separate from Germany anymore because the Slavs are no longer part of the same territory and there is no “top of the caste system” benefit anymore to the Germans there.
Feel free to chime in if I’m wrong. I’ve found topics related to Germany to be hard to actually figure out due to a lot of morality noise that gets injected into those topics (the Nazis wanted to unify and Nazis bad etc stuff like that).
Both Prussia and Austria made their names outside of the historically German lands and ruled over non-Germans. It was in the end Prussia that unified Germany in 1871 having previously defeated Austria. Both settlements following WWs 1 and 2 forbade Germany from unifying with Austria. And now in any case there is no appetite for it regardless of legality.
I can say that the German spoken in Germany and the one spoken in Austria aren't exactly the same; there are some regional differences. But they still understand each other without issues.
You would have issues with providing the reliability levels (read: SLA) that we come to expect from data centers. But, if there are enough services that we don't care about if they go down for a few hours, this could be doable. It still relies on the assumption that we got enough services to justify the effort though. It is way more realistic if you set up your own homelab and provide services to your family, under the caveat that they may go offline every now and then.
I doubt it, there are data centers with several decades of 100% uptime.
People often think of the large cloud providers when they think of data centers -- but their data centers are typically mediocre in terms of redundancy and uptime. Their strategy is generally to have less infrastructure redundancy and rely on software failover... e.g. failover to another AZ
Well, I highly doubt that the kind of rockets they are developing for Lunar and Mars missions will be mich better, if any better at all, than current ballistic missiles armies around the world already have. Those space rockets are huge and meant to more or less safely carry people over a long distance in space. Warheads are meant to carry explosives while also being hard to detect or stop. I'm no rocket scientist, but I believe that huge space rockets would defeat the purpose, as they would consume a lot of fuel for nothing, while also being much easier to spot and stopped by shooting something at them.
So I think the opposite: we are way past the point of space exploration being directly useful for weapons.
The point now isn’t having better rockets for (ballistic) missiles, since satellites became a thing the game has been infrastructure. Future (hypothetical) missions to the moon and mars might not be for military research purposes directly, but the infrastructure that both needs to be and now can be set up to support those missions will absolutely be co-opted for military purposes.
The race is now to bootstrap your nation’s permanent presence in space, because at the moment there is a first mover opportunity for what is slowly but surely becoming just another frontier for economics, geopolitics, etc. to play out over (granted this is already happening, I suppose I’m talking about a step change in scale).
reply