Do AI-designed proteins create a biosecurity vulnerability?
- Thread starter JournalBot
- Start date
I can just see it.
"But how did this happen?"
"I'm sorry, Ms. Jones, but your DNA wasn't patched in time with the latest monthly update. We did send out a notice that Chromosome 12 had a biological vulnerability that was addressed in the November Helixical Update. We can't do much if you don't apply the ribosal updates in a timely manner."
"But surely there's something you can do?"
"I'm afraid not. But since you did enroll in our Insider Program, your kids should have this one mitigated until they also can get patched. Please bring them by tomorrow, and we'll see that they receive it. Outside of that, it's been a pleasure being your doctor, and I look forward to working with the next generation."
"But how did this happen?"
"I'm sorry, Ms. Jones, but your DNA wasn't patched in time with the latest monthly update. We did send out a notice that Chromosome 12 had a biological vulnerability that was addressed in the November Helixical Update. We can't do much if you don't apply the ribosal updates in a timely manner."
"But surely there's something you can do?"
"I'm afraid not. But since you did enroll in our Insider Program, your kids should have this one mitigated until they also can get patched. Please bring them by tomorrow, and we'll see that they receive it. Outside of that, it's been a pleasure being your doctor, and I look forward to working with the next generation."
Upvote
67
(74
/
-7)
That is the key piece of information here at least for the present.
Your odds of getting a working one would not be very high and if you had to order a bunch of them it would get very expensive and odds are decent that sooner or later one of them ends up close enough to trigger the scrutiny at which point you would likely get caught.
I personally don’t consider this a current risk. I would consider it equivalent to an encryption algorithm that is fine today, but will likely be insecure in the future so it is good to update the algorithm now before it becomes a real problem.
Perhaps a more immediate risk is the methods to make DNA are well publicized research. Sure it will be more expensive and slower to do it based on published research rather than ordering the DNA from a vendor, but a decently outfitted molecular biology lab could likely do this and the various reagents etc. have multiple uses so I’m not sure it would be easy to catch someone gathering the materials to do so.
Upvote
41
(42
/
-1)
As the article states, I wouldn't be too concerned about AI-designed protein toxins for now. There are plenty of nasty enough biological (and chemical) weapons in the world that are far more readily available.
Hell, I'd be very surprised if there aren't stocks of smallpox virus around other than at the CDC and Novosibirsk. Its genome sequence is publicly available and could assembled by anyone with graduate level molecular biology skills and an oligo synthesiser.
(a retired biochemist writes)
Hell, I'd be very surprised if there aren't stocks of smallpox virus around other than at the CDC and Novosibirsk. Its genome sequence is publicly available and could assembled by anyone with graduate level molecular biology skills and an oligo synthesiser.
(a retired biochemist writes)
Upvote
71
(71
/
0)
This part, cheaper to buy things that already exist for anyone in the market for such weapons
Maybe, but I would think the problem with weaponizing smallpox is it’s so virulent and by this point so fatal to so much of the population that the only people who would want to use it are literal death cults, everyone else would have trouble vaccinating their own population fast enough for it not to destroy them as well
Upvote
17
(17
/
0)
Sure. You'd have to be an absolute madman to use it as a weapon and reintroduce it to the world, but that doesn't offer much reassurance does it?
Upvote
67
(67
/
0)
I would say that it is impossible to detect potentially dangerous proteins with current crop of algorithmic knowledge. Also that means there is hard to create them in first place as well. Yes, there have been several breakthroughs, but we might be 10 years away from any considerable manipulation capabilities.
That doesn't mean there is no danger. But nature so far has done us way more damage with dice rolls of protein combinations that we have figured out what is deadly and what is not.
That doesn't mean there is no danger. But nature so far has done us way more damage with dice rolls of protein combinations that we have figured out what is deadly and what is not.
Upvote
6
(6
/
0)
I mean, some I guess? It reduces the number of actors who would be trying it at least. But yeah, I get your point
Upvote
4
(4
/
0)
As of now, I'm more worried about current toxins. Having a background in commercial canning, botulism toxin is an ingrained fear. Fortunately the organism that produces it requires a lack of oxygen (hence it's associated with under-processed canned goods - you have to pressure cook hot enough to kill the spores). But given all the accessible biotechnology around, It can't be too tough to splice the sequence into something that can more readily be grown.
Upvote
19
(19
/
0)
ColdWetDog
Ars Legatus Legionis
If you were really planning on doing this, you would probably just go ahead and buy a DNA synthesizer (there are a bunch of them). More steps, more money but if you are in the mood to create several thousand sequences it would be the way I would go.
Upvote
18
(18
/
0)
ColdWetDog
Ars Legatus Legionis
If you are at all patient, all you have to do is wait until we trash the economy and environment all buy ourselves.
"Never interfere with an enemy when he is in the process of destroying himself." -- Sun Tzu
Upvote
30
(30
/
0)
Seems like an opportune moment to suggest the Starfish trilogy from Peter Watts.
This sort of sets up one of the neat background stories from the series.
This sort of sets up one of the neat background stories from the series.
Upvote
4
(4
/
0)
"Shareholders wanted to see results so we fast-tracked this AI-designed protein, sorry about the zombie apocalypse, but some of them earned a cabin on the space-station where I'm sure they'll have learned my lesson."
Upvote
9
(9
/
0)
Upvote
-3
(0
/
-3)
There's a thought, the weight of which is shockingly difficult to understate.
Upvote
0
(0
/
0)
I just don’t see the potential for great harm here. This doesn’t qualify for weapon of mass destruction. A protein is not on its own self replicating, so like a chemical bomb can only be as big as the amount of dangerous material you make. If we want to destroy an entire community, then we are talking about putting whatever it is in high enough concentration across a volume the size of a municipal water supply. The expense would be astronomical, and frankly if you were going to do that, you don’t need an AI designed toxin, just use ricin or bee venom or peanut allergen or any of hundreds of natural products whose behavior is already well understood.
Otherwise what we are talking about here is a very overpriced labor intensive weapon to harm a small handful of people. If we are worried about that, then I think that perhaps we might reexamine 2nd amendment protections because a massacre like that happens on US soil a couple times per day with conventional weapons.
The real threat here is you might be able to evade standard CSI techniques and escape culpability. This really seems like something that only a foreign intelligence / assassination squad could afford / want to do.
Otherwise what we are talking about here is a very overpriced labor intensive weapon to harm a small handful of people. If we are worried about that, then I think that perhaps we might reexamine 2nd amendment protections because a massacre like that happens on US soil a couple times per day with conventional weapons.
The real threat here is you might be able to evade standard CSI techniques and escape culpability. This really seems like something that only a foreign intelligence / assassination squad could afford / want to do.
Last edited:
Upvote
5
(7
/
-2)
How does this even happen? Is it also a prion which miraculously changes the sequence of endogenous proteins to self replicate? To paraphrase a historical meme:
1) AI designed toxin
2) ???
3) world destruction!
2 is an important step, because we’ve seen toxins before over billions of years of years, and whether they are AI designed or naturally evolved doesn’t matter for what they do. I think this is inherently very, very hard / impossible and there is little reason to believe that AI will succeed where legions of state actors and Bond villains have failed.
Upvote
5
(6
/
-1)
And a high pH. But against that it is still the most poisonous substance known to man, with an LD50 on the order of nanogrammes.
I wonder how long it will hold that record?
Upvote
6
(6
/
0)
Given the progress of AlphaFold and similar networks "eventually" may be measureable in single-digit years.
Upvote
1
(2
/
-1)
This is the equivalent of the crypto nerd's imagination regarding security. Yes, it is technically possible to kill a bunch of people with a carefully engineered toxin that carefully skirts the safeguards we have in place. However, we have much easier ways to do this, like Botulinum from poorly canned foods or guns.
Upvote
12
(12
/
0)
An aside, but my first summer lab job as a biochem undergrad was purifying the hemolytic peptide melittin from bee venom. The lab was studying how it interacted with lipid vesicles (simulating a cell membrane) by H/D-exchange NMR.
A few kg of freeze dried bees would arrive from the USSR and I'd grind them up and use various chromatographic procedures to purify the melittin. We each could only do the extraction a handful of times as the risk of developing an allergy to the toxin was considered quite high. An interesting project, though.
Upvote
15
(15
/
0)
ColdWetDog
Ars Legatus Legionis
Actually I think decades is a better guess. It is still really hard to do, AI or no AI.
This is really proof of concept work to come up with techniques and ideas to screen potentially dangerous biomaterials. It has it's basis in the Asilomar Conference of 1975 (50th Anniversary this year, man I'm getting old). The idea being think before you jump.
Upvote
1
(2
/
-1)
Upvote
-3
(0
/
-3)
So, on the 'there are no technological solutions to social problems' beat, it's important to note that most major powers already have and long have had the ability to engineer biological and chemical weapons. We don't do it, we've stopped doing it, because of treaties and collective agreement that 'it's a bad idea'. Now you do get rogue actors 'what's the point of having nuclear weapons if you don't use them' but if we aren't stupid enough to elect them, having easier ways to make the things we all agree to not use isn't really a threat unless you not only ease the design effort but also the production effort such that they can work down to non-state actors, and I suspect that's not soon to happen outside of some billionaire who is granted permission to have enough money to build a BSL-4 facility and hire the PhDs needed to work it. I think we all have a mental list of who those are most likely to be.
So yeah, this sounds scary, but I suspect existing international norms on the use of such weapons are the real thing that matters.
So yeah, this sounds scary, but I suspect existing international norms on the use of such weapons are the real thing that matters.
Upvote
3
(4
/
-1)
I'm grateful that there are systems in place to stop this kind of thing, but they probably need to be ramped up and given teeth.
Upvote
1
(1
/
0)
My wife works in an adjacent field. The barriers even if you have a sample of “bad thing” dna are many.
- Growing the dna one received requires a medium or bio reactor
- Running these are not trivial.
- Distilling down bad thing having run your bio reactor is not easy. Likely easier to just make some dodgy duck liver pate and can it poorly
Upvote
4
(4
/
0)
Upvote
-3
(0
/
-3)
This doesn’t negate your broader point, but prion diseases (caused by proteins termed “proteinaceous infectious particles ”) are an interesting counter example to the claim that proteins are not self replicating.
One has to wonder how many dangerous proteins are actually being screened for - the 72 the researchers tested for seem woefully low.
I agree though that this doesn’t currently seem like a currently viable threat vector.
Last edited:
Upvote
1
(1
/
0)
President:
"Can't we, like, ban molecules and atoms and stuff? Suck all the molecules out of people's body with a long tube? I've talked to <egghead> here in our staff, and I think we can announce something next month, is it not, Jimmy?"
Upvote
0
(0
/
0)
Potential biological warfare trolling: start re-introducing vaccination against smallpox in your country, watch adversaries waste millions on intelligence trying to figure out whether you're planning a viral attack.
Upvote
8
(8
/
0)
Prions do not really replicate themselves like a virus or bacteria. A prion is a pathogenic form of a protein that makes the normal protein already present change to the pathogenic form. The pathogenic forms stick to each other and disrupt cell machinery. I always thought of it as analogous to a crystallization event.
It is not making copies of itself from amino acids or peptides.
Upvote
6
(6
/
0)
ColdWetDog
Ars Legatus Legionis
Nah, wouldn;t work (but your ideas intrigue me and I would like to subscribe to your newsletter). Any country even remotely interested in doing this wouldn't care a whole lot about it's general population. We got robots, right?
Upvote
0
(0
/
0)
some interesting overlap with concerns on 'mirror' life (i.e. left-handed molecules)...
https://news.stanford.edu/stories/2024/12/potential-risks-of-mirror-life
https://news.stanford.edu/stories/2024/12/potential-risks-of-mirror-life
Upvote
0
(0
/
0)
The Sheep Look Up
Ars Praetorian
... most of them? The vacuum decay clock doesn't so much tick; it's got one hand, one mark, and a revolution is one Planck second.
Upvote
3
(3
/
0)
I agree that this is much more of a theoretical threat than an actual one, and making data bases smarter at recognizing the threat here is not going to help at all. The one actual use for this technology I can see is for use by very sophisticated organizations in assassinations. with total deniability Arguably an analogue of something like ricin might be undetectable especially if novel, and the healthy 50 year old just dropped dead with what looked suspiciously like ricin poisoning for no reason whatsoever. There are two big however. First the Livchenko and other poisonings by the FSB show that Russia at least actually wants people to know when it is responsible, with only the most transparent of fig leaves. Second the kind of organization whichh would do this will do the whole process soup to nuts internally, so scouring the web for suspect DNA sequences will be totally irrelevant.
Upvote
0
(0
/
0)
Oh man....people have already lived 100+ years which means they were born before much of this "progress". We totally have a lot to worry about. More like them releasing crap into the water/air and them milking us for money like cows. This is the biggest crock of crap I have ever seen. That is all this proves. We have threats to worry about alright...It's THEM. They have been scheming to do this since what....like the 1960s maybe? This all just WREAKS of PONZI scheme. This whole AI thing. Anything and everything musk has touched thus far seems to be exactly that. If it were so great or for the greater good, do they really gotta bill residential utility subscribers for it too? Just absolutely stinks of musk, Nvidia, and the US and China to me.
Upvote
-7
(0
/
-7)