Nvidia blasts proposals for chip backdoors as US considers “kill switch”
- Thread starter JournalBot
- Start date
Now dont all you who accused China of being paranoid feel a bit foolish? Yes the Trump admin tried to get backdoors put in, and China was right to demand answers.
Upvote
16
(34
/
-18)
How would you even operate such a "kill switch"? GPUs are not directly connected to the internet, but rather sit in datacenters, on internal-only networks, running the client's software. How would a kill command make it from the US, through the Great Firewall, the client company's application firewall, hop to an air-gapped server cluster, be correctly received and executed by the client company's in-house software, AND THEN be passed on to the GPU? Same question for the location tracking.
Upvote
111
(115
/
-4)
Cisco has an extremely complicated procedure for setting the operating region on the first universal wireless access point on a wireless LAN controller. It relies on GPS. Of course, if you're a nation state-sized actor with access to a GPS simulator, you can still set the operating region to any region.
I'd think you could do the same for just about any scheme that attempts to activate/deactivate data center hardware based on physical location as determined from a GNSS.
Edit: assuming you're unwilling to share the P(Y) code keys with a foreign adversary.
I'd think you could do the same for just about any scheme that attempts to activate/deactivate data center hardware based on physical location as determined from a GNSS.
Edit: assuming you're unwilling to share the P(Y) code keys with a foreign adversary.
Upvote
34
(34
/
0)
https://aws.amazon.com/blogs/securi...on-bloomberg-businessweeks-erroneous-article/
"Today, Bloomberg BusinessWeek published a story claiming that AWS was aware of modified hardware or malicious chips in SuperMicro motherboards in Elemental Media’s hardware at the time Amazon acquired Elemental in 2015, and that Amazon was aware of modified hardware or chips in AWS’s China Region.
As we shared with Bloomberg BusinessWeek multiple times over the last couple months, this is untrue. At no time, past or present, have we ever found any issues relating to modified hardware or malicious chips in SuperMicro motherboards in any Elemental or Amazon systems. Nor have we engaged in an investigation with the government."
IOW, total bullshit.
Upvote
56
(56
/
0)
I would imagine the hardware would be designed in such a way that, if it encounters e.g. very specific bytestrings during operation, performance degrades (most likely some very minor memory corruption to keep it subtle; the main goal is to prevent models from training correctly, and a model that seemingly refuses to get better and wastes a year of a team's resources trying to figure out the issue is more effective sabotage than suddenly giving everyone bricks).
The bytestring is never present in any corpus before the 'trigger' is pulled; it is carefully-chosen nonsense. The 'trigger' is pulled by publishing the bytestring in multiple public places. It is eventually scraped, included in training data, and initiates its poisoning effect.
Upvote
27
(38
/
-11)
Can you imagine Trump with an AI kill switch? He'd be flicking it on and off every other week just to piss people off.
Upvote
99
(99
/
0)
It would probably be in conjunction with "location verification" where the chip won't operate without signed authorization from the verification server. If it can't contact that server or doesn't get reverification in N hours (or days or weeks), it won't run.
https://www.iaps.ai/research/location-verification-for-ai-chips
Upvote
27
(28
/
-1)
"an overreaction that would irreparably harm America's economic and national security interests," he wrote."
No need to worry about that, as by now you don't have any feet left to shoot with a bazooka.
Edit:
"This is no time to depart from that winning formula."
This administration is already holding so many beers, that adding another one to also prove THIS wrong will barely be noticed.
No need to worry about that, as by now you don't have any feet left to shoot with a bazooka.
Edit:
"This is no time to depart from that winning formula."
This administration is already holding so many beers, that adding another one to also prove THIS wrong will barely be noticed.
Last edited:
Upvote
32
(34
/
-2)
Technical challenge to be sure, but not something impossible to achieve. Anyone remember what stuxnet and flame were able to do?
Upvote
27
(30
/
-3)
Good luck getting a GPS signal in a datacenter. They usually charge about 2K/mo per run connecting to their antenna if you truly need a reliable PTP time source. Doubt they'd even have the infrastructure available to hook up every GPU.
Upvote
-6
(0
/
-6)
Upvote
88
(88
/
0)
I’d like to see these back doors implemented in guns. Geolocation prevents them from being fired around schools, churches, government buildings, etc. This will stop “unauthorized use”.
/s since it’s ’Merica and you’ll never take my arms away or limit them at all!
/s since it’s ’Merica and you’ll never take my arms away or limit them at all!
Upvote
41
(47
/
-6)
Which means you can't run internal-only-network servers and they have to "go outside to touch grass". While theoretically you could arrange something like that for, say, AWS datacenter, where your workload is forcibly aborted every N whatever (reverification period), server wiped, empty image loaded, network updated to let GPU "go outside", then everything is re-loaded back, it'd be a big pain in the ass.
And for cases where companies want to have own internal-only AI runners this would require physical server re-connect. Giant pain in the ass.
At this point companies might go with more efficient Chinese AI software that runs on older hardware that doesn't demand random internet connectivity (versus current "oh it's slow? let's throw newer hardware at it"). Which would be ironic.
Upvote
18
(18
/
0)
Hey ... remeber that time when Apple got dragged into a court case that had nothing to do with Apple and the Feds attempted to force Apple to create a backdoor to iOS after a shooting/attempted bombing all because the perpetrator had an iPhone during the situation? Apple then refused due to the same arguments NVidia is now making;
https://meincmagazine.com/tech-policy...lp-fbi-unlock-san-bernardino-shooters-iphone/
but then a few years later decided that a limited access backdoor would be a good idea all in the name of the children:
https://meincmagazine.com/tech-policy...ll-scan-photos-for-child-sexual-abuse-images/
... until Apple got a shit ton of pushback from its customer base where Apple reversed course before they added the limited backdoor (likely prompted from lots of closed door conversations after San Bernardino wiht the Feds until some shady deal was made);
https://meincmagazine.com/tech-policy...n-csam-scanning-tool-more-controversy-ensues/
... so a couple of takeaways;
(1) looks like the Feds still have not dropped their "we need a backdoor into technology to spy on everyone on the planet" mantra after a decade ...
(2) and how long until it comes out that the Feds and Nvidia are colluding behind closed doors for a "limited access" option that gets reversed when its leaked to the general public ? /rhetorical
On the upside that would benefit AMD business since people would not buy any hardware from NVidia knowing that they allowed backdoor access of any sort.
Then again I am surprised after all of the Tech CEOs went brown-nosing after the election that said tech companies are not bending over to inject backdoors at the bequest of the Orange TACO.
Upvote
14
(19
/
-5)
They could require active software checkins by the drivers. If the driver can't communicate with NVidia to get a signed authorization key for continued usage, which it then uploads into the hardware, that hardware will shut down.
They could build GPS into the chips, and the location could be part of the signing request. If there's no location, there's no signature.
GPS might be spoofable, but that would not be easy.
Upvote
-10
(2
/
-12)
Any time someone says "It would be easy to do this," in a computer security context, you can virtually guarantee they haven't discussed countermeasures with a security consultant.
There is no easy way to guarantee that only the right people are accessing a piece of software / hardware. The easier a method seems, the higher the chance that it'll be trivially broken. And we're talking about nation-states here, in a context where China views AI leadership as essential and is pouring huge amounts of money into research and deployment.
Upvote
44
(44
/
0)
Taking this even further, a state actor or Baidu-sized organization could easily customize drivers or firmware to basically disable whatever code the hardware needs to use the Internet to receive the kill signal.
All that aside, as comes up in encryption backdoor discussions all the time, a kill switch would only snag the low-hanging fruit, or the "stupidest criminals." It wouldn't catch well-funded or state actors.
So, not useless, but extremely limited in effectiveness.
Upvote
17
(18
/
-1)
There are multiple potential ways to avoid the driver check-in problem, ranging from generating false approval to using test drivers that don't require the same check, to DLL hacking to disable the check, to DLL hacking that allows non-Chinese drivers to be installed on Chinese products. Don't forget that in order to enforce this sort of thing, Nvidia either has to distribute that driver globally to every user or force Chinese people to download China-specific drivers.
If Nvidia starts distributing drivers with required location checks, hackers will promptly find ways to disable said location checks. China, I'm certain, would happily pay for a handful of people to fly to San Francisco or LA once a month, download unencumbered drivers at the airport, and then fly home.
GPS spoofing is already 100% possible. https://www.okta.com/identity-101/gps-spoofing/
"They could build GPS into the chips, and the location could be part of the signing request. If there's no location, there's no signature."
Building a GPS radio into Chinese-only processors would drive more black-market demand, in much the same way that GPU shipments into Singapore exploded once export to China was restricted.
All such measures become taxes on the honest while incentivizing the guilty.
Upvote
20
(23
/
-3)
On a serious note, how do you all think about your chain of trust? Did you have any recommendations for a securely produced and provisioned TPM that I should consider purchasing? Thanks.
Upvote
-1
(0
/
-1)
I kinda wonder if the CAC are actually performing entrapment here.
Regulator in China: "We're concerned that nVidia may have been compelled to do something impractical, counter-productive, and really quite dumb"
nVidia: "That's absurd. Our business model requires us to meet customer expectations to not do such impractical, counter-productive, and really quite dumb things"
US administration: "Hold up! Impractical, counter-productie, and really quite dumb are some of our favorite things. We didn't realize it was an option here!"
Upvote
26
(27
/
-1)
You put the GPS into an appliance in the data center rather than in the individual GPUs and use an encrypted / authenticated version of ping / ntp to ensure the appliance is in physical proximity to the GPUs. That appliance is the only device that needs to be connected back to Nvidia HQ.
GPU asks the appliance for authorization. Appliance gets location from GPS. Appliance asks Nvidia for permission to operate at that location. Appliance distributes authorizations to GPUs as needed.
Essentially every chip arrives dead until authorized. And loses that authorization upon reset or loss of power.
But you can still spoof the GPS.
Last edited:
Upvote
-5
(2
/
-7)
Uneducated, illiterate GOP pol demands the implementation of failed tech to beat the Reds; news at 11!
Upvote
15
(16
/
-1)
I'm not sure how you'd do it secretly; but the usual mechanism is for the system to demand some kind of license activation and fail closed if it doesn't like the result. That's basically how they handle vGPU features.
Given that the newer cards are all 'hardware root of trust' and have their own onboard processing and firmware it would be viable in principle to move the license verification out of relatively easily tampered drivers and onto the card itself. The main limitation would be what the end user would put up with: Nvidia strongly encourages you to set up a 'cloud' licensing instance with them; but defers to necessity by offering locally hostable images and node-locked license files; which obviously limits the ability to kill something quickly.
You could probably also use the console game/blu-ray trick where updates are forced on offline/uncooperative devices by baking them into new releases, by adding a validation step and kill lists to new CUDA releases; though that would only hit people who actually need to update for some reason.
Tom Cotton is a cynical carnival barker who doesn't care if it's possible so long as it's possible to advance a bill whose name has been tortured into a twee acronym about it; but when you've got cryptographic bootloader lockdown what you can do is mostly limited by your willingness to upset the customer; so it's not totally science fiction.
Upvote
9
(9
/
0)
Missing Minute
Wise, Aged Ars Veteran
There would need to be some way to require an always on connection that needs to get a signed ping back in >x ms. I'm not sure that's doable in a way that can't be circumvented.
Upvote
-2
(0
/
-2)
Hmmmm:
The old fashioned way was to slap a security classification on tech and control it that way. This worked when the only real customer was Government. Those days are long gone. Today, the cost of technology is now so high that if it's cut-off from the mass market it won't exist in the first place. Government can't afford to build it, it needs the mass market to pay for it.
Government tech interests have involved riding the coat tails of commercial developments for decades now. Indeed, the DoD has been driving that for decades with its insistence on open standards being used to build military kit. Long gone are the days when government-funded developments were driving the cutting edge (at least when it comes to silicon chips). The only thing government can control is knowledge of how government uses tech that is freely purchasable, and some specific combinations of components (e.g. ITAR'd sub-assemblies built out of non-ITAR'd components like Intel CPUs, Xilinx FPGAs).
A good example is Serial Rapid IO vs Ethernet. Serial Rapid IO (when new) beat out Ethernet (as it was), and got used in various standards like OpenVPX (which is what is used in a lot of military Radar systems, etc). Now it's all Ethernet. The cost of developing performance-competitive switch chips for Serial Rapid IO far exceeeded the value of the Serial Rapid IO market, whereas Ethernet has got a $untold-billions per year commercial / industrial market to fund its development. Good luck controlling the Ethernet market...
- Gov wants control of tech
- There is a free market for tech within the USA and much of the rest of the world
The old fashioned way was to slap a security classification on tech and control it that way. This worked when the only real customer was Government. Those days are long gone. Today, the cost of technology is now so high that if it's cut-off from the mass market it won't exist in the first place. Government can't afford to build it, it needs the mass market to pay for it.
Government tech interests have involved riding the coat tails of commercial developments for decades now. Indeed, the DoD has been driving that for decades with its insistence on open standards being used to build military kit. Long gone are the days when government-funded developments were driving the cutting edge (at least when it comes to silicon chips). The only thing government can control is knowledge of how government uses tech that is freely purchasable, and some specific combinations of components (e.g. ITAR'd sub-assemblies built out of non-ITAR'd components like Intel CPUs, Xilinx FPGAs).
A good example is Serial Rapid IO vs Ethernet. Serial Rapid IO (when new) beat out Ethernet (as it was), and got used in various standards like OpenVPX (which is what is used in a lot of military Radar systems, etc). Now it's all Ethernet. The cost of developing performance-competitive switch chips for Serial Rapid IO far exceeeded the value of the Serial Rapid IO market, whereas Ethernet has got a $untold-billions per year commercial / industrial market to fund its development. Good luck controlling the Ethernet market...
Upvote
7
(8
/
-1)
To address concerns that chips might fall into the “wrong hands”, the following seems like a minimally “intrusive” possibility.
Give every chip a “batch ID”, that requires physical possession to access.
Send all chips with that batch ID to the same country.
If, for example, 5,000 chips with the same batch ID are sold to China, and a Russian-associated device is found to contain one, it should be pretty clear evidence that the chip came from China.
Obviously, this only detects security agreement violations, and has the limitation of requiring physical possession, but it would do something, while being non-intrusive.
Even if a particular "flavor" of processor (or any chip) were sold only to country XYZ, it might be that several companies in XYZ buy these chips - in which case, a batch ID could identify the company receiving each chip.
Give every chip a “batch ID”, that requires physical possession to access.
Send all chips with that batch ID to the same country.
If, for example, 5,000 chips with the same batch ID are sold to China, and a Russian-associated device is found to contain one, it should be pretty clear evidence that the chip came from China.
Obviously, this only detects security agreement violations, and has the limitation of requiring physical possession, but it would do something, while being non-intrusive.
Even if a particular "flavor" of processor (or any chip) were sold only to country XYZ, it might be that several companies in XYZ buy these chips - in which case, a batch ID could identify the company receiving each chip.
Upvote
-5
(2
/
-7)
Honestly this sounds like a Tuesday for a well funded intelligence agency.
Israel remotely blowing up pagers en masse.
Stuxnet
Russia putting a hard to detect EM listening device in a seal back in 1945.
https://en.m.wikipedia.org/wiki/The_Thing_(listening_device)
Countries already infiltrate each others computer networks on a routine basis. It apparently even concerned China enough to make a fuss over it. Doesn't seem like an impossible task for a motivated entity.
Last edited:
Upvote
16
(16
/
0)
Please please please ! Someone, anyone, give Trump an AI kill switch to play with ASAP !
Upvote
15
(15
/
0)
I don't think anything you said above is wrong, but I think it's relevant to draw a distinction between what nation-states are willing to do if they consider an advantage critical to national security versus what the consumer or commercial markets create in terms of hacks and workarounds.
Nation-states have both the funds and the incentive to create workarounds that commercial and consumer users either cannot or will not bankroll for various reasons related to difficulty and legal liability. From the laptop interception program that Snowden detailed over a decade ago, to Stuxnet, to Israel's attack on Hezbollah pagers -- some of which were subjected to X-ray inspection before deployment -- nation-states have proven willing to engage in espionage and hacking on a scale that was literal science fiction just a few decades ago.
Forcing backdoors into Nvidia products would light a fire under the Chinese to build their own competitive hardware and incentivize them to crack the security stack on our own. It would not result in anything being made particularly safer. The bootloader lockdown you refer to might work when the customer is a Fortune 500 company fundamentally uninterested in provoking an audit, but it's less likely to work when China knows it's breaking the law just by owning the hardware in the first place.
Upvote
15
(15
/
0)
To be even more specific - it's been proven to be false, yet Bloomberg has never recanted their story. And the damage done is shown by the OP going back to the poisoned well.
Upvote
16
(16
/
0)
I'm confused: in which jurisdiction does one sue the Chinese Communist Party? Space Court? Dreamland? The UN?
Upvote
24
(24
/
0)
I can imagine a relatively simpler way for a built-in location-based automatic kill switch, without requiring potentially spoofable GPS or phone-home gimmicks.
An onboard rechargeable coin battery drives a MEMS INS chip integrated into the GPU. The INS chip is calibrated (position offset initialized to zero) at the point of sale, controlled by the manufacturer, local to any given sales region. Customers cannot reinitialize the position, without access to a secret cryptographic key specific to any given batch of chips and internally managed by the manufacturer. Loss of power to the INS instantly triggers a self-destruct (as below) powered by an integrated capacitor.
Thereafter, the INS chip tracks and integrates any large/fast (e.g. faster than walking pace) changes in position. When or if the cumulative change in position exceeds a threshold, the chip automatically wipes most of its microcode and firmware, and becomes permanently inoperable until subjected to a factory reset using the aforementioned secret key (which is permanently burned into the chip).
An onboard rechargeable coin battery drives a MEMS INS chip integrated into the GPU. The INS chip is calibrated (position offset initialized to zero) at the point of sale, controlled by the manufacturer, local to any given sales region. Customers cannot reinitialize the position, without access to a secret cryptographic key specific to any given batch of chips and internally managed by the manufacturer. Loss of power to the INS instantly triggers a self-destruct (as below) powered by an integrated capacitor.
Thereafter, the INS chip tracks and integrates any large/fast (e.g. faster than walking pace) changes in position. When or if the cumulative change in position exceeds a threshold, the chip automatically wipes most of its microcode and firmware, and becomes permanently inoperable until subjected to a factory reset using the aforementioned secret key (which is permanently burned into the chip).
Upvote
0
(5
/
-5)
You forget that the average Tech IQ of the average politician doesn't get above "imbecile". SLIGHTLY brighter than an idiot, but not as mentally adept as a moron.
This scale should also be noted for the fact I call politicians morons all the time, and that's kind of a compliment since there are two levels below that I could probably rightly call them instead.
Upvote
12
(12
/
0)