Dozens of Red Hat packages backdoored through its offical NPM channel

Anyone who has downloaded affected Red Hat packages should investigate immediately.

See full article...

 

[stormcrash]

That's scary as shit for an enterprise focused distro like RedHat for their repos to be compromised in a supply chain attack

 

[huskcummerbund]

If you type without rhythm, you won't attract the worm.

 

[pov3rty]

I'm going to lay this at the feet of npm/github. They're not doing enough to address huge fundamental problems with pre/postinstall via package.json.

The fact that any dependency -- anywhere in the entirety of the dependency tree -- can execute arbitrary code upon npm install is batshit crazy. And last I checked and commented over at that repo on github... they're completely dragging their feet.

Fixing npm wouldn't eliminate supply chain in the node ecosystem, but it sure is hell would make it more difficult.

 

[gtrawo]

With all the other Linux exploits and these supply chain attacks … it feels like they are trying wear down all the admins out there!

I think we eventually will get to the point where we just throw our hands up and walk away.

 

[sjl]

Tyop in the headline: “Dozens of Red Hat packages backdoored through its offical NPM channel”. (my emphasis.)

As to the actual article, I have to admit that it doesn’t do anything to improve my opinion of NPM. I’d say that it makes me think worse of the ecosystem, but I’m struggling to see how that could be. Sigh. Can we just go back to the Apple II, please?

 

[ItchyPoo]

Says the bloke making a typo in their own post.....

I gave him credit for trying to be funny

 

[sjl]

Says the bloke making a typo in their own post.....

It is mandatory that any comment pointing out a typo contain a typo itself. I didn’t make the rules; I’m just following them.

 

[zdanee]

Brilliant. I know obscurity is not security, but I'm considering moving onto some weird BSD distro now... Or HURD (okay, that was a joke, ha-ha, fat chance!)

 

[HeadPlug]

Welp, that's June's NPM compromise sorted. See you all next month!

 

[SeanJW]

Brilliant. I know obscurity is not security, but I'm considering moving onto some weird BSD distro now... Or HURD (okay, that was a joke, ha-ha, fat chance!)

Which won't help with NPM issues in the slightest unfortunately.

Edit: And I'd suggest FreeBSD... OpenBSD and NetBSD binary packaging and upgrading are terrible in comparison (I've puttered around with portable Go code to produce Alpine APK, .deb, RPM and various *BSD packages and FreeBSD were the easiest followed by Debian. Alpine uses broken tar and concatenated gzip which can be a pain, and RPM being binary means lots of work)

 

[AdamWill]

Note - I work for Red Hat but this is not an official response, I'm commenting in a personal capacity.

"It’s unclear precisely how the threat actor took control of the namespace"

I don't think that's right. It's known how they compromised the NPM namespace - by pushing malicious commits to the GitHub repos and allowing existing GitHub Actions workflows to do the job. What isn't known (presumably somebody at RH knows or can figure it out, but that somebody isn't me) is exactly how the GitHub repos were compromised.

 

[Malvineous]

Brilliant. I know obscurity is not security, but I'm considering moving onto some weird BSD distro now...

Then good news! With the rise of AI models well suited for finding security bugs in software, even obscure systems will soon start having their vulnerabilities published too.

On a serious note, you can look at it like the sky is falling, but consider that various intelligence agencies around the world are known to already collect and exploit these types of vulnerabilities. Getting them discovered and patched is going to make it a lot harder for any government to target people they don't like, which I personally think is a good thing.

 

[reyan]

Tyop in the headline: “Dozens of Red Hat packages backdoored through its offical NPM channel”. (my emphasis.)

As to the actual article, I have to admit that it doesn’t do anything to improve my opinion of NPM. I’d say that it makes me think worse of the ecosystem, but I’m struggling to see how that could be. Sigh. Can we just go back to the Apple II, please?

Legitimate question: how the heck do typos bypass the editors when they aren’t real words in a dictionary? Do they not run a simple spell-check?

I understand there/their/they’re or similar issues, or a grammar problem/confusing wording, but offical is not a real word. (My phone just tried to correct it twice while I wrote this).

Are the articles being written and then proof-read in VIM or something?

 

[reyan]

Says the bloke making a typo in their own post.....

Yes, because we should hold someone banging off a quick comment in the thread (probably from their phone) to the same standards as the professional author supported by professional editors

 

[sjl]

Legitimate question: how the heck do typos bypass the editors when they aren’t real words in a dictionary? Do they not run a simple spell-check?

I understand there/their/they’re or similar issues, or a grammar problem/confusing wording, but offical is not a real word. (My phone just tried to correct it twice while I wrote this).

Are the articles being written and then proof-read in VIM or something?

All sorts of reasons. Time pressure; CMS that doesn't have spell check facilities; somebody's tired and missed a step. People make mistakes. My reaction to high profile mistakes (like this one) is to flag them so that they can be fixed, and move on. Not my monkeys; not my circus; it's not worth making a big fuss over.

Honestly, even the comment that tried to rag on me for my deliberate typo (and my response to said comment) was more attention than the problem deserved. It's been fixed (not in the URL, but I accept that that's a somewhat harder problem than just fixing the HTML). It doesn't need to be hashed over. Can we move on, please?

(What am I saying. This is the Internet. Of course we can't...)

 

[Komarov]

Welp, that's June's NPM compromise sorted. See you all next month!

You mean next week. Sorry, tommorrow. Oops, I meant yesterday.

NPM is an abomination, always was, always will be – because all examples out there nudge javatypeshitscript developerswriters towards downloading unreviewed two-line packages to cater to their lazyness.

 

[randomuser42]

CMS that doesn't have spell check facilities;

I think this is 100% what they're getting at, and honestly I have the same question. My browser has spellcheck, my phone, my word processor, my phone keyboard app, etc. CMS software comes without even a basic spell check?

 

[mpfaff]

All sorts of reasons. Time pressure; CMS that doesn't have spell check facilities; somebody's tired and missed a step. People make mistakes. My reaction to high profile mistakes (like this one) is to flag them so that they can be fixed, and move on. Not my monkeys; not my circus; it's not worth making a big fuss over.

Honestly, even the comment that tried to rag on me for my deliberate typo (and my response to said comment) was more attention than the problem deserved. It's been fixed (not in the URL, but I accept that that's a somewhat harder problem than just fixing the HTML). It doesn't need to be hashed over. Can we move on, please?

(What am I saying. This is the Internet. Of course we can't...)

I don't know, unless the article was originally typed in notepad or something I figure it would have gotten flagged. Even if the CMS didn't have an automatic spell checker, the browser you access it through would.

I really don't mean to pile on, I didn't even notice first pass and it doesn't particularly bother me.

 

[huskcummerbund]

That sentiment hasn't changed, as it's always included "relevant to Windows" as part of it. Conveniently leaving that out doesn't instantly make it untrue, and only children, as well as the people who take children at face value, have ever believed it's bulletproof.

A lot of these folks seem to think the always-on cybersecurity vulnerability built into their cars is bulletproof.

 

[planetary]

Thanks Ars for this article. I hardened my NPM dependency usage because of it. You've had at least one real-world effect from your efforts today, and most likely many more.

 

[ZenBeam]

Thanks Ars for this article. I hardened my NPM dependency usage because of it. You've had at least one real-world effect from your efforts today, and most likely many more.

As someone running Red Hat, with no idea what NPM dependency means, what am I supposed to do?

I haven't updated in like a week, so there's (hopefully a good thing) that.

 

[planetary]

As someone running Red Hat, with no idea what NPM dependency means, what am I supposed to do?

I haven't updated in like a week, so there's (hopefully a good thing) that.

I am no expert, but here's what I did for starters:

npm config set ignore-scripts true

This tells npm not to run any preinstall/postinstall scripts. This is how the attack fires. I only looked at the problem from the point of view of a developer. So sys-admins probably have more problems that I don't know about.

Also I'm avoiding npm calls until this is over, then afterwards I'll re-grab packages and setup a package-lock.json when I'm done so that I have hashes and don't auto-grab the newest thing. If an old package has an advisory, or I really need something new, I'll update then and not before.

Anyway, I'm new to all of this. Others here will have far better advice than I can provide.

 

[ZenBeam]

Thank you. I'll run that tomorrow or whenever I'm back on the system.

 

[Secondfloor]

Legitimate question: how the heck do typos bypass the editors when they aren’t real words in a dictionary? Do they not run a simple spell-check?

I understand there/their/they’re or similar issues, or a grammar problem/confusing wording, but offical is not a real word. (My phone just tried to correct it twice while I wrote this).

Are the articles being written and then proof-read in VIM or something?

The position of copy editor of web content was eliminated well over a decade ago. Workflows are now publish to CMS, a manager gives it a look and hits approve.

https://www.davidagnew.com/2024/05/25/where-have-all-the-copy-editors-gone/

 

[huskcummerbund]

As someone running Red Hat, with no idea what NPM dependency means, what am I supposed to do?

I haven't updated in like a week, so there's (hopefully a good thing) that.

Chances are you're not running any of the redhat-cloud-services packages. So you should be fine. Also,

“The packages are strictly limited to internal development, and the malicious code was never published for customer consumption via the console.redhat.com system,” the email said. “While our investigation is ongoing, we have not identified any impact to customer or partner environments or Red Hat production systems.”

 

[sjl]

This tells npm not to run any preinstall/postinstall scripts. This is how the attack fires. I only looked at the problem from the point of view of a developer. So sys-admins probably have more problems that I don't know about.

The problem with that is that in some legitimate packages, there's a need to run scripts to configure Stuff(tm), or validate linkages to other things, or check prerequisites that the packaging system isn't designed to handle. Disabling scripts entirely would break such packages. This then creates headaches for the sysadmins who would have to figure out what's broken and why, and then fix it. (Been there. Done that. Still carrying the scar tissue.)

Which doesn't mean that your solution is wrong, mind. Not using arbitrary scripts to achieve such ends is a desirable goal. It only means that it's only part of the solution; another part is fixing the packaging system so that such pre-checks and post-installation configuration can be done without including something that's Turing complete.

And while I'm wishing for the practically impossible, I'd also like the lottery numbers for the next big draw, please. (Sigh.)

 

[pseudobscura]

So ... more like Black Hat, then?

 

[Albino_Boo]

All sorts of reasons. Time pressure; CMS that doesn't have spell check facilities; somebody's tired and missed a step. People make mistakes. My reaction to high profile mistakes (like this one) is to flag them so that they can be fixed, and move on. Not my monkeys; not my circus; it's not worth making a big fuss over.

Honestly, even the comment that tried to rag on me for my deliberate typo (and my response to said comment) was more attention than the problem deserved. It's been fixed (not in the URL, but I accept that that's a somewhat harder problem than just fixing the HTML). It doesn't need to be hashed over. Can we move on, please?

(What am I saying. This is the Internet. Of course we can't...)

How dare you be reasonable, objective and logical. You should be OUTRAGED, if you continue with this kind of despicable behaviour you should be banned from using all forms of electronic communications.

 

[nijave]

I'm going to lay this at the feet of npm/github. They're not doing enough to address huge fundamental problems with pre/postinstall via package.json.

The fact that any dependency -- anywhere in the entirety of the dependency tree -- can execute arbitrary code upon npm install is batshit crazy. And last I checked and commented over at that repo on github... they're completely dragging their feet.

Fixing npm wouldn't eliminate supply chain in the node ecosystem, but it sure is hell would make it more difficult.

Afaik this is pretty common and wouldn't improve security much anyway. Instead of immediate execution you'd just have to wait 5 more minutes until the code is loaded during automated testing and you're still able to run whatever you want.

 

[nijave]

Chances are you're not running any of the redhat-cloud-services packages. So you should be fine. Also,

“The packages are strictly limited to internal development, and the malicious code was never published for customer consumption via the console.redhat.com system,” the email said. “While our investigation is ongoing, we have not identified any impact to customer or partner environments or Red Hat production systems.”

Well that may be true, the attackers could have access to developer machines or CI servers and whatever credentials they had. They very well could have spread laterally and Red Hat hasn't noticed--hopefully not the case and time will tell

 

[macosandlinux]

at on white background Credit: istanbulimage via Getty

typo. maybe give it a proper caption next time, too?

 

[real mikeb_60]

Given that Red Hat focuses on enterprise installations, it's probably a juicy target for this kind of attack. How about the others? There are a lot of Ubuntu and Debian-based distributions that would be in trouble if their underlying repositories were successfully attacked.