Behold: the world's first known chosen-prefix collision of widely used hash function.
Read the whole story
Read the whole story
PGP keys, software securit,y and much more threatened by new SHA1 exploit
- Thread starter JournalBot
- Start date
Upvote
2
(2
/
0)
Actually, let’s dump this whole thread since it is pretty lame because ...
The git switch to SHA256 is in progress and has been since shortly after the first collision. They also switched the SHA-1 to a hardened version.
My knowledge was outdated and based on the hash + length checks they originally had a few years back when the death of SHA1 first came up.
Huh, ninjad by chronomitch right after your post. Bizarrely someone downvoted his pure fact post, weird.
Upvote
-5
(4
/
-9)
Just imported the keys. The fingerprints are different. Since the attack involves manipulation of an embedded ID photo to get the blocks to change for collision this is an example attack on web of trust, and only web of trust. I'm guessing applications that lack degrees of freedom to exploit like this are still mostly secure, just now 1/10th as secure as before.
The 10x cost improvement to collisions is still scary for many applications though. I wonder if incorporating unvalidatable string sections into certificates in general will turn out to have been a very bad idea.
The 10x cost improvement to collisions is still scary for many applications though. I wonder if incorporating unvalidatable string sections into certificates in general will turn out to have been a very bad idea.
Upvote
0
(1
/
-1)
Have you tried Godot? I'm using it for a little project now. It's pretty nice!
Upvote
2
(2
/
0)
Actually, pretty much any PDF file on your machine will have some garbage/ unseen info. It's not unusual, many vendors like to add their own mark ("created with <awesome tool> from <awesome company>"). You could add a image and just never reference it (or reference an updated version of it - yes, you can do that), or use a image format that allow you to put extra data in itself. It's pretty much impossible to look at a PDF file and determine what is usefull info, what is garbage, and what is just random metadata.
Open a PDF on notepad++ and see it for yourself if a judge would be able to understand what is usefull info. I can't (easily and in a reasonable amount of time), and I devoured the PDF especification a few years back to implement PAdES (PDF signatures in a more advanced format than SHA-1) for my previous organization.
Of course, all these attacks are POCs now, and we've started using SHA256 well before SHA-1 became compromised, so hopefully this won't be a problem when the price is finally low enough to carry real world attacks.
Upvote
9
(9
/
0)
na, the guy who said Space Station 13 was right, I'm a maintainer for a certain fork/server.
Upvote
7
(7
/
0)
I recall some very old games that had static space allocated to store the list of Direct3D texture formats (this was back before you queried the presence of a particular format) that would crash if you exposed more formats than they expected (e.g. S3TC formats). Other games would render incorrectly, or crash, if the formats were not in the exact same order 3dfx exposed the formats.
Ahh those were the days...
Upvote
1
(1
/
0)
The big issue here is really preventing fallback to SHA-1 in situations where it can be used for validation. Because as long as that's possible, someone can use this method to spoof a hash. It's not really an issue for other hashing use cases, but is definitely an issue for document validation, including PDFs and public directories.
And you're right: most people would never be able to identify which token in a PDF was being used as a hashpadder. Especially since you could easily just hide it inside a JPEG or PNG as tweaked optimization data, where it's actually processed data.
Upvote
6
(6
/
0)
How does this effect (or does it at all) TLS cipher suites? I know SSL Labs has started marking anything with CBC as weak although scores aren't negatively impacted yet. Does this mean anything ending in SHA (1, not 384 or 256) should be disabled asap?
If so, TLS 1.2 is rapidly running out of usable suites in IIS. Omitting the CBCs, we only have TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, and TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256. It doesn't look like MS is in any hurry to add TLS 1.3 support, either. Server 2019 doesn't even have it.
If so, TLS 1.2 is rapidly running out of usable suites in IIS. Omitting the CBCs, we only have TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, and TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256. It doesn't look like MS is in any hurry to add TLS 1.3 support, either. Server 2019 doesn't even have it.
Upvote
3
(5
/
-2)
Hi. This may be hard to believe, but I am from the future. We just entered the year 2020. I know right about now you must be thinking that GW is the worst President ever, and the war in Iraq is ill-advised. But I'm about to tell you just a little bit about our current President. You might want to sit down...
Upvote
17
(17
/
0)
Sure, I agree that's how things work. But if you look at what you wrote, the root of the problem is revealed: the server bears the computational cost but doesn't bear the fucking loss of private data cost, so there is a fundamental conflict of interests here. There is a basic reason users can't trust or delegate this decision to developers or providers- they will shortchange user security for structural reasons. I've been the developer side, and with a beancounter on one side, and user data that's no skin off your back on the other, is not a place where proper decision making about user security should be done.
So something like a third party who can demand accountable priority to the user's data security is necessary. At the very least, I think users would be more than willing to spend many cpu seconds a day on much longer keys. The fact that the decision makers find that a hard sell, is a demonstration of why they shouldn't be the decision makers.
Upvote
4
(4
/
0)
The amount of TLS cipher suites isn't an issue. In fact including so many in the TLS 1.2 spec is seen as a big mistake. The fewer the better, so long as they are well chosen and has at least one alternative that works substantially differently. That's why TLS 1.3 essentially only has the following:
Cipher suite: AES_GCM, AES_CCM, ChaCha20-Poly1305
Key exchange: DHE-RSA, ECDHE-RSA, ECDHE-ECDSA
Data integrity: AEAD
Upvote
9
(9
/
0)
Upvote
0
(0
/
0)
Was true in 2003 but for years sha-256 is slightly faster than sha-1!
Upvote
-1
(2
/
-3)
Not sure how this matters. First, digital signatures should have no validity between a consumer and a corporate entity for a long list of reasons for any contract of significant complication or amount of money (total value in 5 or more figures). There is no way for a normal consumer to know a "safe" digital contract from an "unsafe" one. Second, none of the methods currently in use for "digital signatures" or digital contracts are remotely close to the standard of a PGP signature in cryptography or document integrity.
Sure, I renewed my apartment via Docusign, which is meaningless as Docusign merely acts as a clearinghouse for PDFs and fake signatures I "approve" added to the PDFs. I keep my own copy of the PDF as well as screen captures of key areas, but frankly nothing will protect me from a fake PDF being submitted by the landlord in a civil action or eviction. There is no legal standard applied for establishing my identity, authorization, or authenticity when signing such effectively fake documents. Doing the Docusign garbage is just a checkbox in the process and everyone nodding their heads.
Upvote
5
(5
/
0)
And further to this... ALWAYS keep your digital copy. So if the landlord says "but you signed this" you can respond with "no, I signed THIS." Since the hashes also match on yours, it would then go to court if the manager didn't back down. And since it costs $120,000 PER DOCUMENT to do this, the manager would have to be making more of a profit off of this than that -- significantly more, since you've still got the original document that throws it into question.
And as you point out, this is moot, because real world signing authorities like Docusign don't even use proper validation like this. They're essentially nothing more than a digital notary.
Upvote
6
(6
/
0)
This is true, but not relevant to today's news. To replace an existing commit with a malicious commit that has the same hash, you would need to mount a "preimage attack", which has never been publicly demonstrated for SHA-1. The research presented here is a "chosen prefix attack", where the attacker starts with 2 prefixes P1 and P2 and can append junk to each one to find a collision.
Git uses hardened SHA-1, and I don't see anything in the paper that addresses the applicability to hardened SHA-1, but if the method described can be used to generated collisions in Git commits, then the following scenario is possible. An attacker who runs a Git server generates two commits with the same hash: one is benign and the other introduces a backdoor. They serve the benign version to almost all users, but they serve the backdoored version to a subset (based on some identifier like IP address, SSH public key, etc.). In this scenario, the person with the backdoored source code can compare commit hashes with somebody who has the benign source code. They can also pull and push commits between other private repositories. The attacker can replay all future commits from the benign repository to the backdoored repository and the hashes will stay in sync between the two repos.
As you point out, Git won't fetch objects it already has, so the collision only affects git clone, not git pull. But many CI/CD pipelines clone the repository on each build, so those could be potential targets for this type of tampering.
Upvote
0
(1
/
-1)
So, back to my original question: does this effect the cipher suites mentioned, and should they be disabled in favor of the GCM ciphers sooner rather than later?
Upvote
-1
(0
/
-1)
No, a notary checks your driver's license or similar to establish your identity by known, approved legal methods. Further a notary signs their own name and stamp, a named individual, as bond of authenticity against the signatures on the document.
Docusign is a house of cards, like every other legally fraudulent "novel method over the internet" application of existing functionality in the world.
I bring this up because as with MD5, this SHA1 defeat has no meaning in the context proposed. Almost everywhere SHA1 is currently in use is as a data transmission integrity check and potentially additional obfuscation, like an advanced CRC (it's primary use in IPSEC) not as a cryptographic cipher. (others have already explained this regarding Git using it against bitrot, but since people wanted to bring up the delusion of cryptographicly signed contracts, additional application of reality was necessary)
Upvote
2
(4
/
-2)
Of those, the RSA ones are ... iffy, as everything with RSA is. TLS 1.2 still used old-fashioned padding. So you're actually down to two: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, and TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256.
Upvote
2
(2
/
0)
There are a lot of subtleties to how Git handles SHA1 hash that it's hard to make a simplistic statement that "it's fine".
Imagine an open-sourced Git hosting service like GitHub or BitBucket. Let's say you clone a repo, make some malicious hard-to-detect changes to a file that results in the same hash, and then push the changes. If GitHub doesn't handle cross-repo caching properly, it's already conceivable that you can silently replace the old file with the new malicious file. Or, imagine if you manage to submit a pull-request or make a dev branch into the repo, now suddenly, GitHub or the Git hosting service could be confused, because the way the URL is structured allows you basically do https://github.com/<user>/<repo>/commit/<hash>, so if you can get the <hash> part to collide weird things may happen.
Also, Git signing (which is done via PGP) won't protect you in all cases. When you sign a commit, you are basically signing the hash to the commit object, which contains the commit message/etc + the hash to the tree to the files. In your malicious repo, you can substitute a particular file with your malicious one (let's say an executable script), and the hash won't change and therefore the signature will remain valid. Now, suddenly you are able to distribute a Git repo with a malicious payload. Yes, you should get your Git repo from a trusted source, but we have been trained to accept that a signed Git commit is *trusted*.
A lot of these issues have sort-of solutions to guard against, but just the fact that you have to think hard about them is an issue. SHA hash functions are *supposed* to be unique and hard to forge. When that's not true anymore you break a lot of builtin implicit assumptions about its behavior.
Git is already thinking about migrating to a more up-to-date SHA hash but unfortunately this takes time as the point of source control system is to preserve change history, which means old legacy commits are still important to keep.
Upvote
-2
(1
/
-3)
This is why large modulus RSA is no longer recommended. Factoring N will break it. Beyond Schol's algorithm, there are new non-quantum methods that people are working on to factor N. I personally have done research in this area myself. This is why the NSA recommends a minimum N of 3072 bits. The same security can be implemented with elliptic curve at around 440 bits. Furthermore, EC uses mostly addition which is much faster than the modulo-exponentiation that RSA requires.
Another point to consider that you brought up is low power devices. Low power (and embedded) hardware tends to be limited when compared to full hardware like Intel and AMD CPUs (ARM is the exception here). So hardware functions that are missing need to be implemented in software, which takes time to execute and that also drains the battery.
For years we have relied on using 128-bit hashes (SHA-1 is 160-bit) for fingerprinting public key certificates. The entire methodology needs to change. Even SHA-2 is showing its age. SHA-3 has an interesting structure with its sponge method. All these algorithms will be mostly going away anyways because once quantum becomes available, these algorithms will be immediately broken.
Three things define the security of a hash function:
1. It is difficult to guess the input from a hash output. In other words H(m) = k. You can't get from k back to m, which is good for passwords (rainbow tables not withstanding).
2. It is difficult to find two inputs that hash to the same value, or H(m1) = k && H(m2) = k where m1 != m2.
3. It is difficult to find an input which matches a known output. In other words H(m) = k where m is unknown and k is known.
A lot of effort is focused on #2, where #3 is a consequence.
Upvote
0
(0
/
0)
It's amazing how many people mistake an implementation detail for a security feature (git using sha1 for its content addressable file system) and seem to rely on it for their security.
And with amazing I mean very, very concerning.
Upvote
1
(2
/
-1)
I have no idea what you mean with "cross-repo caching" but what will happen if you push back to the original repo is that git will see that there were no changes and simply ignore your modified file because the hashes match.
Now you can obviously create your own repo with the malicious code and get people to come from there, but taking code from am untrusted source is a bad idea anyhow. The only thing this really does is that people can't rely on comparing hashes to be sure the two repos are identical. But then that's a really weird use case anyhow.
So again not a problem with git but a purely theoretical problem in one UI. As soon as you try to merge the malicious file with the same hash into the repo nothing will happen - see above.
SHA-1 is an implementation detail. With git as with every other software the age old "only use code from a trusted source" still applies. Hell, if a malicious actor can merge commits into a repository there are much, much easier ways to introduce security problems.
Upvote
2
(3
/
-1)
Upvote
2
(2
/
0)
It's obvious that the people commenting on this article have some understanding of the issues. (Ars Technica readers are generally fairly clued-in on tech topics from what I've seen.)
What worries me is that so many computer users _don't_ know about these topics and may not even know they exist. "SHA-what?"
This "inertia" that slows down the deprecation of insecure algorithms and protocols would seem to be partly the fault of large organizations or corporations and partly due to the consumers. (Given that neither large organizations nor the average consumer likes to just spend money for what they consider to be no reason.)
There has been some success in things like getting large corporations to agree to stop using meat from animals given "preventive" antibiotics. Ditto for agreeing to avoid animal cruelty, growth hormones, GMOs, etc.
I think that we probably need to make the general public more aware of some of these technological issues (like SHA-1) and see if we can get them to put pressure on the large organizations.
Of course we also need to give people a way to properly evaluate risk. I'm not sure how to do that, and I don't think it's been done for some things that many consumers are currently very concerned about.
I wonder if we could also put in place some kind of labeling for products? "This product contains encryption known to be vulnerable by the State of California."
What worries me is that so many computer users _don't_ know about these topics and may not even know they exist. "SHA-what?"
This "inertia" that slows down the deprecation of insecure algorithms and protocols would seem to be partly the fault of large organizations or corporations and partly due to the consumers. (Given that neither large organizations nor the average consumer likes to just spend money for what they consider to be no reason.)
There has been some success in things like getting large corporations to agree to stop using meat from animals given "preventive" antibiotics. Ditto for agreeing to avoid animal cruelty, growth hormones, GMOs, etc.
I think that we probably need to make the general public more aware of some of these technological issues (like SHA-1) and see if we can get them to put pressure on the large organizations.
Of course we also need to give people a way to properly evaluate risk. I'm not sure how to do that, and I don't think it's been done for some things that many consumers are currently very concerned about.
I wonder if we could also put in place some kind of labeling for products? "This product contains encryption known to be vulnerable by the State of California."
Upvote
-1
(1
/
-2)
While this is a good idea unfortunately the implementation still depends on SHA1 - the signatures are on the commit objects (which reference other objects by SHA1). You can see for yourself, in the code in the buffer passed to do_sign_commit: it doesn't pass a buffer containing the binary contents of every object in the commit but rather you see that it includes the tree, parents, other header metadata (similar to what you see when doing git log --format=raw).
This means that if you were to create a commit that included an object for which you created an alternate object that would generate the same SHA1 (note that I say "object" not file, because there's some additional metadata involved but if this improves such that you can create files that will result in colliding objects) then you could swap it out in an colliding repo and signatures on both repos will still look good.
Upvote
0
(1
/
-1)
That's a machine/situation/application specific statement. Certainly, it's not true on my box with openssl.
Code:
$ openssl speed sha1 sha256
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
sha1 82401.77k 222859.52k 514273.96k 752517.74k 728271.55k
sha256 57963.06k 167342.95k 300917.33k 419013.97k 410446.51k
Upvote
2
(2
/
0)
When you're viewing the PDF in an actual viewer it all seems legit.
If you're looking at the "source" however, then it's often pretty transparent that you were tricked.
Random collision attacks (the ones taking advantage of the birthday paradox) typically rely on having both files include all data with the equivalent of an IF statement deciding which one to show. This makes it obvious as long as you get to inspect either "copy."
Chosen prefix attacks can be harder. You start with chosen legit prefixes (so PDF files) and add random garbage that somehow gets ignored to the end of one (or both). If you need to add garbage to both, again it's pretty transparent that something's going on by just looking at either "copy."
If you only need to add garbage to one of the documents, then it's a more powerful attack, because it seems completely legit from the point view of the other one. You'd still be able to provide your "copy" with the garbage added, if you kept it.
It makes most of these "landlord" things a bit ridiculous because they'd be easily solved in a court house.
So it's the ones that rely on automated systems processing the signed data that are more concerning, especially anything where actions are hard to reverse. Think crypto currency transactions.
Upvote
-1
(0
/
-1)
I think it's important to remember with Git we're generally dealing with text files. On top of being able to generate the right SHA1 for a text file, you also have to pass the Diff tool. So any changes no matter how small or "hidden" are pushed to the very top for everyone to see.
Upvote
1
(1
/
0)
Upvote
1
(1
/
0)
The difference is pretty small on my ThreadRipper 3960X in Ubuntu running on Windows 10:
Code:
$ openssl speed sha1 sha256
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
sha1 297560.03k 755899.09k 1541911.98k 2077562.54k 2310995.97k
sha256 285384.49k 694133.01k 1431843.58k 1940314.79k 2165383.17k
Upvote
0
(0
/
0)
Thanks, I've altered my cipher suite order to put those first followed by the RSA GCM ciphers, then the CBC ciphers (ECDSA SHA384/256, then RSA SHA384/256), and disabled the SHA1 ciphers altogether.
Upvote
-1
(0
/
-1)
Me, yes. It has been a lot of years since I've rented.
I've been a landlord long enough to know that bad renters seem more common than bad landlords.
That said, a bad landlord can make your life way more hell than a bad renter can.
Upvote
0
(0
/
0)
Unfortunately git itself relies on the SHA1 object hashes for the signing (even if signing had a good digest) so the use of SHA1 ultimately made it into a security feature. Since you sign a commit header the use of SHA1 all the way down means the signing implementation relies on it.
Making a tarball and signing that is a safer option.
Upvote
1
(1
/
0)
Well Dan does disappoint this time. Mistaken use of terms, incorrect and inaccurate dates, misleading noise about developer tools.
While SHA1 has been slowly phased out over the past five years, it remains far from being fully deprecated.
No, "deprecated" means disapproved of, not replaced. It's similar to "pregnant": you can't be a little deprecated. [1]
Git, the world's most widely used system for managing software development among multiple people, still relies on SHA1 to ensure data integrity.
No, in fact Git relies on two checksums: the SHA-1 checksum and the file length [2]. Rendering a chosen prefix plus attacker-calculated end results in: a different non-matching length.
Also, Git does not rely on "where some kind of collision resistance is to be expected from the hash function": Git just needs a fixed size unique handle for each file.
And as poster ramases points out, if attackers are inside your git you have other problems.
Then, this ridiculous claim: "The full deprecation of MD5 for authentication purposes didn’t come until 2012" Without attribution. This is false. SHA-1 was deprecated in 2006 [3]. MD5 was attacked extensively in the 2003-2007 range, and of course about as soon as it was published (RFC 1321, April 1992). RFC 6151 says (March, 2011) it is no longer acceptable when collision resistance is required, such as digital signatures.
As sometimes the case with Ars, the comments are more informative and accurate than the article.
Hopefully Dan gets it a bit better next time; used to be able to depend on him.
[1] https://techterms.com/definition/deprecated
[2] https://git-scm.com/book/en/v2/Git-Inte ... it-Objects
[3] SHA-1 was deprecated March 15, 2006 https://csrc.nist.gov/Projects/Hash-Fun ... -Functions NIST never appears to have deprecated MD5, but the Internet Society did (RFC 6151).
While SHA1 has been slowly phased out over the past five years, it remains far from being fully deprecated.
No, "deprecated" means disapproved of, not replaced. It's similar to "pregnant": you can't be a little deprecated. [1]
Git, the world's most widely used system for managing software development among multiple people, still relies on SHA1 to ensure data integrity.
No, in fact Git relies on two checksums: the SHA-1 checksum and the file length [2]. Rendering a chosen prefix plus attacker-calculated end results in: a different non-matching length.
Also, Git does not rely on "where some kind of collision resistance is to be expected from the hash function": Git just needs a fixed size unique handle for each file.
And as poster ramases points out, if attackers are inside your git you have other problems.
Then, this ridiculous claim: "The full deprecation of MD5 for authentication purposes didn’t come until 2012" Without attribution. This is false. SHA-1 was deprecated in 2006 [3]. MD5 was attacked extensively in the 2003-2007 range, and of course about as soon as it was published (RFC 1321, April 1992). RFC 6151 says (March, 2011) it is no longer acceptable when collision resistance is required, such as digital signatures.
As sometimes the case with Ars, the comments are more informative and accurate than the article.
Hopefully Dan gets it a bit better next time; used to be able to depend on him.
[1] https://techterms.com/definition/deprecated
[2] https://git-scm.com/book/en/v2/Git-Inte ... it-Objects
[3] SHA-1 was deprecated March 15, 2006 https://csrc.nist.gov/Projects/Hash-Fun ... -Functions NIST never appears to have deprecated MD5, but the Internet Society did (RFC 6151).
Upvote
-1
(2
/
-3)
Reordering does not do anything useful - if someone is trying to MITM you and wants to force you to a lower strength cipher, they'll just advertise that they support only the weak one. Anything that you don't really trust should not be on the list.
Upvote
5
(5
/
0)