what did you learn today? (part 2)

w00key

Ars Tribunus Angusticlavius
8,703
Subscriptor
It has changed and it attempts to map IPv6 to countries. Which is why I asked how feasible that is in the long term.
Should be more feasible than everyone borrowing/buying a /24 block leading to swiss cheese routing.

Most providers / ASNs just get a few huge assignments and they are good forever.

580k ipv4 vs 120k ipv6 aggregated prefixes / CIDRs.
 

sryan2k1

Ars Legatus Legionis
46,412
Subscriptor++
This is the same company that loves to randomly kill BGP peering sessions for no fucking reason whatsoever
I think I've said this, but we peer with them at the DET-IX. Two sessisions to two different routers all set up by them and visible in the Azure peering portal.

One day one drops offline, not unusual but after a day or so it's still offline. We check in the peering portal and the 2nd one is just gone. I open a ticket, after 3 weeks of azure support saying that never existed and that I needed to add it (and me pointing out I can't add it, you guys do) the ticket was closed. Several days later the old peering mysteriously just showed back up and the peer came online.
 

Vince-RA

Ars Praefectus
5,304
Subscriptor++
I'm getting blocked by a 365 conditional access policy because Microsoft can't identify the location of the IPv6 address that is the source of the traffic. The source of the traffic is an Azure Virtual Desktop. How can microsoft not know where their own shit lives? :flail:
Followup here after going back and forth with support ad nauseum:

  • Microsoft reserves the right to route traffic to Entra ID through whatever the hell they want after it leaves your network (whether Azure or otherwise) and prefers IPv6 these days. If this happens, the source of that traffic will appear to be those new routing points. OK, fine.
  • They don't know why they can't geolocate their own IPv6 addresses.
  • Perhaps more concerning, they publish a list of 365 IPv6 addresses, and the ones I'm seeing in our sign-in logs are NOT in that list. They don't know why this is the case either - although they do recommend I allow traffic from the addresses.
  • They don't know why simulating a location block conditional access policy via the "What If" tool correctly respects the cloud app exclusions, but the actual real life policy evaluation doesn't.

I mean, I'm definitely not the smartest man alive, and I realize that this is a complicated interaction involving lots of networks and lots of services and lots of policies, but...what the fuck? I don't think these should be difficult questions to answer. And I don't think anyone with any IT/infosec experience would be comfortable just throwing stuff in an allow list without some answers. Am I out of line here?
 

Danger Mouse

Ars Legatus Legionis
38,791
Subscriptor
You don't need to use the cloud management system. You can run them independently, or run your own instance of FortiManager internally.

There's zero reason to expose that interface to the public internet, and barely any to expose it to even a whitelisted list of public IPs. It should be done internally.
Same as PAN then.
 

Danger Mouse

Ars Legatus Legionis
38,791
Subscriptor
PSA: do not assume your blades have an accessible usb port. Even tho my stupid tv has usb these days, I need one of these fuckers to get USB on my UCS blades:
View attachment 126690
There's probably an internal USB somewhere still, but definitely not externally accessible without some modding.
 
Last edited:

Whittey

Ars Tribunus Militum
1,994
  • Perhaps more concerning, they publish a list of 365 IPv6 addresses, and the ones I'm seeing in our sign-in logs are NOT in that list. They don't know why this is the case either - although they do recommend I allow traffic from the addresses.
Authentication coming from an unknown location? Just allow it bro, we asked Copilot Copilot (Copilot) and it said it was cool.
 

Vince-RA

Ars Praefectus
5,304
Subscriptor++
I mean, it's insane but kind of. That's just "Business as usual" for Azure.
Don't mind me, I'll just be over here crying myself to sleep :finedog:

My boss accuses me of liking to complain about MS/Azure - and he's not wrong! But also, my team and I are the ones who have to put up with the daily insanity in order to deliver a good user/developer experience to the rest of the company. And when it comes to Azure Cloud, I know better, and they do not, having been 99.8% on-prem. People using AWS do not have to put up with a fraction of the miserable nonsense that we do in Azure. Like, my job has been jumping in and fixing broken shit for a couple decades now, and I enjoy it and I am generally regarded as good at it - but the last couple years of all-in-on-Microsoft have been exhausting. </pityparty>
 

sryan2k1

Ars Legatus Legionis
46,412
Subscriptor++
AWS' limitations or insanity are typically rooted in history and there is usually a good reason for the way that it is. The best way I can describe Azure to someone is that it's a Temu AWS. All the same stuff if there, but none of it's quite right. It is truly impressive to me how they've managed to build such a shitty platform and that how it works is okay with anyone.
 
AWS' limitations or insanity are typically rooted in history and there is usually a good reason for the way that it is. The best way I can describe Azure to someone is that it's a Temu AWS. All the same stuff if there, but none of it's quite right. It is truly impressive to me how they've managed to build such a shitty platform and that how it works is okay with anyone.
I generally don't complain about the technical aspects of AWS but having just spent 2 days trying to get SecretsManager Secret rotation to behave sensibly I want to go and poke that team with a pointy stick and make them reflect on the shitness of that product
 
  • Hug
Reactions: Vince-RA
AWS' limitations or insanity are typically rooted in history and there is usually a good reason for the way that it is. The best way I can describe Azure to someone is that it's a Temu AWS. All the same stuff if there, but none of it's quite right. It is truly impressive to me how they've managed to build such a shitty platform and that how it works is okay with anyone.
Azure really is the Sirius Cybernetics Corporation of clouds - things get left half-assed because even that was a battle to achieve.

And everything works just a bit differently across the resource types - not necessarily better or worse but different, so you can't reuse concepts or knowledge, because it's all conditional.

*Edit: Can't
 
Last edited:
  • Like
Reactions: SandyTech

Vince-RA

Ars Praefectus
5,304
Subscriptor++
Azure really is the Sirius Cybernetics Corporation of clouds - things get left half-assed because even that was a battle to achieve.

And everything works just a bit differently across the resource types - not necessarily better or worse but different, so you can reuse concepts or knowledge, because it's all conditional.
I assume you mean "can't reuse", which I would definitely agree with. This has gotten worse as products have been added by acquisition.

Temu AWS (hah) cranks that dial up to 11 by providing multiple services that do the same thing, but work in totally different ways (like Backup Vault and Recovery Services Vault). Even more fun, one is clearly on the way out and not the recommended solution, but the new one doesn't support everything yet and so you have to use the old one, knowingly adding to your tech debt whenever it comes time to migrate to the new one. And to be fair, the backup situation with two options is actually a good one to be in, because with Microsoft usually there are more like 3-5 solutions in various states of decay, deprecation, and/or dangerous preview.
 

sryan2k1

Ars Legatus Legionis
46,412
Subscriptor++
Azure is what you get when you force a bunch of product teams who would much rather be in a shooting war with each other to suddenly pull together and create a single cohesive product suite.
I mean that's been how they've operated internally nearly forever. I have stories from guys that worked there in the 90s-00s where the exchange team refused to give documentation to the outlook team on how something worked because fuck you that's why.
 
  • Wow
Reactions: MilleniX
providing multiple services that do the same thing,
I don't want to get in the way of your Temu bashing but that idea of plurality of functionality does also exist in AWS. I suppose to their credit and unlike the other cloud vendor we've not mentioned they seldom deprecate anything...
 
  • Like
Reactions: Vince-RA

dredphul

Ars Tribunus Angusticlavius
6,178
Subscriptor++
I mean that's been how they've operated internally nearly forever. I have stories from guys that worked there in the 90s-00s where the exchange team refused to give documentation to the outlook team on how something worked because fuck you that's why.

It's time to bring up the tech company org style comic: https://bonkersworld.net/organizational-charts

Knew someone who was offered a job at Microsoft back in the '90s and he rejected it. There were weird things done simply due to some sort of competition with another group within Microsoft. Things like staying at work until 1am every Thursday to seem like they had a lot of work to do.
 

Vince-RA

Ars Praefectus
5,304
Subscriptor++
I don't want to get in the way of your Temu bashing but that idea of plurality of functionality does also exist in AWS. I suppose to their credit and unlike the other cloud vendor we've not mentioned they seldom deprecate anything...
You're not wrong there. AWS is almost too willing to let things live on forever imo. EC2 classic (pre-VPC) lived on way longer than any remotely careful person would have let it. S3 IAM verbs have long since proliferated past the point of insanity, etc. But in general I believe that the people working on AWS have done actual work using their own services and the implementation choices reflect that. I do not believe that anyone involved in designing Azure has done anything serious with it, or arguably, at all.
 

Vince-RA

Ars Praefectus
5,304
Subscriptor++
I've used CIS benchmarks to do things like harden server operating systems for a very long time now. Some of the stuff was of questionable or borderline utility at best, but a lot of it was solid advice - disable unused services, reduce the number of places you can execute things, make sure you're capturing logs, etc. Over time you borrow/build up a good ansible playbook or whatever to run during image building and it ends up being a good investment, especially when you can point to the benchmarks on DDQs etc.

Now that CIS has gone "cloud native" it all feels completely bonkers. Again, there is good advice in there for sure (don't expose your stuff directly to the internet, use MFA, etc) but there is just so damn much now and so much of it applies to multiple pieces of infrastructure. 10 points on key vaults and 68 key vaults in our production environment alone = 680 lines in a report. Even our relatively small environment we'll easily end up with tens of thousands of benchmark data points.

Another annoying aspect of this is that many of the recommendations directly increase costs with questionable benefits (thou shalt have defender for goddamn key vault!) - but if you don't adopt the recommendations, there is your NONCOMPLIANT right there in red and white in the defender for cloud dashboard and reporting and everyone hates that, don't they precious? The flip side of that coin is that clients have started asking for this level of detail on DDQ engagements, so either you dump your report (if it looks ok) or you suffer through insane bespoke spreadsheets and justifications and evidence requests. It's a brilliant design if you're a cloud provider trying to sell security, but it seems like the net effect is a gamified security theater treadmill with a questionable increase in actual security.

(Why yes, I'm old, and I'm especially cranky today, why do you ask?)
 

Quarthinos

Ars Tribunus Militum
2,905
Subscriptor
I remember seeing a security report that said there was a finding because a network-specific DNS resolver wasn't being used (called out by the DNS name, even.) So who had to write an entire paragraph about why that finding didn't apply because the specific network was physically separated from the machine under test? Me :( Because some goddamn checklist follower of a security person that had zero cybersecurity experience was responsible for blessing the machines to be used on PROD.
 

sryan2k1

Ars Legatus Legionis
46,412
Subscriptor++
Nessus sending automated vulnerability reports via email is more useful than 95% of the "security" people I've ever had to deal with.


Most (but not all) of them don't care why or understand how the risk works, they just want their scanner to be green.


One of the third parties we use for public stuff doesn't understand how Ubuntu backports security patches and updates the version suffix. We've had to explain that to them a bunch of times.


Them: you're running apache 1.2.3 that's super haxable!
Us: No, it's 1.2.3 Ubuntu-x.y.z which has those issues patched
Them: ....I-dont-beleive-you-ron-burgendy.gif
 

Vince-RA

Ars Praefectus
5,304
Subscriptor++
scanner to be green
1771589995777.png
 

Demento

Ars Legatus Legionis
15,353
Subscriptor
One of the third parties we use for public stuff doesn't understand how Ubuntu backports security patches and updates the version suffix. We've had to explain that to them a bunch of times.


Them: you're running apache 1.2.3 that's super haxable!
Us: No, it's 1.2.3 Ubuntu-x.y.z which has those issues patched
Them: ....I-dont-beleive-you-ron-burgendy.gif
Every. Fucking. Month.
Look, it's running Debian 13 (I'm on a quest to get rid of Ubuntu where I can), it's fine. Why do I have to explain this to the same security dipshit each and every time he gets a report?
 

Quarthinos

Ars Tribunus Militum
2,905
Subscriptor
Every. Fucking. Month.
Look, it's running Debian 13 (I'm on a quest to get rid of Ubuntu where I can), it's fine. Why do I have to explain this to the same security dipshit each and every time he gets a report?
You reacted to @Vince-RA picture like it was a joke. Pretty sure the scanner is running once a month and returning red, so you get the audit again... Has your morale not improved yet?
 

ComradeXavier

Ars Scholae Palatinae
865
Subscriptor++
Every. Fucking. Month.
Look, it's running Debian 13 (I'm on a quest to get rid of Ubuntu where I can), it's fine. Why do I have to explain this to the same security dipshit each and every time he gets a report?
There's a TFTP in the dev tools for some embedded hardware I work with. Every time someone moves those around, it gets flagged as dangerous by the malware scanner and we have to explain that no, it's not malware; no, we don't use it actively; no, we won't delete it because it's part of the tools package. Why do we have to answer this every time?

My best guess is that the security monitoring rules require them to "take every event seriously", which means they have to investigate and record some kind of resolution. Still annoying that they officially can't learn from what we tell them every time.
 
  • Like
Reactions: Danger Mouse