Eh, it was a custom rule in their firewall, editting out that value fixed it
*shrug* maybe it was a placebo, maybe it was a Magic/No Magic switch.
*shrug* maybe it was a placebo, maybe it was a Magic/No Magic switch.
If it's looking for TTL <= 2, that's pure fucking evil. Or pure genius. It ensures that only machines with a specified TTL of 2, or 30 or 62 hops (depending on OS) can get there. Someone three hops away is fucked.Eh, it was a custom rule in their firewall, editting out that value.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24590957#p24590957:p3rhb532 said:Darkseid[/url]"3rhb532]Eh, it was a custom rule in their firewall, editting out that value fixed it
*shrug* maybe it was a placebo, maybe it was a Magic/No Magic switch.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24590683#p24590683:3hs6hp34 said:ncrand[/url]":3hs6hp34]I think DM needs his own thread entitled "The Perpetual Danger Mouse Infrastructure Thread".
[url=http://meincmagazine.com/civis/viewtopic.php?p=24588409#p24588409:2oox512k said:Danger Mouse[/url]":2oox512k]...that Notepad++ has an awesome plugin available to do visual comparison of files, such as copies of switch/router/firewall configs![]()
man diff
man patch
[url=http://meincmagazine.com/civis/viewtopic.php?p=24590891#p24590891:1ievhavj said:sryan2k1[/url]":1ievhavj]TTL is a hop count, not a time based metric.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24591617#p24591617:298nup93 said:M. Jones[/url]":298nup93][url=http://meincmagazine.com/civis/viewtopic.php?p=24588409#p24588409:298nup93 said:Danger Mouse[/url]":298nup93]...that Notepad++ has an awesome plugin available to do visual comparison of files, such as copies of switch/router/firewall configs![]()
Code:man diff
Code:man patch
And he was enlightened.
For bonus points, have the configs pulled and checked into source-code control -- rancid does this, I'm fairly certain -- and mail or RSS you the diffs realtime. Perhaps you could get a commit-hook to pull AAA information so you'd have culprits on the version history.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24590511#p24590511:34pzx0am said:sryan2k1[/url]":34pzx0am]Protip, plugging a 2nd Cisco phone into a first that has it's upstream port set to BPDUGuard will result in a phone with no power.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24590343#p24590343:26x4f66d said:hernias[/url]":26x4f66d]TIL that caffeine + psuedoephedrine makes everything a little... bouncy.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24586683#p24586683:2wf1zx4d said:Pontiphex[/url]":2wf1zx4d]As a vendor that sells enterprise software that usually gets installed on-premise in these kinds of environments, let me tell you it can be just as infuriating the other way around.
Jackass operations folks that completely ignore our recommended CPU, Memory, and Storage specs - and dig their heels in despite proof showing that "yes, because you only gave us 6GB of memory for the SQL Server there is not enough for caching - see how the page life expectancy metric is a mere 2 seconds", their obstinance requiring us to go over their head and get a VP/Director level person to force the issue (which magically goes away afterwards).
This one was extra special because we pleaded with them to just bump us up to our recommended specs for a few days so the go-live had a shot at going smoothly, and they could later reduce it until they found equilibrium...and they spun some tale about performance issues that occur if you increase memory and then later decrease it - which would require rebuilding the guest VM to resolve! They could never back that claim up.
Or insisting that the SAN is tip-top until we can show proof of insane disk queue lengths or read/write latencies - (Oh, yeah, we actually have a ticket open with EMC, hurr durr).
Those are at least relatively simple to provide proof of deficiency without having access beyond the guest OS...but the one that has been a real pain is demonstrating when a host is CPU starved. I'm not sure how to measure that with just guest access. Thankfully it has only come up once.
On a positive note, TIL that Brocade's ethernet fabric stuff is really cool. Remains to be seen if it will actually do what they say it will, but what I saw on our crash course today had some pretty awesome promise. It also feels a bit like cheating when stuff actually "just works".
Better yet, look at something device specific, like CSM (bleah), FortiManager (slightly less bleah) or JunOS Space (unknown bleah) that can do visual diffs with highlighting that are appropriate for the device-type, sometimes with sections that collapse/expand to make things easier.For bonus points, have the configs pulled and checked into source-code control -- rancid does this, I'm fairly certain -- and mail or RSS you the diffs realtime. Perhaps you could get a commit-hook to pull AAA information so you'd have culprits on the version history.
Fuck it. If Ars wants to create some new forums like they've suggested, a Danger Mouse Train Wreck forum would be lucrative.I think DM needs his own thread entitled "The Perpetual Danger Mouse Infrastructure Thread".
[url=http://meincmagazine.com/civis/viewtopic.php?p=24592871#p24592871:qbviev9s said:ronelson[/url]":qbviev9s] If Ars wants to create some new forums like they've suggested, a Danger Mouse Train Wreck forum would be lucrative.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24592949#p24592949:3sa354g8 said:M. Jones[/url]":3sa354g8][url=http://meincmagazine.com/civis/viewtopic.php?p=24592871#p24592871:3sa354g8 said:ronelson[/url]":3sa354g8] If Ars wants to create some new forums like they've suggested, a Danger Mouse Train Wreck forum would be lucrative.
I avoid schadenfruede generally, but the train wreck I want to read is the one that happens at Danger Mouse's current organization when he departs.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24586683#p24586683:31lgxafx said:Pontiphex[/url]":31lgxafx][url=http://meincmagazine.com/civis/viewtopic.php?p=24571957#p24571957:31lgxafx said:Sunner[/url]":31lgxafx]
This seems to be especially common with developers in my experience. It's never the code, a shitty SQL or anything like that, and if virtualization is involved that's definitely it, damn black magic! Had one of those here earlier, a server running batch jobs was choking every night, the DB load was light so it wasn't even shitty SQL for once. The guy in charge at the time was completely spineless though, so they ended up buying a 16 core server with 32 GB of memory or some such retarded thing, all running on a not particularly heavily loaded SAN. Big surprise, the gains were very marginal.
At a previous job in roughly the same line of business a 3-way UltraSPARC IIi with 4 GB of memory and local slow ass disks ran similar jobs of similar volumes just fine. It also ran a bunch of other stuff for that matter. Of course the devs we had there were actually really good.
But yeah of course it wasn't the code. I think they started blaming the SAN next. Possibly the firewall as well because why not, firewalls are EVIL!![]()
As a vendor that sells enterprise software that usually gets installed on-premise in these kinds of environments, let me tell you it can be just as infuriating the other way around.
Jackass operations folks that completely ignore our recommended CPU, Memory, and Storage specs - and dig their heels in despite proof showing that "yes, because you only gave us 6GB of memory for the SQL Server there is not enough for caching - see how the page life expectancy metric is a mere 2 seconds", their obstinance requiring us to go over their head and get a VP/Director level person to force the issue (which magically goes away afterwards).
This one was extra special because we pleaded with them to just bump us up to our recommended specs for a few days so the go-live had a shot at going smoothly, and they could later reduce it until they found equilibrium...and they spun some tale about performance issues that occur if you increase memory and then later decrease it - which would require rebuilding the guest VM to resolve! They could never back that claim up.
Or insisting that the SAN is tip-top until we can show proof of insane disk queue lengths or read/write latencies - (Oh, yeah, we actually have a ticket open with EMC, hurr durr).
Those are at least relatively simple to provide proof of deficiency without having access beyond the guest OS...but the one that has been a real pain is demonstrating when a host is CPU starved. I'm not sure how to measure that with just guest access. Thankfully it has only come up once.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24593305#p24593305:1skjdn5o said:Danger Mouse[/url]":1skjdn5o][url=http://meincmagazine.com/civis/viewtopic.php?p=24592949#p24592949:1skjdn5o said:M. Jones[/url]":1skjdn5o][url=http://meincmagazine.com/civis/viewtopic.php?p=24592871#p24592871:1skjdn5o said:ronelson[/url]":1skjdn5o] If Ars wants to create some new forums like they've suggested, a Danger Mouse Train Wreck forum would be lucrative.
I avoid schadenfruede generally, but the train wreck I want to read is the one that happens at Danger Mouse's current organization when he departs.
My boss winces, every time I mention a new phone/in person interview.![]()
He's promised, more or less that I'll be promoted in the new FY (starting in July), but we'll see.
databases\.mdf and the log was \logs\.ldf WTF.....I thought it was a wonky screen refresh (showing no filename) but a command line "dir" on the directory showed ".mdf" as well.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24594849#p24594849:3qsa5hrv said:Arbelac[/url]":3qsa5hrv]Maybe he's still running an NT4 domain.![]()

[url=http://meincmagazine.com/civis/viewtopic.php?p=24594699#p24594699:1k9vr023 said:Rick25[/url]":1k9vr023]That SCOM 2012 wSP1 somehow decided to create the Data Warehouse DB on SQL2012 with the following structure
edatabases\.mdf and the log was \logs\.ldf WTF.....I thought it was a wonky screen refresh (showing no filename) but a command line "dir" on the directory showed ".mdf" as well.
Detach the database, rename it and then reattach (fixing the paths) and all is good. Also stupid permission errors on the server unless UAC was completely disabled.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24594849#p24594849:3034nlkd said:Arbelac[/url]":3034nlkd]Maybe he's still running an NT4 domain.![]()
[url=http://meincmagazine.com/civis/viewtopic.php?p=24594277#p24594277:3lgh28r1 said:mdporter[/url]":3lgh28r1][url=http://meincmagazine.com/civis/viewtopic.php?p=24593305#p24593305:3lgh28r1 said:Danger Mouse[/url]":3lgh28r1][url=http://meincmagazine.com/civis/viewtopic.php?p=24592949#p24592949:3lgh28r1 said:M. Jones[/url]":3lgh28r1][url=http://meincmagazine.com/civis/viewtopic.php?p=24592871#p24592871:3lgh28r1 said:ronelson[/url]":3lgh28r1] If Ars wants to create some new forums like they've suggested, a Danger Mouse Train Wreck forum would be lucrative.
I avoid schadenfruede generally, but the train wreck I want to read is the one that happens at Danger Mouse's current organization when he departs.
My boss winces, every time I mention a new phone/in person interview.![]()
He's promised, more or less that I'll be promoted in the new FY (starting in July), but we'll see.
He's shown you the approved promotion paperwork and pay grade you'll upped to?
[url=http://meincmagazine.com/civis/viewtopic.php?p=24594973#p24594973:3lgh28r1 said:ncrand[/url]":3lgh28r1][url=http://meincmagazine.com/civis/viewtopic.php?p=24594849#p24594849:3lgh28r1 said:Arbelac[/url]":3lgh28r1]Maybe he's still running an NT4 domain.![]()
Nope, 2003 (Although it was 2000 until about 2 months ago). I usually use the term "PDC" to mean the primary FSMO (or operations master if your so inclined) holders for a particular domain. Also keeping in mind that "PDC Emulator" is still an opeartions master role as well.![]()
[url=http://meincmagazine.com/civis/viewtopic.php?p=24594277#p24594277:18ltt8rw said:mdporter[/url]":18ltt8rw]He's shown you the approved promotion paperwork and pay grade you'll upped to?
Finger nudging and blam, fixed that too.

Train wreck, remember?He's shown you the approved promotion paperwork and pay grade you'll upped to?
[url=http://meincmagazine.com/civis/viewtopic.php?p=24594211#p24594211:iq7lyikp said:ferzerp[/url]":iq7lyikp]
Probably because vendors *suck* at proper sizing so we have no choice to throw out the specs, make an educated guess on sizing, and increase as needed. They rarely understand the implications of asking for more than they need on vcpus, and most modern oses will happily gobble up excess memory for cache if they miss the mark there as well. It is far easier to increase resources as needed than it is to remove. Until an organization *proves* it isn't just pulling specs out of its ass, we assume it is because they almost all do.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24595023#p24595023:2jv6j626 said:brshoemak[/url]":2jv6j626]
TIL that according to the CEO it's apparently much better to have an update applied to your hosted-app production server without rolling it out to the test server first. Better yet, don't schedule it to be performed over the weekend, instead do it on Thursday night and it should be done by 10am on Friday after people have been sitting around for 2 hours with their thumb up their keister.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24595789#p24595789:2uxvuta4 said:M. Jones[/url]":2uxvuta4][url=http://meincmagazine.com/civis/viewtopic.php?p=24594211#p24594211:2uxvuta4 said:ferzerp[/url]":2uxvuta4]
Probably because vendors *suck* at proper sizing so we have no choice to throw out the specs, make an educated guess on sizing, and increase as needed. They rarely understand the implications of asking for more than they need on vcpus, and most modern oses will happily gobble up excess memory for cache if they miss the mark there as well. It is far easier to increase resources as needed than it is to remove. Until an organization *proves* it isn't just pulling specs out of its ass, we assume it is because they almost all do.
I don't care if your code is amazing with 32GB, 10,000 IOPs and <1ms to the database. It needs to be amazing with all parameters worse by an order of magnitude, because clearly-predictable circumstances will conspire to require it to be run in that fashion.
An old but useful example is when an organization decided to centralise their RDBMS Stateside, for reasons of data confidentiality, data management and RDBMS license conservation. An Eastern European office needed heavy database access. Because of poor infrastructure at the time, the Eastern European office was connected over satellite. Round-trip from geosynchronous means 700ms latency to the database. Do you have any idea what lack of parallelism and stored procedures are exposed by 700ms of latency?
This is why it will be a long time before Citrix/RDS and to a lesser extent X will go away, keep the app near the database and only bring the display to the user.[url=http://meincmagazine.com/civis/viewtopic.php?p=24595829#p24595829:2ryg7kn5 said:ferzerp[/url]":2ryg7kn5][url=http://meincmagazine.com/civis/viewtopic.php?p=24595789#p24595789:2ryg7kn5 said:M. Jones[/url]":2ryg7kn5][url=http://meincmagazine.com/civis/viewtopic.php?p=24594211#p24594211:2ryg7kn5 said:ferzerp[/url]":2ryg7kn5]
Probably because vendors *suck* at proper sizing so we have no choice to throw out the specs, make an educated guess on sizing, and increase as needed. They rarely understand the implications of asking for more than they need on vcpus, and most modern oses will happily gobble up excess memory for cache if they miss the mark there as well. It is far easier to increase resources as needed than it is to remove. Until an organization *proves* it isn't just pulling specs out of its ass, we assume it is because they almost all do.
I don't care if your code is amazing with 32GB, 10,000 IOPs and <1ms to the database. It needs to be amazing with all parameters worse by an order of magnitude, because clearly-predictable circumstances will conspire to require it to be run in that fashion.
An old but useful example is when an organization decided to centralise their RDBMS Stateside, for reasons of data confidentiality, data management and RDBMS license conservation. An Eastern European office needed heavy database access. Because of poor infrastructure at the time, the Eastern European office was connected over satellite. Round-trip from geosynchronous means 700ms latency to the database. Do you have any idea what lack of parallelism and stored procedures are exposed by 700ms of latency?
Agreed. Anyone who is not testing and making the application function correctly with something like 100ms latency between the client and the server (not going to go 700 ms. An amusing number since I'm posting from a plane with 700 ms latency to everything right now though), you have no business writing software currently. Granted there are exceptions, but doing idiotic things like building an app to use cifs for more than just a sequential file write every now and then, database queries that return one result per round trip (or, more generally, ANY client that eschews the application tier and connects directly to a database), etc have no place now. Not with such a large trend towards moving the end points farther and farther from the datacenter.
I build all those development environments transatlantic, where there no chance that a network latency simulator will be misconfigured.[url=http://meincmagazine.com/civis/viewtopic.php?p=24595829#p24595829:1kr4fgwq said:ferzerp[/url]":1kr4fgwq]
Agreed. Anyone who is not testing and making the application function correctly with something like 100ms latency between the client and the server
Granted there are exceptions, but doing idiotic things like building an app to use cifs for more than just a sequential file write every now and then, database queries that return one result per round trip, etc have no place now.
Not with such a large trend towards moving the end points farther and farther from the datacenter.
[url=http://meincmagazine.com/civis/viewtopic.php?p=24595725#p24595725:16x0xqjc said:ronelson[/url]":16x0xqjc]Train wreck, remember?He's shown you the approved promotion paperwork and pay grade you'll upped to?
Say what? You have to take an exam for that?assuming I don't somehow botch the promotional exam.