what did you learn today?

[PVO]

I've learned that you shouldn't fix what was never broken.

Looking at you IBM. -- --

Your previous gen server rails were awesome.

These new ones... not so much.

 

[PVO]

<BLOCKQUOTE class="ip-ubbcode-quote"><div class="ip-ubbcode-quote-title">quote:</div><div class="ip-ubbcode-quote-content">Originally posted by Snuble:<BR>I discovered that not all companies have a "hit by bus" action plan. </div></BLOCKQUOTE><BR><BR>I rediscover, at least once a week, that I am the only voice in the org pointing out the lack of a "hit by a bus" action plan for anybody. Of course, the "hit by a bus"-like events in the last year have had zero impact on action, but a lot of impact on workflow and costs.<BR><BR>It's comforting to know my pain is shared.

 

[PVO]

I've also had zero issues with any of the 27xx and 34xx series - deployed a few dozen 16,24 and 48 port and they just work.<BR><BR>Today I learned that just because I have a full set of utensils in my desk in case I have to eat lunch there, I shouldn't be so quick to whip out the mean looking steak knife when a co-worker has a birthday cake in need of cutting.<BR><BR>Then again, I like that glimmer of fear. lol

 

[PVO]

I learned that despite the last time project management wanted a solution that would be a month in the works by <I>tomorrow</I> (with no advance notice they would need it), their next request won't be for 1 complete turnkey system by tomorrow, but two.<BR><BR>Oh, and they already got the vendor to install the components for one, without telling the technical teams, or taking into consideration how the damned app will communicate with the outside world.

 

[PVO]

<BLOCKQUOTE class="ip-ubbcode-quote"><div class="ip-ubbcode-quote-title">quote:</div><div class="ip-ubbcode-quote-content">Originally posted by Danger Mouse:<BR>And HR sends interviews through an automatic processor that checks for keywords. If MCITP isn't in it, it'll get ignored.<BR><BR>Those job listings had MCSE on them. </div></BLOCKQUOTE><BR><BR>Sounds like you need to register for some courses, then put "MCITP / MCSE - In Progress" on your resume. <img src="http://episteme.meincmagazine.com/groupee_common/emoticons/icon_wink.gif" alt="Wink" width="15" height="15">

 

[PVO]

Today I learned that it's outright hilarious to watch the ripples from a manager asking their own staff to fill out an assessment to have an external vendor provide the services of that team (thereby replacing them).

 

[PVO]

Today I learned (again) that when a vendor loves to blame everything in your infrastructure for performance issues but their software... it's very awesome to have a robust monitoring solution that proves otherwise.

 

[PVO]

And that I hit quote instead of edit. These new forum buttons are tricky.

 

[PVO]

Today I learned (again) that when a vendor loves to blame everything in your infrastructure for performance issues but their software... it's very awesome to have a robust monitoring solution that proves otherwise.

What monitoring solution is that?

Orion. Network performance monitor + application performance monitor = detail, detail, detail.

 

[PVO]

I learned today that Microsoft just doesn't give a damn about their customers, or us poor admins.

Remove the simple copy functionality to tweak and set the local default profile in 7 and 2008 R2? Require you to sysprep the machine as the only "officially supported" way?!

Stupid, stupid, stupid.

Thank goodness for 3rd party workarounds. Of course, 5 years ago, I would never use a workaround in production. My, how times have changed.

 

[PVO]

I prefer my solution. Get jobs where you have little to no contact with them instead.

+1

 

[PVO]

Trying to figure out whether I want to switch over from mRemote. If it supported centralized storage for multiple users (SQL DB, or just a file on a network share) I'd do it in a heartbeat, but as it is... not sure. It does have some pros, but I don't think they're enough for me to manually migrate 200+ mRemote connections.

I'm not seeing a compelling reason to switch from mRemote so far - got tons of connections, groups, and everything already configured there, so there's no "shiny" to make me do that work all over again for RDCman.

This weekend, I learned how nice and quiet my weekend can be when I turn my BlackBerry off. It didn't hurt that the weather cooperated and we had a fantastic start to the summer.

 

[PVO]

Even with my project in the testing phase, and less than a month away from live, it's still subject to a total rework. I'm so glad I'm in a frame of mind where I can point out the time remaining cannot accomodate the new work involved. I like deadlines - I like to wave as they pass me by.

 

[PVO]

The last time I had to deal with something like that was an employee that was tasked with doing a clean install/upgrade on a customer's servers to go from win2k3 to 2k8R2.

del *.*

Fortunately it turned out they had already been offline more or less for the last 4 months so the data in question was basically worthless and they welcomed us just starting fresh and doing things right.

I had that situation happen several times in the past, unfortunately, it was usually a mission-critical app or server with a specialized app on it, and special restore / recovery procedures. My favorite time was when the fellow that never documented anything and kept all his passwords, access, and knowledge known only to himself (job protection FTW) was out of the country, and totally unreachable for at least 72 hours.

Good times.

 

[PVO]

Sounds like you need a change management process, or at least a refresher course for Bob.

Change is all fine and dandy, until it slows the wrong people (executive / upper management) down, and then it just gets approved as a standard change, with no review or documentation anyways.

 

[PVO]

I'm on vacation.

That goes into effect for me this Thursday; and what I've learned is that when I'm on vacation, I am completely unreachable.

While I recognize that there is a certain diligence to the job, and all the political BS that goes with it, I usually both deserve, and need my vacation at that point. I would argue that a fair amount of folks posting here deserve and need a vacation even more than I. Coupled with the fact that every time I've had a crisis or situation that requires executive involvement, and said executive is "on vacation", that situation has to wait until their return. Turnabout is fair play.

Besides, you always have to factor for the bus/lottery factor. As in, "hit by a bus" or "won the lottery" factor removing access to someone, permanantly and immediately.

 

[PVO]

while fixing a few different broken bladepcs, I found out that the VP who was using one, stored his stuff on the desktop. Awesome. Pity we didn't have that desktop folder redirect enabled yet. At least I found his stuff in his network user folder. Most of his stuff.

That turned a potential RGE into a "be sure to save your stuff in your network user folder" conversation.

Luckily, he's not the VP directly over us, or even strictly speaking in our chain of command. That was a few uncomfortable minutes, where I was nearly hyperventilating, until I found his 3gb of email archive files.

Place like that doesn't deserve a good IT fellow like you. I've walked away from nice paying jobs because of the amount of FUD and / or stupidity the organization is willing to lay at the feet of IT. There has to be a balance to the paycheck and sanity / stress, otherwise I'm out the door.

Then again, I'm in this magical place where my amount of care stops at a certain point. And that point's been reached a lot lately lol.

 

[PVO]

Today I learned that no matter how many PowerPoint presentations, discussions, or explainations I give to upper management, most of them are just too obtuse to understand anything technical. Even though they come from a *cough* "technical background".

Also, it's hilarious to be on a conference call where an attendee doesn't know how to mute their headset, and proceeds to cough, breath heavily, slurp a drink, and then go to the bathroom.

 

[PVO]

Today I learned how hilarious it can be when discussing a CPU add to a VM, when someone asks "can we roll back to the original config if that CPU is defective?"

 

[PVO]

That when you're testing a VPN and you see 4x the throughput in one direction than you do the other, don't spend a long time trying to get to the bottom of it, just try a different laptop as guess what, it's probably the laptop NIC

+1 on that. Too bad to balance that out we have a help desk fellow who puts in a top priority ticket when 2 users have a like problem because OMG! MORE THAN ONE USER CALLED! THE APP IS DOWN FOR EVERYONE!

 

[PVO]

The last people to follow directions properly for workstation software are people in "close to IT" departments like app dev, app support, first-tier helpdesk, etc. People that you've briefed twice and advised at least twice more via email, who promptly ignore the deployment directions and "try" to install it "their way".

Roll-back? Issue caused by improper procedure is resolved via a reimage.

 

[PVO]

I've been slowly learning that whatever I've recommended anywhere from 2-3 months in the past, when denied then, is proposed by someone else as a solution now. And that I take great joy in pointing that out.

THEN
Me: With this new solution we're providing, we need to facilitate and perhaps package the required software installs for our clients instead of hoping they have 3rd party tech support that can read.
Them: NO! We don't support end users in any way. We'll post the links, and they're on their own.
Me: ... well, ok then.

NOW
Them: OMG! Our clients aren't smart! They can't read! They don't have 3rd party tech support!!! We need to provide an in-house consolidated software install for them RIGHT NOW!
Me: ... sigh.

 

[PVO]

And then there's people who fall for the 419 scam after being warned repeatedly that it's not real.

Anyone falling for this should be fired. They are grossly incompetent and gullible. They are not of the caliber of people you want making decisions in your company.

This. Stupid people should not be tolerated. Especially when their stupidity causes us undue stress and lots of $$. Making a mistake is one thing... being stupid is a birthright.

 

[PVO]

Have either of you worked in higher education? Are you saying you would expect ramifications or just that there should be?

I have. I expect ramifications... especially if a downtime costing several hours and $$$ occurs. However, more often than not, the blame is placed on someone else for that evil spam email making it to someone's inbox in the first place. And then usually, after a few months of stringent filters in play, someone else complains about not getting a "critical email from a vendor's external account", and the strings are loosened again. And then someone clicks on the 1 piece of spam that gets through. It's a fun little circle.

Life is a rollercoaster of ups and downs. IT is no different. I'm just tired of folks blaming ignorance and / or incompentance when it comes to their PC. If they don't know what they're doing, whether it's one of my domain admins or the receptionist, they shouldn't be fucking touching it.

 

[PVO]

A client calls into the org about a trivial issue that is well documented and easy for help desk to resolve. But this fellow bypasses help desk because of a superiority complex, to have his issue resolved by someone he knows in middle-management (that is completely ignorant of said software). Where do they end up? The illusion of assistance, no resolution, and much more frustration.

I intervene in the email chain after it makes it to my inbox, after days of bouncing all over the realm of the org (never sent or cc'ed to anyone who can help, just middle management and the wrong department). I shake my head as I read through the chain - in some parts they reference the people they should be dealing with, but never include them in the communication. No IMs to these folks, cell calls, emails... nothing.

I jump in, and in 10 minutes I get the help desk to take ownership and resolve it. 10 freaking minutes.

I'm not sure which is worse at this moment, the client for bypassing process, or the folks on our side that accommodated them.

 

[PVO]

Previous management here fought the good fight trying to keep folks honest with actually charging for storage used on the SANs. Now it's a free-for all and we're nearly out of space and swamped with silly requests.

There's another kind of hell for folks who won't tolerate a downtime to delete their 2008 R2 4GB hibernate file, but will tolerate one for 4GB more of 'freebie' SAN space.

And I laugh, and I laugh...

 

[PVO]

I was the lone sysadmin in a room of senior managers, and I think they didn't take me seriously

I'm running into this more, and more, even at my management level sideways with non IT managers, and with the executive. Why employ us if you won't listen to us?

"Don't listen to the guy in the room that knows our infrastrucutre inside and out, and we pay big bucks to because I read this article in Shittech Magazine and OMGITSAWESOME."

Yes, I'm horrified that someone in IT would cavalierly suggest that you just open 3389 to the internet.

I'm running into more and more people that *should* know better, especially when you take into account their training and background. These usually end up being the brown-nosers that want to be the "yes man / woman" that management likes to keep around, and question logical decisions because they have some fractured belief of "what's right for the client / business".

 

[PVO]

WHEN I TELL YOU TO GIVE ME ADMIN RIGHTS ON YOUR VCENTER SERVER GIVE ME ADMIN RIGHTS GOD DAMN IT

Only if you can let me trim my excessive domain admins from people that have no business having a domain admin account.

 

[PVO]

Today I learned how much fun it is to record (and mention it while I'm recording) conversations with vendors who aren't exactly... honest... and then play them back for reference.



Now waiting for something negative to happen to balance out this newfound happiness.

 

[PVO]

Today I relearned that no matter how polite you are in asking for other departments' consideration to not put in requests for change over the holiday break because 2/3 of your staff are AFK, that some knuckleheaded, dumbass, senior exec will demand an emergency change... to a test system... because the vendor told him it was important. Which it isn't. Which we explained. Thrice.

My staff is ready to revolt.

Inconsiderate fucking senior management. May Santa shit in their stockings.

 

[PVO]

Did I mention that there is a change freeze? And that the execs overrulled the change board and told them to do it anyway?

Of course, I couldn't help but point out we don't need change boards / meetings if the recommendations are overturned.

 

[PVO]

Digicert's SSL Checker Tool is the bomb. IIS installed the certificates, but missed the intermediates. No errors, no logs, we're all scratching our heads because cert checking was s l o w. Fixed.

 

[PVO]

Speaking of the tactical nuke button... is it wrong that all the males in my family (including my dad and 9-yo son) all mumble "aft torpedos FIRE!" in our best Montalban-y voice when we have flatulence?

Back on topic, today I learned that executives and senior management lie. A lot. And it's entertaining to catch them doing so. Engage backpedal device. FULL POWER! DAMN YOU!

 

[PVO]

We're back down to two sysadmins for all virtualization/servers/AD/Exchange/storage/Citrix/anything else.

For a company with 18,000 employees.

Two sysadmins
18,000 employees

But the Security department has 11 employees.

We're a little bigger org wise than you. We also have a security department that outweighs our entire server / infrastructure staff.

We had a great team 2 years ago that had enough staff and management to stay on top of operations and project work without getting lagged behind.

We're now half the size (8) with no proper management, and have just been told we're actually only "funded" for an FTE less than what we have.

You can imagine how low our motivation, morale and overall quality of work is. Sick days have skyrocketed, stress levels are through the roof, and on top of it all, our senior management has the gall to patronize us.

The needle on my "give-a-fsk-o-meter" isn't even twitching.

 

[PVO]

I think he meant, why does Security have 11 peeps?

A better question: Why are half of them domain admins?

Even better - Why are ANY of them domain admins? If compliance is at all a concern, there is a huge red checkmark awaiting which reads: "Segregation of Duties"

Pffft.... we have over 50 domain admins. Only about 8 of us actually need that level of access.

 

[PVO]

Oh we have separate admin accounts. Thank god for that. I've been a pusher of that myself almost my entire IT career. We even have admin accounts for local server admin so that we don't have regular user GPOs applying all over the place. But we have stupid people being domain admins. Especially managers that have no business fiddling in there because they're technically incompetent. It's entertaining when they do screw something up though. When our systems go down, people's lives are potentially at risk.

Our security group is responsible for access requests, group management, privacy and overall security. I'm not knocking what they do - 90% of it has value - but the remaining 10 is just useless twaddle.

Hey... you guys remember that 2008 R2 RemoteApp environment I was building a while back? Oh man, it was all nice and shiny. Worked very well. Unfortunately, senior exec has decreed that users cannot possibly be expected to endure the 20-50 seconds to have the environment log you in (because 5 people out of the 5k that use it complained), so we jacked the timeouts to 16 hours. All of the timeouts.

The funny side effect is that server performance is a dog, due to the dozens of disconnected sessions on each box using up resources, and login times have skyrocketed to 2+ mins.

It's annoying having something that works well torn asunder by people that have no clue.

 

[PVO]

Did I mention that a fairly influential user complained to senior exec about having to type "domain/" before their username? And for context, the amount of typing required is close to "domain/user", not "dominacompanynameblahblah/user1235.roflcopter".

I quote from the email...

It pains me to have to type in domain/. This a tedious impact to my productivity.

DIAF.

 

[PVO]

IE does save the domain/user on RDWeb, but people are still dumb. I can't / don't manage users' home / external office PCs either.

It's minorly annoying to have to type it, agreed. But to complain that it impacts your productivity?! It's like users who complain that an icon's shade of green isn't appealing.

Srsly?

Another personal favorite from an in-house technical manager this time:

It asked me to install .NET. I don't have enough space on my hard drive for that!

/facepalm

This project has really brought out the stupid in people I thought I could expect better from.

 

[PVO]

It was the cleanest, most pristine pallet I've ever seen; I actually still have it at home in my garage.

:eyebrow: Funny, I have a couple of those too.

 

[PVO]

High priority ticket hits our storage team...

I have files on my Z: drive that are critical to my work. They are missing! Can you restore from backup please?!! ANNOYING!

3GB of family photos and videos. Why.