what did you learn today?

Status
Not open for further replies.

stash

Ars Tribunus Angusticlavius
6,858
Originally posted by Barmaglot:
Originally posted by Rick25:
Why does it have to be so early? Use Storage vMotion to move the VM's while they are up and running during the day.

AFAIK, Storave VMotion only works on FC in ESX 3.5. You need 4.0 for it to play with iSCSI and NFS. In any case, no license for it.

That is incorrect, at least for iSCSI. Not sure about NFS. It just isn't exposed in the VI client.
 
I spent an hour today fighting with the "export-mailbox" cmdlet on Exchange.

It seems there is a bug somewhere that makes it so you can't export to a PST file if you use the -startdate and -enddate options. However you can use those option to filter the contents into another mailbox and then dump the entire contents of that mailbox out to a PST file. -- :mad: --
 

bigmikebrooklyn

Ars Centurion
345
Subscriptor
I learned that if you have a polycom system and you are trying to troubleshoot why international ISDN video calls can not come in or out, it will take 2 weeks of back and forth finger pointing between your PBX vendor and your PRI provider before they finally figure out that A: maximum digits needs to be changed and B: the PRI vendor has a PIC code sending all international D-channel traffic over to MCI because they have no terminating agreements, and that they only applied that PIC code to 1 of your 2 PRI/trunk groups, so that when it hit the first PRI/trunk group and found it busy, and rolled to the next, there was no PIC code to route it to MCI.
and also C: certain polycomm and picturetel units will not communicate with each other. even though i'm pretty sure polycomm ended up aquiring picturetel.

That was a joyous task.

I also learned that even though the documentation says you can use ESX based VMware guests to create a riverbed RSP win2k3 package so you can have a virtualized win2k3 server on your branch office steelheads (gonna be print servers, EPO and WSUS repositories, i hope), that they are actually full of sh!t and only VMware server 2.0 guests will work to make the RSP package.
Dammit, MONOLITHIC disk -- :mad: --

i also am continually reinforced taht riverbed has one of the best support orgs out there, and that i can forgive one piece of bad info in one of their docs for the other 30,000 times they have pulled my ass out of a firey inferno.

it was a day of culminations today.

i also learned that if you go to boston during the thanksgiving weekend and drink "beer punch" at cambridge commons with your cousin and his lackeys, you will come back with the cruddiest sinus infection ever. swine flu got nothing on this crap. and no sick day in sight.
 
Originally posted by bigmikebrooklyn:
i also learned that if you go to boston during the thanksgiving weekend and drink "beer punch" at cambridge commons with your cousin and his lackeys, you will come back with the cruddiest sinus infection ever. swine flu got nothing on this crap. and no sick day in sight.

I love that place and that stuff. Circus Boy poured into lemonade and vodka? Yes please. If you want to drop a serious couple bucks though, next time you're up here, is Lord Hobo on Hampshire St. Frickin fantastic selection of draft belgians, like you've never seen.
 
Originally posted by Spatula:
Although I walk by it every day to and from work, I have yet to go to Lord Hobo (or Pickwick Ale, as I call it). I was never a fan of the B-Side, and I think some of that lingering feeling has stuck.

It's a new owner, the guy who owns Moan and Dove in Amherst and the Dirty Truth in Northampton. They still have good cocktails, but the emphasis is on the crazy-good beers. It's expensive, no joke, but eminently worth it.
 

Rick25

Ars Tribunus Militum
2,511
Subscriptor
Originally posted by Barmaglot:
Originally posted by Rick25:
Why does it have to be so early? Use Storage vMotion to move the VM's while they are up and running during the day.

AFAIK, Storave VMotion only works on FC in ESX 3.5. You need 4.0 for it to play with iSCSI and NFS. In any case, no license for it.

Works great in 3.5 with iSCSI, if you have vMotion you also have Storage vMotion it's not a extra license feature in 3.5, not 100% sure about it in 4.0
 

Barmaglot

Ars Legatus Legionis
11,783
Subscriptor
Originally posted by erratick:
Not sure about NFS


Storage VMotion works on NFS as well. I only had 3 until recently:

http://lewaness.wordpress.com/...tion-gui-in-vcenter/

Works awesome.


Thank you for this -- :) -- I only checked this thread and saw your post after migrating 4 VMs out of 12, but this is definitely helpful. I installed the plugin and can confirm that it works on 3.5 with iSCSI and NFS. Hopefully I won't run into problems with free space for the snapshots it creates - those tiny individual datastores only have a gig or two free - but with system volumes, at night, it shouldn't be an issue.
 

afidel

Ars Legatus Legionis
18,224
Subscriptor
Originally posted by bigmikebrooklyn:
I learned that if you have a polycom system and you are trying to troubleshoot why international ISDN video calls can not come in or out, it will take 2 weeks of back and forth finger pointing between your PBX vendor and your PRI provider before they finally figure out that A: maximum digits needs to be changed and B: the PRI vendor has a PIC code sending all international D-channel traffic over to MCI because they have no terminating agreements, and that they only applied that PIC code to 1 of your 2 PRI/trunk groups, so that when it hit the first PRI/trunk group and found it busy, and rolled to the next, there was no PIC code to route it to MCI.

Sounds oddly familiar, had a similar problem in 2001 with SBC, they somehow dropped LD service from the second B channel on line 3 of a 6 line bundle in some kind of billing snafu. We could connect at 256Kbps but the quality was awful, 384+ connections dropped on negotiations. Took 3 truck rolls, an ISDN protocol analyzer and finally a tech who decided to call a remote loopback phone number on each B channel before they figured it out.
 

Ciconia

Ars Praetorian
567
Subscriptor
Originally posted by Rick25:
Originally posted by Barmaglot:
Originally posted by Rick25:
Why does it have to be so early? Use Storage vMotion to move the VM's while they are up and running during the day.

AFAIK, Storave VMotion only works on FC in ESX 3.5. You need 4.0 for it to play with iSCSI and NFS. In any case, no license for it.

Works great in 3.5 with iSCSI, if you have vMotion you also have Storage vMotion it's not a extra license feature in 3.5, not 100% sure about it in 4.0

Not to beat a dead horse here but I'm rather sure vMotion works with iSCSI and 3.5. If it does not, someone needs to tell my little 3.5 cluster it's doing something it cannot. I've moved live disks lots of times over iSCSI. There is just no VMware provided GUI to do it. A third party plug-in adds a simple GUI to the situation.
 
Sounds oddly familiar, had a similar problem in 2001 with SBC, they somehow dropped LD service from the second B channel on line 3 of a 6 line bundle in some kind of billing snafu. We could connect at 256Kbps but the quality was awful, 384+ connections dropped on negotiations. Took 3 truck rolls, an ISDN protocol analyzer and finally a tech who decided to call a remote loopback phone number on each B channel before they figured it out.



Same. PacBell and ATT did this to me at different times. ISDN most be fully misunderstood everywhere now. -- ;) --
 

afidel

Ars Legatus Legionis
18,224
Subscriptor
Originally posted by erratick:
Sounds oddly familiar, had a similar problem in 2001 with SBC, they somehow dropped LD service from the second B channel on line 3 of a 6 line bundle in some kind of billing snafu. We could connect at 256Kbps but the quality was awful, 384+ connections dropped on negotiations. Took 3 truck rolls, an ISDN protocol analyzer and finally a tech who decided to call a remote loopback phone number on each B channel before they figured it out.



Same. PacBell and ATT did this to me at different times. ISDN most be fully misunderstood everywhere now. -- ;) --


At my current employer we moved our partners to H.323 over IP because once we setup our gateway and got a local provider to setup their end it has *just worked* and as a bonus it's significantly cheaper to operate than a whole bunch of international LD calls on ISDN lines =)
 

Cool Modine

Ars Tribunus Angusticlavius
9,194
Subscriptor
I once spent months troubleshooting dropped video teleconference calls... international long distance ISDN from Maryland to the UK. Dealing with one telco is hell enough; in this case our calls went from Verizon to Sprint, across the Atlantic to BT and then to Cable and Wireless. Naturally, everyone pointed their fingers at their neighbor. Eventually we got Sprint to find an issue with one of their transatlantic trunks, and everything worked reliably after they fiddled with the routing.
 

ronelson

Ars Legatus Legionis
21,399
Subscriptor
Still waiting on McAfee to come up with something. 2 hours later. Go Platinum support!
Just curious, since we have platinum support as well. Did you guys get the email about reduced hours for Thanksgiving last week? That is total bullshit - that is when we need the support the most, when half of our workforce that has experience with the product is unavailable. Of course, our company buys the lowest level Platinum contracts (or last year's platinum level and pays extra for the sub-par level) so I am curious if it is just us or their standard support level.
 

Whittey

Ars Tribunus Militum
2,006
Originally posted by ronelson:
Still waiting on McAfee to come up with something. 2 hours later. Go Platinum support!
Just curious, since we have platinum support as well. Did you guys get the email about reduced hours for Thanksgiving last week? That is total bullshit - that is when we need the support the most, when half of our workforce that has experience with the product is unavailable. Of course, our company buys the lowest level Platinum contracts (or last year's platinum level and pays extra for the sub-par level) so I am curious if it is just us or their standard support level.
Yah, regular (daytime phone call) platinum support was closed. You could still call your SAM and have them get someone intelligent on the line if there was an issue. It's better than HP's "we're taking the last couple weeks off" policy.

So now we're at 15 hours mark. Two SAM's, voicemail for the overall account manager and very little. I have an extra.dat. I have a name. And I have an email that says "Attached is the extra.dat for this detection. Please use the beta.dats for registry cleanup." No more information. Solid platinum...


-=Whittey=-
 

ronelson

Ars Legatus Legionis
21,399
Subscriptor
Yah, regular (daytime phone call) platinum support was closed. You could still call your SAM and have them get someone intelligent on the line if there was an issue. It's better than HP's "we're taking the last couple weeks off" policy.
Ha, okay, same thing. That is not platinum in *my* book, but since I do not matter... FWIW, our HP deals are 24/7/365 at 4 hour and we have never had an issue with it. McAfee are the only people that take off for the holidays at the top tier support level.
 

jshiplett

Ars Tribunus Angusticlavius
6,710
Subscriptor
Originally posted by Sunner:
I learned that working at a place where management lets the developers dictate how the production environment should look like instead of leaving that to the ITO department will inevitably lead to endless pains for the latter. Not to mention for the company as a whole.

I hear ya. "No, we cannot operate our single-threaded application infrastructure without 10 x 4 vCPU VMs running on 3 dual-quad-core blades."

Hello CPU ready time.
 

gregthomas

Wise, Aged Ars Veteran
189
Originally posted by V_Max:
I learned that it's possible to create a RAID10 array on an R710 consisting of two 73GB 15K disks and two 300GB 10K disks. I learned this because we received it from Dell that way.

Sometimes their LUNS and partition makes no sense. Like they throw darts to decide how to carve it up. Oh well, rather stage the hardware from scratch myself anyway.
 
Trying to determine what line cards, supervisors and chassis work together for Cisco gear can be a nightmare; especially when two pages on Cisco's site contradict each other.

http://www.cisco.com/en/US/pro...900aecd8035cf2b.html :: "The supervisor engine IV is compatible with the classic Cisco Catalyst 4500 and the E-series Cisco Catalyst 4500 Line Cards."

http://www.cisco.com/en/US/pro...ucts_Data_Sheet.html :: "Cisco Catalyst 4500 E-Series Line Cards -- Supervisor Engine 6-E, Supervisor Engine 6L-E,"

Which is it Cisco?
 
Originally posted by Digitali:
I learned today that if you delete a user's mailbox in Exchange 2007, it will also delete the associated AD account.


Ouch, you got bit by the "Remove" command. -- :( -- Have no fear, everyone does. It doesn't help that in Exchange 2000/2003 "Remove" meant "delete the mailbox, leave the user alone" and then they changed it in 2007/2010.
 

Soko

Ars Praefectus
4,068
Subscriptor++
That you don't fsck with database server unless the Netflow Analyser machine that uses it is shut down. Go-go gadget database re-init!

That it's near impossible to debug a HP network team on a full flight production server. I have between 8pm and 9pm each day to figure out why the bitch keeps dropping connections, and I'm lucky to get that. At least the HP support site has a "Submit a ticket", or I'd have to wade through the support phone line.

"Wassat honey? Oh, you bought Christmas gifts? Nice! Who's this fo - er wha? Not enough money left for booze this weekend? Nice."
 

WhiteHat

Ars Scholae Palatinae
1,280
Originally posted by BajanDude:
Trying to determine what line cards, supervisors and chassis work together for Cisco gear can be a nightmare; especially when two pages on Cisco's site contradict each other.

http://www.cisco.com/en/US/pro...900aecd8035cf2b.html :: "The supervisor engine IV is compatible with the classic Cisco Catalyst 4500 and the E-series Cisco Catalyst 4500 Line Cards."

http://www.cisco.com/en/US/pro...ucts_Data_Sheet.html :: "Cisco Catalyst 4500 E-Series Line Cards -- Supervisor Engine 6-E, Supervisor Engine 6L-E,"

Which is it Cisco?

FWIW We have a 4507E and use the supervisor IV cards without any issues.
 

ronelson

Ars Legatus Legionis
21,399
Subscriptor
SMARTS In-Charge is a total piece of shit. I knew it was a piece of shit, but I just learned that it is 100% shitty.

I import a seedfile that has the new SNMP string for 100+ devices. They all fail, yet snmpwalk works fine. I experiment by deleting a device and then importing it - works fine. So, I yank the devices out and reimport them from scratch. SMARTS locks up importing a single device for two hours. Suddenly it dumps a ticket for every modified device with a down trap. HOW? I just deleted them, there is no up state established! DIE DIE DIE!
 

ambit

Ars Tribunus Militum
1,982
Eve ate my rant about the Exchange 2007 installer running perfmon invisibly in the background. -- :( --

Long story short.. somehow a brand new Server 2008 ENT build would crash out perfmon everytime you added counters to it.
It took about 3hrs with MS Support to figure out that is what was crashing the Exchange installer.
Then another hour trying hotfixes, regedits and resetting the perfmon counters to get the Exchange 2007 installer working again.
About 15mins later I had my 5th node back and running in my cluster.

Good news:
While I had this downtime I ran some offline defrags and cleaned up ~900Gb from my Exchange stores -- :D --

[edit]
Here's the hotfix we ran: http://support.microsoft.com/kb/961382 My Exchange nodes have about ~400 multipathed LUN's presented to them, which seems to be what caused this.
 

ambit

Ars Tribunus Militum
1,982
Figures... it lets me post that^^ just fine. Grrrr.
Been awhile since I posted here, have some catching up to do!

Please note, if you have a vendor running their 'Emergency Department Information Systems' from an Oracle database on a Win2003 cluster, they probably don't know what they're doing.
They will somehow pick the one time that a Oracle backup is running (that no one knew about/scheduled??) to fail the database over.

This will:
Corrupt the database
Cause a 10min outage to run 2.5Hrs
Require a complete restore of the 20Gb .ora from RMAN
Cause 5 hospital ER's to fall back to paper on a Friday night
And end a ~$15 million/50 hospital rollout at 5 hospitals and a lawsuit.

They will also under-spec their required enviroment by almost 70% and cause a ~$200,000 budget overrun in server hardware alone.
Fun times!
 
Status
Not open for further replies.