what did you learn today?

Status
Not open for further replies.
Originally posted by afidel:
Originally posted by WingMan:
I learned that you can pull a drive out of the EVA for several seconds and pop it back in and the EVA will be all "Hey I know you, get back in your group!" and not have a problem. 2 disk failures and 2 lost loop connections in one disk group in one week, no data loss...


It's even a configurable timeout.


I can just imaging the EVA saying to the disk, "Get in mah belly!" -- :) --
 

bigmikebrooklyn

Ars Centurion
345
Subscriptor
Originally posted by bigmikebrooklyn:

Thanks Everyone for the ideas.
I think i have a design issue and bi-directional sync may not be an option. I have a conf. call with Netapp tomorrow, and you've all given me some good ideas. I'll talk with the experts and let you know what they suggest.
i'm thinking i'm going to seperate the HQ file shares and replicate one way for two sets of data, one to each location, overnight. possibly through DFS-R if I can get the VFM license.
Just to follow up:
Thanks for all the Ideas and help, I just redid the logon script for the shares and it tested OK. if the branches want to access HQ data, they have to go through our network line, slight added latency, but not a killer by any means. each set of shares will be robo copy replicated overnight to the opposite machine and we'll have acceptable DR until we can get teh other netapp in and the snapmirror thing starts working it's "changed blocks only" magic for even quicker overnight replications (in the mean time, thanking stars we have riverbeds).
Lesson learned: though you are using vendors to lighten your work burden, just do the work yourself and use them for double checking, because you can't always rely on an honest assesment from someone trying to sell you something, in conjunction with an outsider not always understanding your internal environment / needs.
8 years on the job and I feel like a n00b *sigh {kicks can, draws circle with toe}.
 

akro

Ars Scholae Palatinae
1,309
Why does this make me feel sick?

Originally posted by WingMan:
I learned that you can pull a drive out of the EVA for several seconds and pop it back in and the EVA will be all "Hey I know you, get back in your group!" and not have a problem. 2 disk failures and 2 lost loop connections in one disk group in one week, no data loss...
 

akro

Ars Scholae Palatinae
1,309
Don't worry I work for a vendor "HP" and I never trust a sales guy. Although I have met a few that are okay I tend to not trust them as they have too much vested. Maybe because I use to be a customer or maybe when they screw the pooch I am the guy who has to unscrew it. I dunno you decide...


Originally posted by bigmikebrooklyn:
Originally posted by bigmikebrooklyn:

Thanks Everyone for the ideas.
I think i have a design issue and bi-directional sync may not be an option. I have a conf. call with Netapp tomorrow, and you've all given me some good ideas. I'll talk with the experts and let you know what they suggest.
i'm thinking i'm going to seperate the HQ file shares and replicate one way for two sets of data, one to each location, overnight. possibly through DFS-R if I can get the VFM license.
Just to follow up:
Thanks for all the Ideas and help, I just redid the logon script for the shares and it tested OK. if the branches want to access HQ data, they have to go through our network line, slight added latency, but not a killer by any means. each set of shares will be robo copy replicated overnight to the opposite machine and we'll have acceptable DR until we can get teh other netapp in and the snapmirror thing starts working it's "changed blocks only" magic for even quicker overnight replications (in the mean time, thanking stars we have riverbeds).
Lesson learned: though you are using vendors to lighten your work burden, just do the work yourself and use them for double checking, because you can't always rely on an honest assesment from someone trying to sell you something, in conjunction with an outsider not always understanding your internal environment / needs.
8 years on the job and I feel like a n00b *sigh {kicks can, draws circle with toe}.
 
I have no idea, why does it? Is it because the EVA is a smart device that has quite a bit of tolerance built into it?

Originally posted by akro:
Why does this make me feel sick?

Originally posted by WingMan:
I learned that you can pull a drive out of the EVA for several seconds and pop it back in and the EVA will be all "Hey I know you, get back in your group!" and not have a problem. 2 disk failures and 2 lost loop connections in one disk group in one week, no data loss...
 

Whittey

Ars Tribunus Militum
2,006
Originally posted by WingMan:
I learned that you can pull a drive out of the EVA for several seconds and pop it back in and the EVA will be all "Hey I know you, get back in your group!" and not have a problem. 2 disk failures and 2 lost loop connections in one disk group in one week, no data loss...
That an x400 with older (more than a couple months) firmware?


-=Whittey=-
 

erratick

Ars Legatus Legionis
11,106
I learned that you can pull a drive out of the EVA for several seconds and pop it back in and the EVA will be all "Hey I know you, get back in your group!" and not have a problem.

Software ownership on netapp is similar (except one step further, if you trust your shipper not to kill more than 2 drives per raid group- you can ship the drives, put them back in any order mixed up and have your data).

Do not try with SUN gear.

I have been around when both happened (I advised against both). The Sun did not come back up.
 

Facekhan

Ars Legatus Legionis
10,392
Originally posted by akro:
Don't worry I work for a vendor "HP" and I never trust a sales guy. Although I have met a few that are okay I tend to not trust them as they have too much vested. Maybe because I use to be a customer or maybe when they screw the pooch I am the guy who has to unscrew it. I dunno you decide...

I have often said my job description as a delivery engineer is to make it work the way the sales guy said it would even if he didn't know what he was talking about.
 

akro

Ars Scholae Palatinae
1,309
No I think I am just overly paranoid. Of course in 10 years of storage work I have never lost a bit of customer data, there is always tomorrow.


Originally posted by Tsun:
Originally posted by WingMan:
I have no idea, why does it? Is it because the EVA is a smart device that has quite a bit of tolerance built into it?

Originally posted by akro:
Why does this make me feel sick?


Or maybe it doesn't have nearly the tolerance you think it does and akro knows something you don't. -- :D --
 

akro

Ars Scholae Palatinae
1,309
Luckily I have a very small customer base and I get to be involved on the presales side even though I am not presales. Keeps us from hanging ourselves. When I use to work the whole US I had plenty of emergency concalls with sales saying "uh yeah you sold them this to do what?"

My favorite of course is when the customers tells me he isn't sure what he bought. Had a fun time describing the advantage of his new Windows Storage Server versus his existing Windows File Servers.


Originally posted by Facekhan:
Originally posted by akro:
Don't worry I work for a vendor "HP" and I never trust a sales guy. Although I have met a few that are okay I tend to not trust them as they have too much vested. Maybe because I use to be a customer or maybe when they screw the pooch I am the guy who has to unscrew it. I dunno you decide...

I have often said my job description as a delivery engineer is to make it work the way the sales guy said it would even if he didn't know what he was talking about.
 

Metzen

Ars Scholae Palatinae
1,042
Originally posted by Rick25:
Originally posted by Rick25:
That the version of Robocopy that ships with Windows 7 supports multi-thread copy.

...and that it's native x64....which sucks if you're trying to use it on a x32 server....here's to a quick install just to grab 1 122kb file

You could use imagex.exe to mount the wim on the x32 installer and then extract the x32 version that way.
 

thatek16

Ars Scholae Palatinae
696
In the past few months I learned how to create a GRE tunnel bewtween our dispatch office and each of our radio towers. Then I learned the hard way why PIM Dense Mode shouldn't be used on low speed WAN links. Then I learned how to reconfigure PIM for Sparse Mode. All of this because I realized during our new radio system cutover that we needed multicasting enabled on our WAN links to each tower.
 
Originally posted by akro:
No I think I am just overly paranoid. Of course in 10 years of storage work I have never lost a bit of customer data, there is always tomorrow.
Now, that I can understand. I refused to do any pulling of the drives until the system was back to being in an "Ok" state even though they assured me it would be fine. Made sure the replication was working correctly and then did it. Being precautious is not a bad thing.
 

ronelson

Ars Legatus Legionis
21,399
Subscriptor
Now, that I can understand. I refused to do any pulling of the drives until the system was back to being in an "Ok" state even though they assured me it would be fine. Made sure the replication was working correctly and then did it. Being precautious is not a bad thing.
First thing we did when we saw our first RAID cabinet in '97 was throw a bunch of data on the drives, start up a process, and yank the drives. It is fun to see a spectacular failure when you yank two at once. That, or work for a phone company where you always work on things that are live. Either one will get you over the fear of whether a hot-swap will fail -- :) --
 

afidel

Ars Legatus Legionis
18,224
Subscriptor
Originally posted by ronelson:
Now, that I can understand. I refused to do any pulling of the drives until the system was back to being in an "Ok" state even though they assured me it would be fine. Made sure the replication was working correctly and then did it. Being precautious is not a bad thing.
First thing we did when we saw our first RAID cabinet in '97 was throw a bunch of data on the drives, start up a process, and yank the drives. It is fun to see a spectacular failure when you yank two at once. That, or work for a phone company where you always work on things that are live. Either one will get you over the fear of whether a hot-swap will fail -- :) --


I did the same when we got our last array, to test their bay redundancy feature I pulled both power cords from one of the bays. It worked just fine and since there was little data on the array rebuild time was reasonable (yaya for virtualized arrays, it only needed to rebuild the v-raids that were on the disks, not the entire disk).
 

Laslow

Wise, Aged Ars Veteran
199
I learned, or rather, confirmed, that the BC Provincial Government still makes no goddamn sense.

(Back in July)
My Boss: "We have a new position starting on September 21st, and need you to order a new laptop and get it setup for them."

Me: "Okay, I'll get the order off right away."

Boss: "No, you can't do that! You have to wait until September 21st, or they won't pay us back for it, and we'll be out-of-pocket for it."

Me: "Errr, so wait - we need the new laptop for as soon as they starts, but can't order it until they start?"

Boss: "Yep. Try to dig something up for them to use, okay?"
 

SideAngleSide

Ars Scholae Palatinae
1,472
Subscriptor
Originally posted by Laslow:
I learned, or rather, confirmed, that the BC Provincial Government still makes no goddamn sense.

(Back in July)
My Boss: "We have a new position starting on September 21st, and need you to order a new laptop and get it setup for them."

Me: "Okay, I'll get the order off right away."

Boss: "No, you can't do that! You have to wait until September 21st, or they won't pay us back for it, and we'll be out-of-pocket for it."

Me: "Errr, so wait - we need the new laptop for as soon as they starts, but can't order it until they start?"

Boss: "Yep. Try to dig something up for them to use, okay?"


If it makes you feel better, many companies are like this. So have comfort in the fact that the idiocracy does not exist in a vacuum, and that you arent crazy.
 

Laslow

Wise, Aged Ars Veteran
199
Originally posted by indigo258:
Originally posted by Laslow:
I learned, or rather, confirmed, that the BC Provincial Government still makes no goddamn sense.

(Back in July)
My Boss: "We have a new position starting on September 21st, and need you to order a new laptop and get it setup for them."

Me: "Okay, I'll get the order off right away."

Boss: "No, you can't do that! You have to wait until September 21st, or they won't pay us back for it, and we'll be out-of-pocket for it."

Me: "Errr, so wait - we need the new laptop for as soon as they starts, but can't order it until they start?"

Boss: "Yep. Try to dig something up for them to use, okay?"


If it makes you feel better, many companies are like this. So have comfort in the fact that the idiocracy does not exist in a vacuum, and that you arent crazy.

*That's* about as comforting as someone saying, "Don't worry, if it breaks, you'll probably die instantly." :/
 

scorp508

Ars Legatus Legionis
10,214
Originally posted by Laslow:
I learned, or rather, confirmed, that the BC Provincial Government still makes no goddamn sense.

Have no fear, state government here is the same.

Me: If we wait 2 months we can save 45% on our capital expense for the upgrade.

My Boss' Boss: Put together quotes to migrate next month.

Me: If we wait 2 months we'll not only save 45% on our CapEx, but our OpEx will go down 15% and our datacenter footprint will drop by 35%.

My Boss' Boss: What did I just say? My boss made promises to move by November so just make the quotes happen. End of story.

Me: <deer in the headlights while tax money is absolutley wasted>
 

montegard

Ars Scholae Palatinae
996
Originally posted by Metzen:
I learned Windows Server 2003 R2 "Print Management" is retarded. There is no way to set a Default Printer? WTF kind of oversight is that!?!?!?!?

I think even in 2008 you'd have to set that via a group policy. Print Management just lets you set printing defaults such as tray, color, things of that sort.
 

Metzen

Ars Scholae Palatinae
1,042
Originally posted by montegard:
Originally posted by Metzen:
I learned Windows Server 2003 R2 "Print Management" is retarded. There is no way to set a Default Printer? WTF kind of oversight is that!?!?!?!?

I think even in 2008 you'd have to set that via a group policy. Print Management just lets you set printing defaults such as tray, color, things of that sort.

You can't set the "Default Printer" even via group policy. The only way to set it is via a startup script.
 

pokrface

Senior Technology Editor
21,557
Ars Staff
Oh, I've got a winner.

What did I learn today? I learned that the version of RHEL 4 I have deployed includes a feature wherein if there are a lot of delayed writes, the affected file system will be set to read-only in order to preserve its integrity. I found this out on a (very, very) busy VMware server, where the sudden conversion of the underlying file system to read-only, rather than protecting anything, instead caused the corruption of three VMs. All test boxes, sure, but it certainly wrecked my afternoon.

Make a change to protect data that corrupts data. Gee, thanks. This behavior is corrected in later 2.6 kernel revs, but arcane change control rules mean I can't upgrade. Damn it.
 

Metzen

Ars Scholae Palatinae
1,042
Originally posted by llib:
I am learning the Remedy training is incredibly boring. I will continue to learn that for the remainder of this long, long day... -- :( --


Learn to script it! Remedy had a pretty challenging scripting language the last time I used it (3 years ago). By challenging, I mean it was pretty stupid and complicated.

BUT(!). It was pretty rewarding to create a script where you didn't have to open that gawd awful GUI that took 3 minutes to open. Just put in a ticket number, comment, and it put in my username and all the other misc information and auto-set it to close or whatever.
 

afidel

Ars Legatus Legionis
18,224
Subscriptor
Originally posted by Pokrface:
Oh, I've got a winner.

What did I learn today? I learned that the version of RHEL 4 I have deployed includes a feature wherein if there are a lot of delayed writes, the affected file system will be set to read-only in order to preserve its integrity. I found this out on a (very, very) busy VMware server, where the sudden conversion of the underlying file system to read-only, rather than protecting anything, instead caused the corruption of three VMs. All test boxes, sure, but it certainly wrecked my afternoon.

Make a change to protect data that corrupts data. Gee, thanks. This behavior is corrected in later 2.6 kernel revs, but arcane change control rules mean I can't upgrade. Damn it.

Ok, I've got about as bad of change controls as I've personally encountered and even we would patch a data corruption bug. What possible justification can your QA or Internal Audit department have for keeping you from patching a bug of that magnitude?
 
Status
Not open for further replies.