I learned that the Linux drivers for Ath9k 802.11n cards still don't work right (on the 2.6.31.x kernel). :/ Compiling 2.6.32 to see if any of the changes made to the drivers actually allow me to have a signal for longer than three minutes.
what did you learn today?
- Thread starter chalex
- Start date
- Status
Learned that Vista can lose network connectivity due to flaws in the IPF. There's a hot fix for this that fixs it, but apparently MS patched this in Windows 7 and didn't back port it to Vista. -- 
--
Of course this problem reared it's head on a Vista management workstation that has been really stable over the last 12 months. Of course said workstation was being used to migrate databases at the time of failure... -- --
--Of course this problem reared it's head on a Vista management workstation that has been really stable over the last 12 months. Of course said workstation was being used to migrate databases at the time of failure... --
--
That ONTAP bug #250630 can be really, really nasty, and it isn't triggered just by snapshot deletion as description would have it - LUN deletion triggers it just as well.
Getting new desks in the office (and thus having all PCs, monitors etc put in boxes) is a fantastic time to disconnect every user-side network and phone patch cable and re-cable. So much neater now, even without a proper cable management system. Plus, we expanded the user switch to 4x48 10/100 cards.
I learned that despite warning management that running DNS on desktop-class hardware is a bad idea, I still had to go in and fix it when the system failed a year later.
I also learned that if you request to virtualize even just the primary DNS server that you are pretty much laughed for for trying to virtualize such a crucial service. (Even though we have a proper VMware cluster already)
I also learned that if you request to virtualize even just the primary DNS server that you are pretty much laughed for for trying to virtualize such a crucial service. (Even though we have a proper VMware cluster already)
VSphere relies on DNS itself, which is justification to run your primary DNS server on tin instead of a VM. Having your DNS inside a VM kinda produces a chiken and egg situation if the cluster needs to be restarted or is somehow brought down. Even migrating teh DNS between VM hosts could produce some quirks.
That being said, any Windows server can provide DNS, so pick one that won't be virtualized and install away. We have one DC running on a 1U server (with redundancy up the wazoo) with all FSMO roles assigned to it. Our secondary DC is a VM. Makes VSphere a lot nicer to deal with.
When you need Microsoft's VLSC to be online (eOpen) so you can download 2008R2 to build a server, it will be offline for maintenance. And has been since Friday.
Meh, I say. I have a setup with both vCenter and DNS virtualized. For us, it has been a non-issue. During situations of major maintenance like that, when you have to bring everything down all at the same time, you simply have to point your vSphere Client at the ESX server where those VMs live to start them up.
He said 2003, and yes quite a few shops run Oracle on Windows, mine included. The decision was made with the previous staff which had no experience with anything outside Windows and they were unwilling to learn. We'll be evaluating multiple platforms for the replacement boxes next year, I'm leaning towards a Linux variant or possibly Solaris x64 if the 11gR2 port ever materializes.
Originally posted by ambit:
Eve ate my rant about the Exchange 2007 installer running perfmon invisibly in the background. --
Long story short.. somehow a brand new Server 2008 ENT build would crash out perfmon everytime you added counters to it.
It took about 3hrs with MS Support to figure out that is what was crashing the Exchange installer.
Then another hour trying hotfixes, regedits and resetting the perfmon counters to get the Exchange 2007 installer working again.
About 15mins later I had my 5th node back and running in my cluster.
Good news:
While I had this downtime I ran some offline defrags and cleaned up ~900Gb from my Exchange stores --
[edit]
Here's the hotfix we ran: http://support.microsoft.com/kb/961382 My Exchange nodes have about ~400 multipathed LUN's presented to them, which seems to be what caused this.
Hmm, um wow, 400 LUN's * x paths = big number. Does any vendor actually certify such a configuration? I know Xiotech limits you to 64 LUN's per host, haven't checked HP's matrix for that yet as we are currently only planning a few per host.
I knew the before, but I keep bumping into it! Academic environments tend to be focused not on the exact technical requirements but on the need to spend some grant money right away. So you end up buying things on short notice that it turns out later are not quite what you need.
I can't say I have seen a FC host having that many volumes before. 30-50 is a "big" host. I am not surprised arrays would allow that many - if you place a virtualizer in front of a large frame, you might want to have that many volumes to that "host" (read: the virtualizer)
Yeah... Oracle on Win2003. Good times.. good times. Thankfully I'm not the DBA for it, but we were here about 6hrs recovering the mess this vendor made.
My DBA came in after this was implimented and pulls his hair out everytime he has to work with it.
Exchange:
Our 5-node/9billion LUN cluster was setup (and certified, hhahah) by EMC. Its been running for about 6 months with ~20,000 users.
Here's what disk manager and SanSurfer look like on one:
-- View image here: http://i160.photobucket.com/albums/t185/vwambit/LUNS.jpg --
Our masking on the DMX4 is AWESOME. We have ~40 storage groups + ~40 log LUN's, presented to 5 possible owners, for over 400 possible combinations -- --
My DBA came in after this was implimented and pulls his hair out everytime he has to work with it.
Exchange:
Our 5-node/9billion LUN cluster was setup (and certified, hhahah) by EMC. Its been running for about 6 months with ~20,000 users.
Here's what disk manager and SanSurfer look like on one:
-- View image here: http://i160.photobucket.com/albums/t185/vwambit/LUNS.jpg --
Our masking on the DMX4 is AWESOME. We have ~40 storage groups + ~40 log LUN's, presented to 5 possible owners, for over 400 possible combinations --
--Originally posted by ambit:
Yeah... Oracle on Win2003. Good times.. good times. Thankfully I'm not the DBA for it, but we were here about 6hrs recovering the mess this vendor made.
My DBA came in after this was implimented and pulls his hair out everytime he has to work with it.
Exchange:
Our 5-node/9billion LUN cluster was setup (and certified, hhahah) by EMC. Its been running for about 6 months with ~20,000 users.
Here's what disk manager and SanSurfer look like on one:
-- View image here: http://i160.photobucket.com/albums/t185/vwambit/LUNS.jpg --
Our masking on the DMX4 is AWESOME. We have ~40 storage groups + ~40 log LUN's, presented to 5 possible owners, for over 400 possible
combinations --
Volume Mount Point hysteria!
For some reason Windows 7 doesn't like our wireless network and won't connect. Sure wish I could have known that before the boss needs the laptop in a couple hours.
Originally posted by ambit:
Yeah... Oracle on Win2003. Good times.. good times. Thankfully I'm not the DBA for it, but we were here about 6hrs recovering the mess this vendor made.
My DBA came in after this was implimented and pulls his hair out everytime he has to work with it.
Exchange:
Our 5-node/9billion LUN cluster was setup (and certified, hhahah) by EMC. Its been running for about 6 months with ~20,000 users.
Here's what disk manager and SanSurfer look like on one:
-- View image here: http://i160.photobucket.com/albums/t185/vwambit/LUNS.jpg --
Our masking on the DMX4 is AWESOME. We have ~40 storage groups + ~40 log LUN's, presented to 5 possible owners, for over 400 possible combinations --
And people wonder why I prefer virtualized arrays....
No, this isn't about real LUNs vs virtual.
Look at those sizes, 33.71GB time WAAAAAAAAY TOOOOOOO MMMMMMMANY!
This was just a mistake. And I'll bet you even paid extra for it. I'd set up a meeting with your EMC sale rep and the original project manager for the "professional services engagement". Either your end completely misstated the problem, or they mis-proposed a solution and you guys bought it.
Look at those sizes, 33.71GB time WAAAAAAAAY TOOOOOOO MMMMMMMANY!
This was just a mistake. And I'll bet you even paid extra for it. I'd set up a meeting with your EMC sale rep and the original project manager for the "professional services engagement". Either your end completely misstated the problem, or they mis-proposed a solution and you guys bought it.
Oh yeah, there are quite a few places running Oracle on Windows. The last job I worked at (healthcare IT) had a custom web-based case management app running 9i on Server 2000; and no other platform was supported by the development firm (and it was used by quite a few others in that subset of the healthcare industry). Never could wrap my head around that even as a Windows admin.
I've got a CC tomorrow for upgrade planning regarding a major logistics software provider's upgrade path, and I'm pretty sure it involves Oracle on Windows as the main supported path.
Some companies, even for open systems, have policies about volume size. This results in a 10k port FC environment having > 200k host to volume paths. I don't know how many individual HBA to volume paths that is - probably ~600k.
Today I learned that I was very right to go with Mozy for online backups for both home and my dad's small business. It's good to know the data is on some well fed EMC frame's instead of Promise controllers with SATA disks using RAID-5 Yeah, they say they've upgraded to RAID6, but they understand it so poorly they think they can lose 3 drives without data loss, wrong.
And it's back now. Somehow I think they had problems getting the new site live. /me downloads lots of software.
Today I learned that the VMware Server 2 console has no mouse-based way of sending ctrl-alt-del to guests. It's been completely excised. The only way to send ctrl-alt-del to guests on VMware Server 2 using the console is by pressing ctrl-alt-ins....WHICH YOU CAN'T DO FROM A MAC BECAUSE THERE'S NO GOD DAMNED INSERT KEY.
Screwed around with key mappings and stuff for an entire hour, with my jaw hanging down in shock the entire time as my blood pressure climbed and climbed. Ended up having to drive a half-hour into work, get into the data center, find a crash cart, plug it into the damn VMware server, log in, bring up the console, and THEN press ctrl-alt-ins to log onto the guest. Now I'm working the rest of the day from my office instead of from home and I'm mighty pissed off.
Screwed around with key mappings and stuff for an entire hour, with my jaw hanging down in shock the entire time as my blood pressure climbed and climbed. Ended up having to drive a half-hour into work, get into the data center, find a crash cart, plug it into the damn VMware server, log in, bring up the console, and THEN press ctrl-alt-ins to log onto the guest. Now I'm working the rest of the day from my office instead of from home and I'm mighty pissed off.
While I do have like nine keyboards at my house, they're all PS2 Model Ms -- -- The only USB keyboard I have is my Mac's. I would have just gone to my wife's PC and done the whole mess from there, but the remote access stuff won't work from her box because my employer is overly paranoid.
-- The only USB keyboard I have is my Mac's. I would have just gone to my wife's PC and done the whole mess from there, but the remote access stuff won't work from her box because my employer is overly paranoid.It's been awhile, but isn't there a ctrl-alt-del option under the troubleshooting menu in the console?
Pretty sure there is a "send ctrl alt del" option under one of the menus--at least there was in the class I was in last week.
We're just using the classic linecards currently WS-X4548-GB-RJ45 is the model number.
You can also find that in the .edu sector. We had Oracle 8i on Windows NT4/2000? and then Oracle 9i on Windows 2000 and now Oracle 10i on Windows 2003 (VM). That may sound bad, until you realize that those installations replaced various DOS based versions of FoxPro and dBase III.
Luckily, the last two FoxPro apps are destined to be on Oracle within the next 6 months.
Now, if I could just get the jackass who wrote on APL+Win v1 to program in a newer version or anything else, I could move away from the DOS/Win 3.1 style installation of copying files manually and then editing ini files and hoping for the best.
That along with finishing the decommissioning of our Novell 5.5 server will make me a very happy sys admin.
--
I learned or maybe accepted (not today exactly, but I've been mulling it over for the last week or so), that while I have been doing an "Enterprise" Sys Admin's work quite well and even solving critical issues on platforms I've never touched before, I feel like an IT fraud.
I don't have many certs (the few I have are laughable ones), nor do I feel entirely comfortable like I could build an enterprise class network with my own two hands without stumbling along at points.
I did some soul searching and the thing that had been bothering me for a while now, is that I don't feel like I've mastered anything, even while being good or even very good at a number of things (e.g. "Hey, our VMWare cluster is down, fix it" as an entre into the world of modern VMWare or "Hey, here's $10 million CNC machine, make it work with this cabinet making software that's 20 years old and the new one that is 1 year old and we've got no docs on the CNC or the old software" or "Hey, why is our external email down, rebuild the email av gateway and save our email!", etc). The lack of formal IT training irks me. I've got a bachelor's degree (bus admin/CIS) and I'm in a master's program in CIS, but I feel IT inadequate at times.
Everything has been a situation where I inherit a disaster and am forced to learn everything while recovering from a disaster that a coworker (primary admin on whatever) may or may not have caused. No documentation of changes. No documentation of the original state. No documentation of critical tasks or maintenance tasks.
So, I'm going to be spending the coming year doing as many certs as I can, with the training for the exams doubling as professional/career training. When I can, I'll be layering it in with my graduate degree work.
The Visible Ops books have been comforting as has the Systems Administration book that everyone has been bandying about.
I would just like to say thank you to everyone in this and the other windows tech fora. I've learned a lot by reading and inferring from context and anecdotes.
On the other hand, you have a skill that no cert can teach - the ability to function under high pressure and with no knowledge and come out on top.
To some extent, I feel like you some days - college drop-out, one real cert that expired a few years ago and some vendor-specific certs that a baby could pass, no real formal training. On the other hand, when my bosses need something done, they come to me because I can get it done. Plus, they pay me well. If that makes me a fraud, at least I am a well paid fraud --
--Oh, it's there if you're using the Windows version of the client. It's there on my VMware server at home. Just checked. Unfortunately, when you have no Windows boxes that you can access anymore because of security restrictions, and all you can get to are Linux boxes, this shit right here is all you get:
The option is inexplicably missing, apparently because VMware thinks anyone using a Linux box has better things to do than log onto Windows boxes, and so streamlined the menu item right the hell out of the client. At this point, I have several dozen gray hairs all named "VMware", and this just adds to the pile.
Originally posted by erratick:
Same. PacBell and ATT did this to me at different times. ISDN most be fully misunderstood everywhere now. --
Yeah man, i've been so over it for a couple of years now, i got all of north america on IP (hey boss, with a small capex on some new hardware that we need anyways, we can get decent connectivity over a $50 a month dsl line for all the branches and a $79.99 sdsl line at HQ for multipoint (we have two units here and can squeeze 3 connections on each of the SDSL's and then connect them), and we don't need to spend $250+ monthly for the lines along with usage charges at each branch, and ip looks better anyways), working on south america now, europe got the idea and they are starting to switch, japan switched some sites, but asia(read:China)/pacific/oceana in general... they're... resistant. Is the cost of a new unit really gonna set the budget in jakarta back THAT much? --
--learned today: The VAR that my homey works for get's much better pricing on dell servers than i get from my dell premier page. Guess he's got a new client --
--dude, i love belgians, but i gotta say, sunset grill (or whatever that place is in alston (i think it's alston...)) has an amazing selection of everything as long as you don't mind wading through a few fratboys and stealing their dates with your far wittier conversation and raucusry (if that's a word). plus their nacho platter could feed an army of midgets (yeah, it's all class, but it has like 180 taps and 800 bottles or something close to that ridiculous amount of beer in one spot).
finni and spatula, sounds like we might need to go drink gods gift to man in excessive amounts when i'm up there for xmas in a couple of weeks. hit me up with a PM if you want to make some arrangements. The drunken nerdus factor could be off the charts.
for the record, i got 2 (count 'em, in a row, 2!!!?!) sickdays. a-freakin-mazing. and i was only VPN'd in for like 2 hours on those days.
- Status