What are you replacing vSphere with? Or: Broadcom gets absurd on pricing.
- Thread starter Arkannis
- Start date
I agree... all the more so as back in 2019 I was able to get a big response from them when I reached out to the point that I was even invited to a local demo they ran for a number of potential SMBs to show their offering since it was so new. I liked it, but ultimately it didn't work for us because at the time they required we use their hardware and we had just purchased new hardware.
Try a different VAR? We aren't that small but we aren't all that large either. Nutanix engaged with me last year before they even knew what size environment I was thinking about.
I did ultimately get a response. I'd like to think this is mostly sales growing pains as they are likely busy. I met with one of their engineers and even gave them their sizing calculator files they needed... well it wasn't enough for them yet and so I have yet another call setup for Wednesday with them to hopefully get a price together at that point.
One thing that does look like it might be deal-breaker for us is what it might cost to enable any DR environment support... Hopefully what I heard was wrong, but the minimum cluster size is 3 hardware nodes and this holds firm for DR as well, so if you want a production environment as well as something you can turn on in the event of a major DR event, you need to have another 3 physical nodes even if you don't require that same level of performance on the DR site... unlike Hyper-V or vCenter, where you can have your 2 or 3 node production cluster and then have a single host at a secondary site to run DR if needed. Sure, they support you putting that into dedicated cloud hardware, but that is likely $$$$ unless I find I can spin that up quickly during a DR event and not have to maintain that in standby all the time.
One thing that does look like it might be deal-breaker for us is what it might cost to enable any DR environment support... Hopefully what I heard was wrong, but the minimum cluster size is 3 hardware nodes and this holds firm for DR as well, so if you want a production environment as well as something you can turn on in the event of a major DR event, you need to have another 3 physical nodes even if you don't require that same level of performance on the DR site... unlike Hyper-V or vCenter, where you can have your 2 or 3 node production cluster and then have a single host at a secondary site to run DR if needed. Sure, they support you putting that into dedicated cloud hardware, but that is likely $$$$ unless I find I can spin that up quickly during a DR event and not have to maintain that in standby all the time.
well, a 2 node cluster is asking for a split-brain incident and all vms are lost as the disk image gets corrupted. Is a low end desktop/server an option for a third node to be a witness node and prevent split-brain issues?
It's not a good place to be in but you're tripping over dollars to pick up dimes here. If you need a DR site, it's part of the cost of doing business.
the ability for a single node to function would be nice, but I get that this is of course complicated by the whole shared nothing and redundancy required thing... It would have been nice if there was a "no redundancy" option if you wanted it.
while not ideal, two-node clusters are certainly possible and there are ways to address the split-brain situation as evident by the fact that both vCenter and Hyper-V both support 2-node clusters and they also support single nodes operating individually. The fact is, our live production environment will run perfectly well on a single "normal" sized host... we currently run two of those hosts to achieve the capacity redundancy needed to maintain full availability in vSphere. Going from a total of three physical hosts to six physical hosts isn't exactly "dimes" when it comes to both hardware and licensing costs... we are "small", but I'm figuring that will come in at over 50-70k for hardware and another maybe 40-50k in licenses plus another maybe 10k/year for support? That isn't much for big enterprises, but it is at the SMB level.... At that point, Broadcom pricing IS cheaper, but at least with Hyper-V I can stick to three nodes at 1/2 the cost of 6 nodes and with 1/2 the licensing.
while not ideal, two-node clusters are certainly possible and there are ways to address the split-brain situation as evident by the fact that both vCenter and Hyper-V both support 2-node clusters and they also support single nodes operating individually. The fact is, our live production environment will run perfectly well on a single "normal" sized host... we currently run two of those hosts to achieve the capacity redundancy needed to maintain full availability in vSphere. Going from a total of three physical hosts to six physical hosts isn't exactly "dimes" when it comes to both hardware and licensing costs... we are "small", but I'm figuring that will come in at over 50-70k for hardware and another maybe 40-50k in licenses plus another maybe 10k/year for support? That isn't much for big enterprises, but it is at the SMB level.... At that point, Broadcom pricing IS cheaper, but at least with Hyper-V I can stick to three nodes at 1/2 the cost of 6 nodes and with 1/2 the licensing.
last time I tried pricing it, doing VMs, that came out to be over 4k/month for VMs or ~2.5k/month for dedicated hardware, but the break-even on the cheapest way there still was about 15 months when compared to local hardware. There is some VM sprawl I can probably clean up and maybe some other services I can migrate from VMs to SAAS functions, but I've yet to see any that aren't way more expensive when we are so static and stable and located at just a single location.
I'm holding out hope that Nutanix does have the equivalent of a ROBO single host solution that can do what we need for the secondary site. The tech I was on with mentioned that one was available, but thought it "wouldn't work" for whatever reason... guess maybe it requires the primary site to be online (which sounds like a terrible design). If that doesn't work out, then I'm 99% sure our direction will be to go to Hyper-V with our existing 2022 Server Datacenter licenses and just hope that M$ doesn't really kill off Hyper-V in its near future server releases.
We're in a similar position -- too many VMs for IaaS to be affordable, but also too many esoteric systems to be able to push most of it to SaaS (not that SaaS is a whole lot cheaper in the long run). I'm looking at an ESXi hardware refresh next year, and with the switch to per-core licensing in VMware I'll probably have to make some serious compromises when it comes to how much resource headroom I want to keep available.
Is this just an issue of what will work vs. what they support? You can run proxmox on a single machine. Have it part of the DR plan to add additional physical nodes after X days of running in DR so that you can be supported and you will need the redundancy as its apparent you will be running prod out of the DR site for an extended period of time.
I'm hoping it was just a green sales engineering confusing recommended with required, but I guess I'll find out tomorrow. I get that a single machine is obviously a SPOF and that carries its associated risks which we can weigh the cost/benefits if technically able. I'm just assuming the all-in cost of two or three additional nodes is likely "cost prohibitive" for us.
That was always the downside of HCI for us. Everything had to grow in chunks (I think they have storage only now, maybe?) and you ended up buying a lot of what you didn't need to expand another portion.
That's also what's kept us from HCI as well. Normally I wouldn't mind Nutanix or something along those lines for our hosted VDI product, but our storage growth on that tends to outpace other resource usage.
Our problem was the other way. Our LOB apps need a stupid amount of RAM/CPU but not a lot of storage. When we looked you couldn't really just buy compute.
HCI isn't for us.
We're also not idiots and have no issues running a bunch of Arista + Pure + Dell.
I can see the allure for small shops or if big words like "Converged networking" scare your IT people.
HCI isn't for us.
We're also not idiots and have no issues running a bunch of Arista + Pure + Dell.
I can see the allure for small shops or if big words like "Converged networking" scare your IT people.
well... got pricing and waiting on clarification. I don't know what the vSAN licensing cost is for Broadcom and I'll accept that maybe my vCenter license isn't exactly feature parity with the Nutanix starter bundle, but with discounts, their per-core license cost to us was literally 669% higher than Broadcom's 
Hardware was reasonable, but I didn't get a definitive answer yet on the DR side other than that it likely is a 2-node or 3-node cluster and they had to check if 2-node was even possible.... meanwhile, for 3-years with hardware it looks like the literal smallest option is going to be over $250k. Suddenly my two site 3 server vCenter setup with a 2-node cluster looks like a bargain 
Hardware was reasonable, but I didn't get a definitive answer yet on the DR side other than that it likely is a 2-node or 3-node cluster and they had to check if 2-node was even possible.... meanwhile, for 3-years with hardware it looks like the literal smallest option is going to be over $250k. Suddenly my two site 3 server vCenter setup with a 2-node cluster looks like a bargain We looked into them 10 years ago and that was the same story. It was excessively expensive for less features, functionality and flexibility.
They're almost there on features with the release due out this month. Almost. To be fair, they have a few things that VMware don't have, as well. But obviously I'm not using them right now so it's hard to calculate the value. We haven't had Broadcom's quote yet, but the promise from the sales rep is to beat it by at least redacted amount. The important part to us will be if Dell will let us convert vxrail to nxrail, either by a firmware change or trade in. I can't get the funding for a new hardware stack, we have to get some value out of the vxrail hardware, even if it's just a generous trade in.
Another reason to have a bunch of generic compute and storage, I can put anything on it I want!
There are some esoteric VMWare things like FT that nobody else has and if you use those features you're well and stuck.
There are some esoteric VMWare things like FT that nobody else has and if you use those features you're well and stuck.
I was suitably amazed when some jackass accidentally reset the UPS in the primary datacentre and everything came up at the second site in under 5 minutes (that's a good 250 VMs). But I also understand that 5 minutes is too long for some people. Also just noting that we have a really big warning sign on the UPS now that no-one is allowed to fucking touch it without someone from the Ops team present.
You can convert a vxrail to a generic power edge with a BIOS recovery. It's support afterwards that gets tricky.
VMWare FT is really really niche, and comes with some pretty serious caveats in addition to things like latency limitations. If you're using FT and your primary VM crashes due to anything except a hardware failure your FT VM will likely also crash for the exact same reason, due to the 'exact replica' nature of FT.
Unless there are some (existing but very boutique) concerns where you absolutely have to have 'exact replica' FT having multiple hot sites will always be better than FT.
Yeah, perhaps they were thinking of another technology. FT is very rare, and not for 250 VMs.
We use Nutanix for our remote sites that need a handful of VM's locally with redundancy. 3 years ago they said we could do 2 node clusters, but we went with 3 so that everything can stay up while doing cluster upgrades/etc. We ended up adding a bunch of clusters last year due to some downtime procedures changes around the system. Our hosts are pretty small, as the VM's have low requirements, but each 3 node cluster cost us just over 50k USD each last year. That was hardware+software prepaid for 3yr. Single socket 16 core with 256GB of RAM and 4x3.84TB SSD each. We do use per-vm licensing instead of unlimited on those environments though, so I suspect that's a large cost difference.well... got pricing and waiting on clarification. I don't know what the vSAN licensing cost is for Broadcom and I'll accept that maybe my vCenter license isn't exactly feature parity with the Nutanix starter bundle, but with discounts, their per-core license cost to us was literally 669% higher than Broadcom's
they were pushing 32 core single sockets with 512GB ram and 2x 15TB nvme drives per node. Hardware price was pretty reasonable IMO, but excluding the hardware and hardware support, the subscription for NCI starter bundle software was over 50k for three years for three nodes making it over 100k just for the subscription. 33+k/year for the software is a far cry more than our vCenter Essentials Plus software we came from. I've been busy with other items, but looks like I'm going to pull the trigger on moving back to hyper-v 
The VMWare ROBO per VM licensing used to be an amazing deal, we almost went that way for our branch servers a few years ago.
The software breakdown for us ended up at about 15k for the per-vm licensing for 3 years. That's for 6 VM's, so per-vm for me is ~$833/yr. That is pretty hilariously expensive, but it gets the job done.
Your 50k/96cores/3yr is $173.61/core/year, and that's the starter edition? That's absolutely ridiculous.
Your 50k/96cores/3yr is $173.61/core/year, and that's the starter edition? That's absolutely ridiculous.
yea... I wasn't expecting to be required to ultimately need 192 cores costing a total of ~33k/year in order to have a system with a secondary site doing pretty much what I've been able to do with 96 cores at what now would be roughly 3.5k (which used to be less than 1k)... of course that doesn't include our existing netapp infrastructure costs, but I can safely say our C190 + A150 were still much cheaper than the price difference here.
We almost moved to VMWare's DRaaS a few years ago. It really checked all the boxes.
You installed sync VMs that pushed everything to their cloud (inside AWS), you only pay for storage while "off". When an event happens it spins up ESX-in-AWS, starts up a vCenter, and boots all your VMs. Minimum of 3 hosts, the rate was roughly $1000/day and you got a handful of test hours. The most important part was when you were ready to fail-back to on prem the sync occurred in reverse and it would then gracefully power down the cloud VM, do a final delta, and power the VM back up on prem.
With our three main LOB apps moving to vendor SaaS next year we'll probably end up forklifting everything else to AWS/Azure and use druva for backups/DR.
You installed sync VMs that pushed everything to their cloud (inside AWS), you only pay for storage while "off". When an event happens it spins up ESX-in-AWS, starts up a vCenter, and boots all your VMs. Minimum of 3 hosts, the rate was roughly $1000/day and you got a handful of test hours. The most important part was when you were ready to fail-back to on prem the sync occurred in reverse and it would then gracefully power down the cloud VM, do a final delta, and power the VM back up on prem.
With our three main LOB apps moving to vendor SaaS next year we'll probably end up forklifting everything else to AWS/Azure and use druva for backups/DR.
and... I caved. While the price increase is really frustrating, the incremental costs from going to anything else is way more... and I chickened out on Hyper-V... we had issues with it in the past that I chalked up to attempting to use sub-par shared storage (which we now have excellent options with NetApp), but getting into the nitty gritty there, it just wasn't worth trying to reshape things here... got a 3-year support contract so kicked that can down the road and will address again in 2027.
I'm seeing a lot of that, you're not alone. For many of my customers, the cost and aggravation of leaving just barely outweighs that of staying ... for now. I predict a lot of the "almost there" solutions will catch up a lot in the next few years, and people in your position will be better positioned to make the change.
I'm kinda shocked and kinda not. On one hand, MS and many of the hardware vendors are flogging the crap out of Azure Stack HCI right now, which is fine for customers who want new hardware, but it's not a one size fits all option. On the other, Hyper-V is absolutely still an option, and it's silly that you don't have a VAR willing to have that conversation with you.
Veeam is rolling out test/support for XCP-NG and Proxmox, if I remember right. That is a pretty big green flag that those options are actually... options, or they believe they will be soon. Not sure I agree right now but XCP-NG at least is very much acting like they are hungry to eat Broadcom's leftovers.
Veeam support for Proxmox is now official, and they did it the smart way - instead of reinventing the wheel, they simply integrated support for the existing Proxmox app into Veeam. I haven't had a chance to play with it yet, so I don't know how feature-complete it is compared to Veeam backing up VMware, but it could be a game-changer for a lot of my customers.
I have zero love for Broadcom and now that I am ditching the IT room at my dayjob I am moving things to Azure. Both AVMs and AVDs. Not that I want to really deal with Micrososft, but it becomes the "devil you know" aspect for the users and general management of things.