I write this post only because it falls under the category of "What the Fuck? Life should not be like this."
I have a beefy VMware server at home. Read below for the fun I've had there. This box is a dual proc 2.83 Quad Core 12MB L2, 8GB of RAM, 16 x 750GB SATA II drives, and an Adaptec 16 port card. 8GB was the limit with VMware's Starter Edition, but that limit was raised to 128GB with the recent 3.5 release (again for a good time.... read below).
I had ordered another 8GB of RAM from Pearson/Computerlink.net (they rock) in anticipation of the 3.5 release so I could go to 16. With VMware, #1 is IO - 16 (actually 12, long story) drives was as best as I could afford do with my raid card. #2 is RAM even though VMware is particularly good in this arena, it's best to stay off disk, no matter how fast, as often as possible. The last being network and that is always disk bound somewhere, coming or going.
I went up into my loft. Let me explain what my "Loft" is. It is a bit of poorly built, far from code, plywood ontop of the garage rafter that gives some additional storage. It's supposed to hold suitcases. The absolute highest point of the "Loft" is 5 feet and effectively, it's about 3, maybe 4. In my Loft I have 2 half racks with patch panels and servers mounted. Square hole, yes it's better than round, you are a fool to use round hole. ;-) That said, getting this server up into the "Loft" really sucked and then potentially having to get this server down from the "Loft" was something that I was dreading.
So I throw in the additional 8GB of ram, and turned it on. Red lights everywhere, alarm sounds everywhere - the machine was pissed.
I move the additional 8GB of ram - same red light and siren story.
I say "Fuck it" and pull the 8GB to try and go back to 8GB and same crap, sirens and red lights.
I turn off the machine and I'm just bummed. My Fucking VMware Server - toast. Data probably OK, but big pain in the ass to get it working.
I work on some other things, even some wine (blech, zin + screwtop = poop).
I go into my office and I just stop and think abount nothing but this problem. It's not as dumb as bad ram, the original ram was good. Something else is up.
I go back into the loft and I realize I should re-seat the new ram, but I'm going to pull the power supply so everything is really turned off (ethernet lights were on still). Then it hits me. I only have one power supply hooked up. Normally this is ok but 16GB made it say "I want 2"
I move the memory to where it should be for DDR II and plug in both pwer supplies and wallah, I'm golden..
Big Mystery - why didn't going back to 8GB make things right with a single power supply? I dont know, I can only guess a BIOS setting got changed.
So, file that under, strange shit happens and remember it for your next upgrade that goes badly.
When I first arrived at Media Machines, they were renting cubes from a company called Coghead in Redwood City. They had a wireless access point and that was connected into Coghead's network. All basic, except for one important aspect - Network performance was so bad it was quite literally useless. Coghead had a couple admins but they didn't know what to do and it was clear they weren't going to figure it out anytime soon. So since Media Machines was about 8-10 people, I figured I would give my Junxion Box a shot.
The Junxion Box had a EVDO Rev A card from Verizon. Performance was great and people were really happy to have something that works. Remember this is 8-10 people sharing at most a couple mbits and now being happy.
Media Machines grew and moved. We couldn't get any type of Internet connection except wireless which is supposed to be turned up tomorrow. As we grew I was asked to get 2 more Junxion Boxes so people could associate to a specific one and get some connectivity as the company was maxing out a single EVDO card.
I was working with Les, Ops Mgr, on a thought around "Why can't put all these together somehow to have one network that utilizes as man Junxion Boxes that we have?"
So with that in mind we did exactly that. We dusted off the original Cisco wireless access point and made it a DHCP server and router. We set the IP addresses of the Junxion Boxes and created a separate network for clients to use internally. Then we made sure Cisco Express Forwarding and destination-based load balancing were enabled on the Cisco AP and simply set up 3 default routes to each of the Junxion Boxes.
It's not been 'great', but we have 25 people operating off this network and have been for weeks. I still think it's pretty cool to have that many people off of what to most clients is a regular Internet connection.
We did have to switch from Verizon to Sprint however. Verizon's "Unlimited" data plan is not unlimited at all. They used to disconnect account but got into legal trouble, had to cough up a million bucks to those disconnected, etc. They now define Unlimited as 5GB per month. After 5GB they will not stop your card from working nor disconnect you, they will slow it down to the point of it being useless. Go Verizon - @$%^@ Anyway, turns out that Sprint is truly Unlimited, so word to the wise. Just a point of reference, as an individual, I have never come close to 5GB. When I had the Junxion Box setup the very first day, the 8-10 people did 1GB in 4 hours. And to add insult to injury, we have a corporate account with Verizon and they permit even less data transfer and charge more (tried to anyway) than consumer cards.
There's a whole other story about Verizon allowing anything that passes a test onto their network, which I think is very cool, but I would like to see Sprint put the hurt on Verizon to change some of it's policies. If you can download 5GB in a day and have 4 hours left, seems like 5GB sucks.
Just a story for startups in a pinch or even for bigger companies who need a "In case of emergency break glass" setup. Our Junxion Boxes aren't going anywhere, just our default routes.
Out of order update: VMware worked it out today. I think the secret was to ask a manager to actually call and talk to me. I was always getting "we have forwarded your ticket to the responsible department". Today when I asked for a phone call it all happened and came together. So, I wanted to put this update at the top of the entry so people know it did end well. How I feel about VMware and their ability to support their customers (it took no less than 15 days to get this done) is still not good but I am happy that I can move on. Now to get my VMs working and create more, and add another 8GB of RAM to the system.
VMware recently upgraded their ESX from 3.0 to 3.5. 3.5 has great features like no limit (well 128GB) on RAM (was 8GB), no limit of processors, and now local, nas, or iscsi storage. Then even reduced the price by 200 bucks.
I bought 3.0, 5 days before 3.5 shipped (no pre-announce, completely surprised by the release). I logged in to my account on VMware, and the 3.5 software is there ready to download. No extra documentation, just a simple ISO to download, burn, boot, and upgrade, which I did.
Turns out I need a new licence file as I did not gain any of the new feature and I actually lost virtual SMP support (i.e. allocating 2 or 4 core to a VM). This broke a VM of my laptop which was a dual core.
I call them asking for a new license file and it has been a nightmare. It's 14 days later and they still can't product a 3.5 license file to which I'm allowed to have but no one can generate
The best quote so far:
Amit Das: I understand your dissapointment, Jon. We are from the licensing team. Unless the license is generated, we would not be able to proceed further.
That would be the licensing team saying they can't generate the license. Sounds like I'm pretty screwed.
Thanks VMware - you are worth every penny of your market cap - not.
--------------------------------------------------------------------------------------------------------------------------------------------
An update:
I have been pushing this issue really hard with VMware to the point that I actually got a phone call today. I can put my server into eval mode and it'll work for another month and a half with the features turned on. That's not the point though. The gentleman who called was gracious, apologetic, and was obviously frustrated too.
The entire point of this is; do I as a professional in the industry want to work with VMware? I have a clear need for machine consolidation at home. I have also done a test where it was clear that for a starter/foundation VI3 license, the product can pay for itself in a development environment. These examples are why I spent my own money on a server and their software.
What’s clear is that if I need help with something, I wont get it and the company has utterly failed to be effective. Something went terribly wrong as they grew and the executive team should know it.
Another example of dysfunction: I used my LinkedIn contacts to get the VP of Support. I wrote her a long letter explaining the situation and asked, actually begged, for help. I got no response but what she did do was make it so she is not findable on LinkedIn anymore. Very classy.
What I do like is the fact VMware has found and read this blog. Go VOX!
What I do hope is that executive staff read it. As it stands now, a 20 year Ops veteran would not purchase nor recommend VMware products at all. VMware; show me that your company can actually move and be effective when necessary potentially we can talk. VMware; you do have good technology but you are a company that is so dysfunctional that it simply cannot be a viable option for any business due to the risk exposure.
