I've acquired a new lab machine, for virtualisation and deployment development. The goal is quickly spinning up VMs based on snapshots, do a deploy and end-to-end test of cluster software, and spin it all down again. I'm doing that regularly on remote machines, but wanted that capability locally, for ease of use, speed, and lack of resource competition. In addition I want to have some VMs running distro repository cache, run Jenkins, do development etc. My existing server hardware is old and not very scalable, so it was time to invest in better capability.

The requirements were:

  • support virtualisation (VT-X/VT-d)
  • scale to lots of VMs, so:
    • CPU: many cores. No need to be super fast. Keep electricity consumption low and heat down. Dual CPU to get the core count up while keeping CPU costs in check.
    • RAM: a decent amount to start, with future expansion potential. ECC. 64G to start with.
  • fast IO for installs, but no huge requirements for large datasets. Dual SSD (one for the system, one for VMs), with one fixed disk for storing distributions. No need for RAID.
  • pedestal case (no rackmount), and reasonably quiet, because it gets deployed in a test lab / office.
  • remote management
  • get a system that will last and be useful for many years, but not spend silly money.

which gets you into professional server territory. I've been self-building workstations for years, but this time I resisted temptation and decided to save myself the time and risk. I opted for the following specification from Sentral, where I had sourced a 1U Tyan server some years back:

SuperMicro case X9DRi-F motherboard

  • Case: SuperMicro 4U SM Tower c/w 665W Super Quiet (25dB) PSU (CSE 733TQ-665B)
  • Motherboard: SuperMicro X9DRi-F. Dual Socket R (LGA 2011). IPMI 2.0 and KVM, 16 DIMM sockets.
  • CPU(s): Dual Intel E5 2630L (6/12*2Ghz/15Mb cache) 60W. Note that this is the low-power variant.
  • Memory: 64GB DDR3 1600 ECC REG (4*16GB)
  • HDD: 1* 3TB SATA Seagate Barracuda (ST3000DM001-1CH166)
  • SSD: 2* 240GB Intel 520 Series SSD (SSDSC2CW240A3)
  • CDRW: LITE-ON DVD Burner (iHAS124-04)

Cost a bit over £2500 ex VAT delivered, which I think is good value, if quite an investment. The build and delivery took two weeks; a bit longer than normal because of my non-standard CPU choice, and testing that much RAM. Sentral was pleasant to deal with: no on-line configuring ordering of credit card payment, but a few quick emails back and forth, a BACS transfer, plus regular pro-active updates worked as well. Plus, it's actually nice to have the personal interaction (hi Dave).

SuperMicro unboxed SuperMicro inside SuperMicro motherboard SuperMicro frontpanel SuperMicro trays SuperMicro back

Initial impressions:

  • Nice looking case. The side door is a bit fiddly to close, seems to work best lying on its side.
  • Quality of components is good. Sentral did a nice job of cable management.
  • It's much quieter than a 1U rackmount, but louder than my self-builds where I optimise for quiet. In particular the front case fan is noisy; the vents in front of the fan and the shroud behind probably don't help. I may decide to move the machine, or look into replacing the front fan, but for now it's not too bad.
  • I connected the management interface and first normal interface to the LAN, found the new MAC address on my router, connected to IPMI, and configured it from there.
  • It feels fast.
  • Ubuntu 12.10, KVM and VirtualBox run fine.
  • It's sipping around 120 Watt at idle.

Obligatory screendump, 2 CPUs x 6 cores x 2 HyperThreads = 24 processors in htop:

htop output

It will take some time to do some experimentation and complete the full setup of the lab environment. I'm looking forward to it already.