If you work on the IBM i platform, the Midrange Mailing Lists may also be a place to visit and subscribe. Also, check out the IMHO Midrange Blog.

The x3650 M2 comes with a new IMM (Integrated Management Module) that replaces both the BMC and the RSA II Slimline. In order to get remote KVM capability, a physical licensing key must be installed into the server. Standard features like remote power on/off are available without the licensing key – which is more expensive than the physical RSA II slimline adapter. With the IMM also comes UEFI, as a replacement to the aging BIOS.

The x3650 M2 also gets rid of the Adaptec ServeRAID 8k controllers, and introduces us to the ServeRAID 10 series manufactured by LSI. The ServeRAID 8k series have been plagued by several extremely heavy bugs that never caused me loss of data (but several other people), but nevertheless cost me a lot of my nerves.

The two standard RAID controllers in the x3650 M2 are the ServeRAID BR10i, which is the baseline controller without BBWC and without support for RAID5. The ServeRAID MR10i is the better model, which comes with 256MB BBWC and support for RAID5/RAID6. Unlike the ServeRAID 8k/8k-l story, these are entirely standalone controllers, that are located in a special daughterboard position with a standard PCI-e x8 interface.

Both controllers support only 8 drives – in order to get 12 drives, you need a special enabler kit that comes with a SAS expander hard and several of other stuff that doesn’t look all that trivial. I’ve used such a kit, and so i can’t comment on how it works exactly.

The power supplies have gotten a lot smaller, the server seems to look a lot more organized, the 2.5″ SAS HDDs are no longer as finicky as they were in the x3650 and now fit very well into their slots, the Lightpath diagnostics panel now looks like it belongs to an expensive server and locks into place securely.

Of course, all the usual changes that come with the new Nehalem based Xeons, triple channel DDR-3 memory, both processors needed for using more than 8 slots, using to many memory modules will downgrade the speed, etc.

So much for the general rundown – now for my assorted observations:

• The ServeRAID BR10i seems to be a slightly newer variant than the SAS RAID Controllers found in the x3250 and x3250 M2. The configuration interface is simple, but it works reasonably well.
• The ServeRAID MR10i is a controller i haven’t dealt with before. It does not offer a standard character based interface for configuration, only a graphical interface called WebBIOS. It’s completely awful and half done – half of the buttons have no text on them, every button press requires several seconds until something happens. Configuring a controller with this interface requires you to guess actions based on the manual, since they aren’t labeled onscreen.
• As usual, both LSI based controllers use the MegaRAID management software. Compared to the old Adaptec software, it’s really awful. It runs extremely slow – even on these new servers, is much more complicated than the old ServeRAID software and offers less options in terms of notification.
• The IMM webinterface has gotten even slower than the already slow RSA II interface. Web2.0 style “loading” icons have been added, but viewing the status screen can take up to a minute now. This is retarded, and clearly a step back. At least IMM standard now comes with every server.
• The IMM’s KVM capability have gotten a lot better. Instead of a java applet running in the browser, a java application is launched using java webstart. While IMM itself is slow, the remote KVM capability is actually very fast, and even works with decent speed through a VPN connection.
• While the ServeRAID 8k with newer firmware usually spent 2-3 minutes looking for the drives, the new LSI based ServeRAID 10 series now only takes a few seconds. This is compensated by UEFI which now takes roughly 2 minutes instead of the 10 seconds the old BIOS took. With this, IBM is successfully keeping the server at roughly 4 minutes until OS boot
• The IMM connects to the OS using an USB LAN interface. This is a real problem on Windows, since it confuses the Windows Firewall (switching it into “Public network” mode) and the Windows DNS client. Install the driver and disable it in Windows. Ensure you never enable it on DCs! Run the IMM firmware updates from CD. Clearly a step back.
• The UEFI configuration screens act a lot slower than the old BIOSes. But the options available are decent.
• You can install Windows Server 2008 Standard in UEFI mode. I did that on our internal x3650 M2 which is going to run our WebFOCUS deployment
• You cannot install SBS 2008 in UEFI mode. It will work fine with legacy BIOS emulation, though.
• The IMM can’t send alert messages to email addresses with a – (Dash) in them. Retarded.
• The machine is extremely silent. Compared to the Power 520, which will kill your ears within minutes, they’re a blessing.
• Just like the RSA II interface, the IMM web interface has a tendency to lock up randomly and stop working. Requires a physical power cycle on the server to fix.

That’s it for now. Lot’s of negative stuff in here, but the machines are actually extremely good performers. I hope that IBM will fix those outstanding bugs soon.

Central point of the new IBM offering is the ToolsCenter, which serves as a starting point for all important IBM tools.

The two most important tools, which every admin dealing with IBM System x servers should know are now available in new versions, which offer improved functionality.

UpdateXpress System Pack Installer

UpdateXpress is now available in version 3. Pictured to the right is the new user interface, which offers much needed improvements. The previous versions looked like a leftover from the cold war.

UpdateXpress allows you to update all your System x drives in one automatic swoop, without the need to meticulously check the IBM web site for newly released drivers.

Dynamic System Analysis

DSA is now available in version 2.20. While the handling of the tool hasn’t changed much, there is now a 64bit version available. A few bugs i’ve encountered on 64bit systems are fixed with this new release.

If you’ve never heard about the x3650 M2, i suggest you to read my introduction post first.

Planned availability, other new products

Planned general availability (GA) of the new x3650 M2 is the the 20th of April, or in pretty much 3 weeks. Orders, configuration and pricing is available – so if you want to buy a new server now and can wait for three weeks, you should order a x3650 M2.

At the end of April, the M2 versions of the x3400 and x3500 will be announced. Judging from the current timeline, this will put GA of those new products near May.

4 CPU machines are planned for Q1 2010. I don’t care much about those since we run all our heavy DB workloads on IBM POWER.

Positioning of the new Intel 5500 Xeons

Just like the Core Microarchitecture brought many changes to servers, the new 5500 Xeons bring even more changes. It’s especially important for system administrators to understand the differences, and even more important if you’re selling systems to customers.

Here are a few key differences:

• Memory speed depends on the CPU purchased and the amoung and type of memory installed into the server
• Memory slots are only usuable if the associated CPU is installed
• HyperThreading is reintroduced in 2/3rds of the CPUs – systems will show twice the amount of logical processors
• TurboBoost is a new functionality that allows the CPU to run at higher clock speeds, depending on load and cooling

To the right you can see Intel’s official spec sheet. Intel introduces a “garbage bin” of CPUs that you should never use – the E5502 / E5504 / E5506 models. These CPUs do not support HyperThreading, TurboBoost, 1066 Memory and only have 4 instead of 8 megabytes of cache. Make sure to use E5520 or faster CPUs to ensure best performance. The performance difference between an E5506 and an E5520 is 15-20%, while the price difference is much smaller! In my opinion, the E5520 is currently the sweet spot between price and performance.

Order the right memory configuration

With FB-DIMMs, memory configuration was simple, because FB-DIMMs were slow no matter which way you put it. However, with the new integrated memory controllers, memory of much higher speed is now available. Now, as a technician or sales things will get more complicated.

I wrote in an earlier post that i didn’t understand why IBM only put 16 DIMM slots into the machine, while HP installed 18 DIMM slots – the reason is that in most cases it makes little sense to populate all DIMM slots, because this will heavily reduce the bandwidth available, as the memory must run at lower speeds.

While HP has decided to offer registered and unregistered memory for their DL3xx G6 models, IBM only offers registered DIMMs. The x3650 M2 COG Guide offers a lot of in terms of possible configurations. The most important thing is keep the numbers of memory modules down – this makes it easier deploy them correctly.

Another important part is that memory is no longer ordered in pairs, but again separately as it was a few generations before. Most servers ship with two 1GB modules standard. This is not an optimal configuration, since you have three channels that could be used.

There are two ways to deal with this: either add a third 1 GB module and then add the rest of the memory you need. Or discard the two memory modules that come with the servers and just install the higher capacity modules you bought. My recommendation would be to discard the 1GB modules and install three 4GB modules – for most SMB environments, 12GB of memory suffices for almost all services.

Other part changes

The onboard RAID-Controller is gone, there is a new specially positioned but otherwise standard PCI-E slot for the RAID controller. The system ships with a non-BBWC controller called the BR10i. In most cases, it makes sense to replace the standard RAID-Controller with a ServeRAID MR10i.

Some configurations also require an enablement kit to drive all 12 disk slots. Only 8 disk slots are standard.

The RSA II Slimline is gone. IMM now offers a lot of the RSA II functionality by default, but the most important functionality, remote KVM still requires a so called “Virtual Media Key” (as it enables Remote Media as well). In general, if a customer has used RSA II Slimline up to now, also include a Virtual Media Key. This enables full IMM functionality.

SSD offerings have also been added. Currently, the pricing for the 50GB SATA SSD is 3970 CHF in SSCT. My assumption would be that this is a pricing error, but i’m not too sure about that.

Pricing

Pricing hasn’t changed much. DDR3 memory seems to be a bit more expensive than the FB-DIMMs were, but that’ll pass as soon as DDR3 volume ramps up. I’ve created a sample config that is probably valid for most SMB deployments that clocks in at around 10k CHF. This is roughly the same as it was before with the standard x3650, except that the x3650 M2 will deliver a lot more performance for the money.

HP DL380 G6 Quickspecs
HP DL360 G6 Quickspecs

There are some interesting things to see there – unlike IBM, HP chose to go with 18 memory slots, allowing each CPU’s three channels to drive three memory modules. HP also offers a flash-based Memory configuration tool, that alleviates having to learn complicated placement rules.

HP also offers more disks per system, allowing a DL 380 G6 configuration with 16 2.5″ drives in 2U server. The DL360 G6 can have up to 8 2.5″ drives – that’s the same number of drives that previous generations had in a 2U form factor!

Having the ability to add lots of local storage is a good thing in smaller businesses. Larger companies probably won’t need that as much. Interestingly, HP choose to stay with a traditional BIOS with their G6 series. IBM is already offering UEFI on their x3550 M2 / x3650 M2.

Another new thing that i gleaned from those documents is the ability to choose a power supply based on the expected load of the server – making it possible to favor higher efficiency (for example, if you do not intend to have local disks in the server, as you’re planning to use your SAN).

The first Nehalem-based servers have already been released, and as always IBM is a quiet compared to other server manufacturers. However, today while looking at some support documents i saw that IBM finally offers downloads of IBM x3650 M2 and IBM x3550 M2 documentation:

x3650 M2 Installation Guide
x3650 M2 Service Guide
x3550 M2 Installation Guide
x3550 M2 Service Guide
IMM User Guide (RSA II Successor)

All Documents on Scribd

I’ve now skimmed through the documents a few times and found several things worth mentioning – please note that i haven’t seen the actual hardware, so there may be mistakes here. If i’m aware of the, i’ll correct them of course.

RSA II Replaced by IMM

The RSA II adapter has been succeded by the IMM. The IMM is integrated into both the x3650 M2 and the x3550 M2, but certain features need to be enabled by purchasing a “Media Key”. This is similar to how HP handles this issue, except with a physical key. The Media key enables additional features, as described in the IMM User Guide:

In addition, IMM Premium has the following features:
- Remote presence, including the remote control of a server
- Operating-system failure screen capture and display through the Web interface
- Remote disk, which enables the attachment of a diskette drive, CD/DVD drive, USB flash drive, or disk image to a server

More Disks

The x3650 M2 will support up to 12 hard disks. The x3550 M2 will support up to 6 hard disks. Only 2.5″ of course. I currently didn’t see much mention of SSD, but i’m sure both these machines will see SSDs during their lifecycle.

There are also several changes regarding the hard disk controller. The Onboard ServeRAID 8k made by Adaptec is gone. It will be replaced by MR10i and BR10i ServeRAID Adapters, made by LSI Logic. For more details about those controllers, see the ServeRAID Quick Reference. These adapters will fit into a special slot, and are no longer fully integrated into the system board.

This is a good thing – RAID controller troubles in the x3550 and x3650 always meant replacement of the system board, which could be problematic because it changed the various IDs of the system (most notably the MAC adresses).

More Memory

Nehalem brings CPUs with On-Die memory controllers to Intel-based servers. If you’ve never handled AMD servers before, this means a few changes.

* Memory is local to the CPU, meaning you will need both CPUs if you want to use the full 16 memory slots.
* Nehalem features a triple-channel memory subsystem, making memory placement interesting
* If you’ve wondered how triple-channel and 16 slots works, see the graphic to the right

More Ethernet ports

The x3650 M2 comes with 4 Gigabit ethernet ports. The x3550 M2 with two, but an option for an additional two without requiring a full pci-e slot. This is especially good when you’re using virtualization and need to segregate hosts or need the extra bandwidth.

UEFI & other features

UEFI is the successor to the common BIOS. This will get interesting – i have no real idea what to expect from this. I only know that WS08 supports UEFI boot directly, and that IBM will provide emulation for legacy operating systems.

There’s also an in-system USB port for running a Hypervisor directly from flash. The system also features an integrated TPM module, which can be used together with Bitlocker to provide full disk encryption in physically insecure servers.

An important Whitepaper from IBM about Transitioning to UEFI is also available.

NOS support

Both new systems have also appeared in IBMs NOS support list. While full Windows support is offered, it looks like VMware will not be supported, at least initially.

Support for high performance graphics cards

The system board lists an 8 pin power connector for high performance graphics cards. The parts list also lists NVIDIA FX1700 and NVIDIA FX570. This might be needed for CUDA or similar applications.

Conclusions

I want one. According to my sources, they should be available for sale starting on the 21st of April. As of today, the System x New Generation information page is finally online.

First, lets talk about the hardware. It’s important to know that small businesses handle their infrastructure completely differently than large businesses, and in my opinion there are some things that require “unusual” thinking.

Reliability: an SBS server is extremely critical for operation of a small business and they usually do not replace servers after three years

Maintainability: small businesses do not have dedicated IT personnel. Usually, most “heavy” tasks are done by an IT service provider, and the daily IT tasks are done inhouse by someone as a secondary job

Functionality: small businesses are sometimes just as demanding as larger companies – the small size requires setups that maximize the productivity of each employee

So, there are three main aspects one should focus on when deploying an SBS server.

Reliablity is a key aspect. An SBS server is critical for the business and this requires hardware that is highly capable and reliable. After all, an SBS server can serve a Business for up to five years without replacing the hardware. This is why it makes sense to buy really good hardware that lasts that long, combined with appropriate maintenance contracts to get it back up in case you run into problems.

In this case, we decided to use the following hardware:

IBM System x3500
Intel Quadcore 2.66 Ghz 12M (leaving 1 slot available)
10 GB Memory (leaving 6 slots available)
8 2.5″ 147GB 10kRPM SAS Disks (leaving 4 slots available)
ServeRAID 8k with 256MB BBWC (for the first 8 disks)
ServeRAID 8s with 256MB BBWC (for the other 4 disks)
Disk configuration:
RAID 1 consisting of two 147GB Disks
RAID 5 consisting of five 147GB Disks
Global Hotspare
IBM SAS HBA (for tapedrive)
LTO4 SAS attached HH internal tapedrive
Redundant fans & power
IBM Remote Supervisor Adapter II
5Y of IBM ServicePac with committed service option

As you can see, the system has lots of storage and redundancy. It’s also important to know that the SBS server does not run any third party applications (except those necessary for operation), the ERP runs on an IBM POWER machine. With SBS 2008, i would not recommend running any third party applications on the SBS itself – if necessary to run third party apps on a server, purchase SBS premium and run the third party apps on the second server.

The second aspect is ease of use, for which we should use software that can be automated as well as possible. SBS handles lots of things on his own, but we opted for a third party backup application because we still see tape backups as the best way to fulfill most of a customers needs. Especially since LTO drives have WORM media, that can help to comply with certain local laws.

As for the software, i’ve installed BackupExec 12.5 to handle the backups to tape. Tape backups are easier to handle for customers, offer superior performance, and make archival and external storage of data easy. Unfortunately, BackupExec 12.5 does not integrate with the SBS Console (yet?).

For virus scan, we’ve opted for McAfee VirusScan Enterprise. A central management application was unfortunately not yet available, so we deployed McAfee manually on each client, and on the server.

Exchange is protected using ForeFront for Exchange, which has served me well in the past.

SBS 2008 has a nice reporting function, but there are other important notifications: the RSA Adapter notifies for all hardware failures like power supplies, fans, etc. independently through e-mail (which can contact external adresses and even works if the failure killed the server), and allows remote troubleshooting in case the machine does not boot. ServeRAID manager and BackupExec also send daily reports to be viewed by the customer.

Last, but not least, is functionality. After all, customers aren’t like me that want an SBS because they like technology – no, they want an SBS to fulfill certain needs their business has.

In this case, there were several unique requirements regarding mailflow – thanks to the included Exchange 2007 server, which offers a very flexible transport rule system i was able to implement these requirements without having to purchase third party software or even program event sinks on our own.

So far, i’ve had zero issues with SBS 2008 – it worked without any problems and hardware support wasn’t a problem either. Looks like IBM got all the kinks figured out since the release of Windows Server 2008 at the beginning of the year.

The System x3500 is also very nice hardware – it looks like a tower variant of the x3650, which i also like very much. The only criticism i have for the machine is that installing the redundant fan kit is total pain in the ass, mostly because the documentation covers both the x3400 and x3500, and some parts don’t apply for the x3500 (but aren’t marked as such).

It shows that IBM can still deliver top notch hardware at affordable prices.

In case you couldn’t figure it out from reading this far, i really like SBS 2008, and it’s ready for action in a production environment. In case you’re thinking about deploying a new SBS, go with SBS 2008!

• Intel Xeon 3065 2.33Ghz 4MB L2
• 512MB ECC RAM
• E200 SAS Controller (8 Ports, 128MB BBWC)
• 2x 160GB 7.2kRPM SATA Disks

For less than 400 CHF. As i needed a machine to run SBS2008 at home, and my current one wasn’t 64bit capable, this seemed like a good buy, especially because the E200 with BBWC alone is worth around 300 CHF.

Of course, i needed more RAM and disk space. I also ordered 4x 2GB memory modules (with ECC) from a third party memory manufacturer (Transcend) – priced at around 80 CHF each. I also ordered 4x Western Digital 1TB disks that are optimized for 24 hour use, priced at around 180 CHF.

This brought me to a total price of around 1500 CHF. I had two 160GB disks that i didn’t have any use for (except throwing them at people i don’t like).

1500 CHF is a lot of money for me, but for a company it’s nothing – still, this is ideal for experimentation. The free ESXi supports the E200 SAS controller, making it easy to build a test lab based on VMware – also, Windows Server 2008 x64 and Hyper-V also run flawlessly on the machine.

The machine is also very quiet, making it possible to use it in a normal appartment or in your office.

You get what you pay for still applies – the machine has no remote management features, only a single network port, forcing you to use the same port for management and virtual machine traffic, which can be acceptable in a test environment. HP’s System Insight Manager is not supported on this machine, either.

The case is very small, resembling a normal HP client minitower. The mainboard supports ECC memory, which is becoming more and more important with todays memory sizes. Unfortunately, it only offers four memory slots with a maximum capacity of 2GB per stick, maxing the machine out at only 8GB of RAM.

The integrated E200 SAS RAID Controller has a 128MB BBWC card, that allows it to use it as a write cache, and enables licensing to use RAID5. In my case, i used RAID10. The disk performance is better than anticipated, even though i’m using slow consumer drives, the performance for running VMs is acceptable.

The machine has three x8 PCI-E slots and a single PCI slot. One of the x8 slots is used by the E200 controller.

This offer is still available under HP Part# 470064-639, and there are still some companies that are selling it for the lower promotion price.

I’m currently running SBS2008 directly on the hardware, with not virtualization in-between. The performance is good, but i’d still never use such a setup for a production deployment at a customer – the management options, hardware flexibility, redundancy etc. just aren’t fit for production.

Update: I was asked about Linux compatibility on this machine. See the official HP Linux compatibility list. The E200 SAS RAID Controller is supported by the cciss driver, which is in the vanilla linux kernel. So most distributions will be able to install on this box – support is another matter, though.

There is no easy way to get official support for non-corporate versions of Linux, like Ubuntu. My usual way in those scenarios is to run Linux as a VM under ESXi, but that doesn’t work with the ML110 as ESXi is not supported (but works).

I’ve seen the “big” ESX in a few places and worked a bit with it, but i decided to refresh my knowledge on VMware a bit. For this, i first had to scrounge up a machine that was able to pass the rigorous HCL from VMware.

Unfortunately i didn’t find something that was really a Small Business machine – i used a HS21 Blade from my BladeCenter S testing environment.

The HS21 blade has 4GB RAM, a 2.66 Ghz QuadCore CPU and two 500GB SATA Harddisks attached to an LSI1064 SAS Controller. Fortunately, this configuration is supported.

Installing ESXi

Similar to the installation of Windows Server 2008 or Windows Vista, the ESXi installation is extremely streamlined. All you have to do is pop the CD in, select the disk where you want to install ESXi and then let it continue. The whole setup took around 15 minutes, most of the delay owed to the extremely slow Laptop CD Drive installed in the BladeCenter S.

After installation, the Blade rebooted and you will be greeted by an extremely simplistic interface that allows you to change basics like the password of ESXi and reconfigure the management network interface and also display a few logfiles. On first startup, it also showed my a Web address where i can download the VI Client that is used to manage ESXi.

A very pleasant experience.

Installing the VI Client

After accessing the ESXi host through HTTP, i could then download the VI Client. Installation on another Blade running WS2008 was smooth. It also installed an Update Service that allows me to upgrade ESXi.

Configuring ESXi for the first time

After logging on using the VI Client to ESXi, i was greeted with a nicely detailed instructions that i would need to create a datastore. After few clicks i had a datastore created on the RAID1 that ESXi was installed.

The VI Client looks very impressive and neat. It looks like ESXi can read diagnostic information from the Blade, and can monitor RAID, Fan and other stati easily. One of the things i really like about this is that you get a standardized interface for monitoring your hardware – on Windows you usually have to use tools like IBM Director that are just one big mess to handle. Here, i didn’t have to configure anything – it just worked.

After entering licensing information, configurating a static IP Address, changing hostname and DNS information, i rebooted the blade.

Creating the first Virtual Machine

I decided to create a first virtual machine – the blade i killed for running ESXi was previously running Exchange 2007. As this is just a demonstration setup, i decided to recover the preexisting Exchange server into a VM, in order continue having a full featured demo setup.

So i created a new Virtual Machine, configured for running Windows Server 2008 x64. Now, i didn’t have WDS setup in the Demo Environment, so i had to find a way to boot the Blade from an ISO. Previously i used scp to copy the ISO to the ESX Management Partition, but that didn’t work on ESXi. Luckily, the VI Client has a “Datastore Browser” that allowed me to upload files to the vmfs3 filesystem.

After uploading the ISO, i booted from it. The installation was pretty slow, but comparisons to my Hyper-V hosts aren’t fair as those run 10kRPM 147GB SAS Disks in a RAID5 configuration instead of the slow-as-molasses 500GB 7.2kRPM SATA Disks.

After OS installation, i immediately installed the VMware tools. One reboot later, i had a working Windows Server 2008 machine.

One of the things i noticed: When running WS08 virtualized on Hyper-V with 4 virtual CPUs on a Quadcore machine, WS08 thinks i have on Quadcore. On VMware, WS08 thinks i have 4 real CPUs (Sockets). This can bite you if you want to give a WS08 Std Machine more than 4 Cores – as WS08 Std is only licensed to four sockets.

The next step obviously is restoring the Exchange server, but that doesn’t really have to do all that much with ESXi.

Conclusion

ESXi is great. One of the biggest advantages over Hyper-V is the VI Client that consolidates a lot of information that is all strewn about in Windows. For example, it has built-in performance metrics, raid status monitors, etc. You can get all the same information with a machine running Hyper-V, but you’ll have to use other tools for that (of course you can customize a MMC do include Perfmon, but it’s not exactly the same).

VMware shows that they have gained long term experience with Virtual Machines, and the VI Client clearly shows the maturity of their product.

Permission management seems much better than Hyper-V, but i didn’t find a way to use Active Directory integration. Maybe Virtual Center is required to this, or i just wasn’t able to find it in ESXi – it exists, because there are numerous references on the Web.

I’ll certainly consider using Hyper-V when i have to run non-Windows guests. For Windows guests, Hyper-V with it’s VMbus architecture seems better suited. For non-Windows guests, VMware can’t be beaten right now.

The BladeCenter S

The BladeCenter S i received came with 10 500GB SATA Disks and two DSMs, four power supplies, an Advanced Management Module, a Server Connectivity Module and a SAS Connectivity Module. The power supplies use standard 230V type 23 plugs, which do require a little special installation, but much less so than industrial plugs used with the bigger BladeCenters.

The big point about the BladeCenter S is that it does not require an external SAN to provide Storage to the Blade – an integrated SAS Switch that allows very flexible disk configurations is integrated. Configuration can be done using a Webbrowser against the SAS Connectivity Web Interface, using SSH/Telnet to access the SAS Connectivity Commandline, or using a fully graphical interface using IBM’s Storage Configuration Manager. There are some predefined configurations, but none of them suited my configuration – creating new configurations using SCM is easy enough though.

The disks in the BladeCenter’s DSMs (Disk Storage Module) are hot swappable – currently, only 3.5″ DSMs are available, with a 2.5″ DSM in the pipeline. Most of the blades support one or two internal disks, but the problem here is that these disks are not hot swappable. Depending on your Blade loadout, 12 disks might not be enough. For example, the HS21 XM Blades only fit one internal disk, and running without RAID on the System partition seems pointless, so you would be using at least 6 disks (without hotspares) for a basic Exchange deployment.

The Webinterface on the AMM is nicely done, although it lacks a bit of flashiness. That’s not a requirement though, it does a very solid job at what it needs to do.

After powering up the BladeCenter S for the first time, i connected to it using a web browser and upgraded all the firmwares. There are quite a lot of them (AMM, SAS, Server Connectivity), but it all worked out flawlessly. Time to move on to the real course: the Blades.

The HS21 and the HS21 XM

Starting with the familiar first, i started with the HS21 Intel Blades first. The two HS21 Blades both had a 2.66 Ghz Quadcore and 4GB RAM, the HS21 XM Blade had a 2.5 Ghz Quadcore and 9GB of RAM (more about that later).

When starting the first HS21 Blade, after configuring all the storage using SCM, it failed to POST it’s LSI Logic SAS/RAID Controller. I searched for the error message on the net, assuming that i screwed up the configuration. I didn’t find anything meaningful, so i tried to do what everyone else would do in this situation: Apply every Firmware update for the Blade i could find.

Of course it wasn’t as easy as i wanted it to be. The controller not POSTing was an endless loop, i couldn’t get the machine to start from the AMM virtual floppy drive. I used SCM to disconnect the storage (by disabling the Blade’s SAS port). Now, the blade booted flawlessly, indicating that i probably had a problem with my disks. When browsing the IBM website, it became obvious that only newer firmwares support SATA drives. After upgrading the SAS Firmware, i was able to boot the blade without disabling the Blade’s SAS port. Unfortunately, the onboard SAS controller only supports RAID level 1 and 10. Probably owed to the fact that most blades are using SAN storage – IBM promised that there would be SAS RAID adapter that supports other RAID levels – these are especially important for the cost-conscious SMB market.

I booted a Windows PE 2.0 using WDS, and was able to install Windows Server 2008 x64 without any issues.

The HS21 XM blade on the other hand complained when booted for the first time that it’s memory configuration was invalid – it only supports 2, 4 and 8 DIMM configurations – 6 DIMM configurations are not supported. I removed two 512MB modules and booted the Blade with 8GB – it worked flawlessly and without complaining.

The JS12

First, read this document about i on Blade. It explains everything better than i ever could.

The JS12 is a POWER6 based blade that is able to run IBM i. The first time i turned on the blade, all the HS21 blades (already running Windows Server 2008) crashed hard. When rebooting, they no longer found their drives. I turned off all the blades, disconnected the JS12′s SAS port and turned everything on again. The Intel blades booted, and after i was sure that they’re up and running again, i powered on the JS12 again. This time, no issue arised. I tried to reproduce the behaviour i’ve seen before, and the same thing happened again.

My current assumption is that the issues were caused by the SAS Controller which does not have a Firmware update yet, and can’t deal with the SATA drives located in the DSMs. Further investigation told me that there’s no firmware upgrade for the SAS Controller in the POWER6 blade, and that SATA drives are not supported when running IBM i on the blade anyway. I ordered 4 147GB SAS drives, disabled the SAS port on the blade, and tried booting the POWER6 blade again. It booted flawlessly again.

The next step was to install VIOS – this is a rather complicated multi-step process. First, you have to turn on “Serial over LAN” aka SOL, then logon to the AMM using SSH, connect to the POWER blade using serial passthrough and then boot the blade from the VIOS CD. The install is pretty self explanatory, but takes forever. Expect 3 to 4 hours.

Next is connecting to the Integrated Virtualization Manager (IVM) running on the VIOS partition. The IVM is basically a HMC light minus the console functionality. The only way to get a console on the JS12 blade is using a LAN console (which can only run on consumer versions of Windows, and is not supported on most of the Blades).

I installed the latest VIOS patches (around 4GB) and enabled mirroring on the two 147GB SAS disks in the blade itself. The next step will be installing IBM i, with which i have to wait until i receive the ordered SAS Disks. Installing the patches also takes quite some time, around 30 minutes.

Preliminary Summary

The BladeCenter S is great. Yep, not everything ran flawlessly from the start, but nobody’s perfect from the beginning. The BladeCenter brings an innovative new perspective to the SMB market. The problems that IBM needs to address are the addition of 2.5″ DSMs (already in the works) and more capable RAID controllers (also in the works). A BladeCenter S with the ability to use around 20-40 disks could prove interesting.

The POWER6 Blade is interesting, and while VIOS adds complexity, it is as streamlined as possible. I’m interested about seeing IBM i running on the machine.

If you have any other question about the BladeCenter S – or anything you would like to see in detail, post a comment. I’ll try to figure it out.

The fact that i haven’t got any interesting project running right now, and IBM isn’t shipping me new toys i just decided to complain about delivery times. This might not actually be interesting.

When it comes to delivery times, my experience with IBM has always been abysmal. The local swiss distributors (Avnet, TechData, Also) are usually low on stock, don’t have any System i stock (automatically giving you 2-8 weeks, depending on IBM’s mood). Software orders also take ages, especially additional software on new Systems (we usually buy 5722-IP1 for our customers to generate PDF). These software orders usually take 2+ weeks after the system was shipped.

Even System x servers are rather hard to come by. While there are usually a few (less than 5) models on stock, additional parts (like disks, memory) are sometimes out of stock too. Out of stock articles usually take up to three weeks.

The situation is even worse with IBM printers – no stock anywhere, and delivery times are 4+ weeks.

Other manufacturers understand their supply chain even for smaller countries like Switzerland. One of the good examples here is HP. You usually have many servers plus parts available on stock – every reasonable printer model (less than 10k) is also available on stock. Microsoft doesn’t have any problem shipping their licenses (one or two days for electronic licenses), and shipping time for media kits is a simple next business day thing.

Why can’t IBM get their act straight and have their (expensive) stuff available on a simple next business day basis?

It’s hard to sell a customer an IBM printer if you have to tell him that it takes several weeks until he gets it. Most smaller businesses don’t buy maintenance contracts for their printers, and just buy a new one when the old one dies – this doesn’t work when you have 4 weeks of lead time for a simple InfoPrint 1532.

One thing i would be interested in: Is this better in the USA? Or in Germany?

The x3200 brother is the rackmounted System x3250. Both of them are IBM’s low end entry systems. The x3200 we sold to our customer was one of more well endowed models, featuring redundant power supplies and hot plug SAS disks.

The disks come in a standard 3.5″ form factor, there are no 2.5″ models available (which makes sense, as towers are not really space constrained, which is clearly visible when looking at the x3200 bulky frame).

The exact configuration ordered:

• System x3250 Xeon 2.13Ghz DC, with 2×512MB Base Memory, 3.5″ HP SAS, redundant PSU
• 2x 72GB 10kRPM SAS

Unpacking and opening

The machine was shipped in a box where you’d have thought it contains a 5 year old desktop PC, meaning it was a bit bigger than the Lenovo ThinkCentre tower shipping boxes. As always, removing the machines from these boxes is not as fun when you’re alone, because the styropor sticks to the machine.

Another thing to note is that the machine shipping without any power cables, which is normally not the case. But this might’ve been a mixup at our distributor.

The machine itself is big and bulky (exactly as it looks on the photos), but the case is very well done, much better than the xSeries 226 had. Everything is tool less, and the opening mechanisms for the front and side cover work nicely, and fit like a glove when putting them back on.

Interiors

Even though this is a budget machine, the interior is done rather well. The cables are packed together nicely, and the system has room for expansion. 4x 3.5″ HP SAS disk trays, 3 PCI slots, 2 PCI-E Slots (1x, 4x). The LSI Logic SAS RAID Controller is mounted directly on the mainboard, saving expansion slots. Again, this machine only accepts 4 DIMMs, which aren’t that accessible. But this isn’t a huge problem, as maintenance on tower models has always been awkward and finicky – that’s what rack servers are for.

The machine isn’t quiet, but it isn’t loud either. I wouldn’t mind having it in my office, the noise is not a high pitched scream like you usually get from a 1U rack server. There is no inline documentation like IBM usually provides with their rack mount servers, and there’s no LightPath diagnostics either. That’s perfectly normal for this price.

What isn’t usual for this price class is the fact that this machine has redundant power supplies. While this is the norm for more expensive servers, it isn’t for entry level servers. The redundant power supplies do not cost a lot more than the normal model, and it’s always nice to have redundant power (as UPSes account for a large number of power failures, at least here in Switzerland).

Installing options

We only got a few disks with the machine, and installing them was a breeze. They are hid behind an easily openable lid, and come in a standard IBM hotplug mounting tray.

The machine we received had horrible outdated firmware, so the first step was to get everything up to speed. This worked fine with an USB floppy drive, as IBMs Update CD’s weren’t current (again). I still think there should be some method that does all this whole box-update thing via the internet. Not sure how this could be implemented without astronomic cost, but i still want it.

The Onboard LSI Logic RAID Controller supports mirroring and striping, and brings it’s own horrible management software – it’s not an IBM ServeRAID family controller. I didn’t even find a way to automatically send mails in case of a disk failure.

Booting the server

We’ve installed Windows Server 2003 R2 SP2 on this machine, not using the ServerGuide procedure. Again, the install went through without any problems after supplying the LSI Logic driver on an USB floppy. I’m still waiting for Windows Server 2008 which will make this a lot easier with it’s Windows PE 2.0 based installer.

As far as my first impressions went, the disk performance is very good. It’s quite noticeable if you use 10kRPM SAS disks against 7.2kRPM SATA disks.

Resumee

Even though it’s a budget machine, the build quality and the features of the System x3200 are quite impressive (i really, really liked two PSUs in such a small machine). I still don’t like tower machines, but the System x3200 is worth it’s money if you don’t have a rack at a given location. The system is very well designed, and could even serve as a small business server for a very small business.

Also, the obligatory plug to DATALINE AG which sells this server and other IBM System x or System i servers.

So i got stuck with a probably refurbished systemboard, and instruction sheets on how to replace them. I’ve also made better pictures of the missing pin, and the whole replacement procedure. Unfortunately due to the digital camera which seems to be over 9000 years old, and the low light situation, the pictures weren’t that much better.

One thing that irked me though is what the HP rep said to me yesterday. He estimated shipping at “one or two weeks”, which is far too long for replacement parts of a still more or less modern server. On the other hand, the parts arrived today with overnight shipping from around the world. It seems that HP does better than their reps say.

Replacing the systemboard in a DL140 G2 is not as bad as one might think it is. While there are several screws to unscrew, and of course the processor replacement itself, i never had a problem during this procedure. Only one thing i wasn’t sure with – HP’s docs stated that i had to empty my entire syringe worth of thermo conductive paste onto the processor. I did that, and the processor temperature seems to be okay now.

I restarted the machine a few times, and the machine brought the message that a critical error was encountered, and the system thus halted – right after the BIOS welcome screen. I tried to enter the BIOS, but failed a few times. The third one worked, and i looked at the event log in the BIOS. There were several error messages regarding voltage, but they were over a year old. I deleted all error messages, rebooted the machine until it halted again, and then entered the BIOS again. This time, there was an error Message regarding ECC Memory being defective.

I turned the machine off, set it on a table, and opened it. The problem was obvious immediately: The Northbridge cooler no longer was attached to the Northbridge. A tiny little metal nook which held the cooler in place was not found immediately – it was later located in the RAM slots.

I’m sure that i am the first and only person who opened this machine in the last few months. The Northbridge cooler obviously broke off by itself, indicating cheap manufacturing as being the problem.

This machine is a year old by now. Warranty expires tomorrow. Luckily we’ve opened a case today, and hope we can get a fix soon. I’m very interested on how this plays out with HP.

When first looking at the specs for this exams, i found the required passing score to be very low: 59%. HP’s exam had a passing score of 71%. As such, i expected a very difficult the exam, but that wasn’t the case.

This exam focused on three points:

• Normal, hardware neutral Windows/Networking Knowledge
• IBM Director Knowledge
• IBM System x hardware knowledge (especially of bigger products like the System x3850)

I’ve passed this exam with an okay score, but was guessing at almost all the IBM Director and the System x hardware knowledge. The problem here is that neither IBM Director nor 4 socket machines are being used in the small businesses i work for. The fact that i still passed shows that good guessing can give you lots of points, because the answers are sometimes rather obvious.

There was also a good deal of normal Windows Knowledge required, and several very very basic Networking questions, which could probably be answered by your average gamer kid. If you already hold an MCSE/MCSA on Windows Server 2003, you might be able to pass this exam without having touched an IBM server before.

Even though this may sound bad, i though the exam in it’s whole made a lot of sense. Most questions where detailed, and the answers where short an concise. Again, for several questions you had to know what the limitiations of a given IBM hardware platforms are. I still don’t really like this, because i don’t deal with every piece of System x hardware IBM has to offer (even though i’d like to know, that’s currently not the case).

As expected, there were many performance tuning questions which were usually answerable with general IT knowledge. You’ll just need to know which system performance values indicate what kind of performance bottleneck.

Again, i have no idea why, and there is no information from Layer One available yet, but we can rest assured that they will state some nonsense again. As things are right now, i can only recommend against choosing Layer One for your colocation needs. They’re obviously unable to ensure even basic things like power delivery, which is a lot worse at Layer One than it is here, at home, without an UPS, directly on the power grid.

I’ve written about the IBM x3650 before, and i’m generally very fond of this machine as it’s build quality and features are impressive.

However i’ve heard from several customers that they want the similar Model from HP, the DL380 G5, with the only reasoning being that the machine is going to be less expensive.

I’ve configured two almost-identical servers, and the price difference is clearly there. Please note that the configuration of the two machines is not the same, mostly due to real world restrictions in the Small Business segment when ordering machines. These restrictions are here in Switzerland – they might not exist in your country, and they might have changed in a few months. So take this with a grain of salt.

The configuration for these machines are thought to serve a Small Business with Windows Small Business Server 2003 R2.

Configurations

IBM System x3650

Configuration:

• Intel Xeon DualCore – 2.00 GHz
• 4 GB Memory (2×1 GB, 4x512MB)
• 4 x 147 GB SAS 2.5″ 10kRPM HP Disks
• RAID-Controller ServeRAID 8k 256MB BBWC
• DVD-ROM 24x/8x built-in
• PCI-X Raiser Card
• PCI-X SCSI U320 Adapter for tape drives
• Builtin Ethernet-Adapter Gigabit; 2x RJ-45 Copper
• Remote Supervisor Adapter II
• Redundant PSU
• 36 Months Warranty on-site NBD

Total list price: 9500 CHF
Warranty Upgrade to 7×24: 1000 CHF

Total: 10500 CHF

HP DL380 G5

Configuration:

• Intel Xeon DualCore – 2.66 GHz
• 4 GB Memory (2×2 GB)
• 4 x 147 GB SAS 2.5″ 10kRPM HP Disks
• RAID-Controller P400 256MB BBWC
• DVD-ROM 24x/8x built-in
• PCI-E SCSI U320 Adapter
• Builtin Ethernet-Adapter Gigabit; 2x RJ-45 Copper
• iLO Advanced License
• Redundant PSU
• 36 Months Warranty on-site NBD

Total list price: 7000 CHF
Warranty Upgrade to 7×24: 1450 CHF

Total: 8450 CHF

Conclusions

Okay, so we have a pretty major price difference between those two machines. The HP machine costs 20% less than the IBM machine. These prices don’t apply when you buy them by the truckload, but they do when you buy one or two.

Objective differences

About the IBM System x3650:

• 12 instead of 8 memory slots available
• No PCI-E slots needed to drive integrated disks (ServeRAID has a seperate slot)
• RSA II offers integrated email functionality (great advantage when not using IBM Director)
• Space for integrated tape drive (I’ve never used those, though)
• PCI-X raiser needed for U320 attachment
• Way better fan redundancy (N+N instead of N+1)

About the HP DL380 G5:

• 20% less expensive than IBM
• Greater choice on disk adapters
• iLO is standard, though KVM/Media functionality must be licensed
• More flexible memory ordering options
• U320 adapters with PCI-E is available – no PCI-X raiser card needed
• One slot always occupied by disk adapter (However, the machine has 5 PCI-E slots, the IBM has only 4)
• Less memory slots available

Unobjective differences

About the IBM System x3650:

• It’s black. Black is cool.
• The LightPath diagnostics seem much more mature than HPs new diagnostic system
• The RSA II offers an older GUI, but more features than the iLO Advanced License at a lower price point
• Rack mount kit looks more sturdy and generally better than the HP one

About the HP DL380 G5:

• Legacy PS/2 ports still available
• The front looks way better, but it’s not black

Final conclusions

Make up your own. I’ve tried to be as truthful as possible, and both machines have advantages and disadvantages. Usually the choice is already made if a customer already works with a given brand. Both vendors have pushed for different feature sets and their machines, and this shows quite clearly. The IBM machine is more expensive, and the additional capacity might not have much worth in your company. It really depends on your needs and your budget. Both machines are cool, though.

This machine was purchased for use in a colocation center as a general web and mailserver for non-profit purposes, using Debian GNU/Linux 4.0. As such, there were serious budget constraints involved, which is why the machine chosen is on the lowest end of available (rack) hardware – as such, the machine doesn’t have a diagnostic panel (like HP’s DL360 G5 do, or IBM’s x3550 or x3650).

The machine was ordered using the new ALSO IVIS HP TopConfig configurator, and as such was shipped assembled (the delivery time was 9 days). Most in-stock articles have builtin CD or Diskette drives, which are a factor in price. With configured machines, you can save a few bucks with these things, and also get a better memory configuration (this is very important in low end machines, because they usually only have 4 memory slots (compared to 8 or 12 that better machines have)).

Here’s the configuration:

• HP DL320 G5 with 2.13 Ghz Xeon DC CPU
• 2×1 GB Memory
• 2x160GB 7.2kRPM SATA Disks

Unpacking and opening

This machine ships with the usual features known from HP – iLO 2, but without any licenses for advanced use (you can use it with a serial console that can be accessed with a Java Client). Also the slide-out serial number tag (an awesome idea in my opinion).

The machine also shipped prebuilt (which is nice, as i’m probably the most lazy person on earth) – the hard disk in the machine look like they’re hot -pluggable, and they theoretically are – but the onboard SATA controller does not support hot plugging. If you want that, you’ll need to buy a RAID controller from HP. I like this setup a bit more than IBM’s Simple Swap harddisks, but there’s no real advantage in daily use.

The components where assembled nicely, and the build quality is generally good.

Interiors

The general interior of this machine is very much the same as the x3250 – with a rather interesting difference: the machine has built in USB port, with lot’s of free space around it. I really have no idea what you’d want to do with this (if it was a desktop machine, you could probably use it with Flash for Vista’s ReadyBoost technology).

The fans in the machine are not hot pluggable, but redundant. The cabling is en par with the x3250, but there’s a slight disadvantage in extensibility. While the x3250 has a separate slot for installing specific IBM RAID controllers, there’s no such thing in the HP machine – you’ll have to use one of the PCI-E 8x slots for installing one of the HP RAID controllers. I do not consider this to be a problem in daily use either, because there’s usually not much need for expansion adapters in 1U machines.

Booting the server

The onboard controller is a standard Intel AHCI SATA controller (You will need to enable RAID functionality in the BIOS, and leave the RAID BIOS unconfigured – this will expose both disks to Linux using the AHCI driver). As of Debian GNU/Linux 4.0, this controller is now natively supported in the default configuration. There are many options in the BIOS to save a variety of contact information. Maybe useful in enterprise environments, didn’t play with them that much.

iLO configuration works fine as usual, but netbooting the machine was awkward – it refused to load our WDS bootloader several times, but succeeded finally (WDS then boots RIS, which then boots PXELinux – here’s how to do this). I have no idea if it is just an old switch acting up, or an actual problem.

Resumee

Is this machine better than the x3250? Make up your own opinion. The only real world advantage i’ve seen is that iLO 2 is included in the machine, and allows you to access a serial console for free. The iLO advanced option which allows KVM access (and a variety of enterprise integration features) is quite expensive at about 450 CHF. The x3250 does not ship with an RSA II, but the RSA II is only 250 CHF, a lot less than the KVM access license.

When you’re using Windows, you can use EMS to use the serial port, on Linux or other Unix based OSes you can usually redirect both the console and the kernel to a serial port (and of course you can redirect the BIOS).

I liked the disks, which seem to handle much more nicely than the Simple Swap SATA disks from IBM.

On this topic, i’ve found a very interesting link, IBM’s ServeRAID Reference, with lot’s of pictures and detailed specifications of each controller.

They’ve had power outages before, and again. However, it seems that they didn’t change anything. This is the fifth power outage, and we’re there for at most 1.5 years.

Today, there was a smaller power taking down only of the two power lines we had. But it still lasted for several hours, and recovery and information was incompetent and slow. Don’t go to Layer One. Their Power Grid sucks as much as their service and their information policy.