In silent mourning and loving memory, his family and friends keep Lukas’ website available.

Everything changes depending on who walks with us on our journey and who is missing.

WE LINGER ON THIS EARTH LIKE TRAVELERS WHO ARRIVE AT NIGHTFALL AT A SHELTER AND CONTINUE THEIR JOURNEY AT DAWN. WE WALK TOWARDS A DESTINATION FROM SHELTER TO SHELTER, ONE STAGE AFTER THE OTHER.

For each one, the destination holds a different meaning, and each one will use different words to describe it.

We were privileged to walk a part of our journey with Lukas. He has passed the last stage; there we will follow him one day. His gift to each one of us, which he gave so generously, was the time he shared with us. This gift has become even more precious now that he has gone from us.

We hope that all users of this website may find a piece of an answer to their questions on IT related topics, bringing them one step closer to a solution. Save journey!

Unfortunately, SCVMM 2008 R2′s P2V crashes when run on this machine. disk2vhd can produce a proper VHD from an EFI/UEFI install of Windows Server 2008 R2, but there’s not way of getting it to boot in Hyper-V (i tried a myriad of ways, including several Linux tools that can convert GPT disks to MBR-style disks, got the Windows Boot Manager installed, but it still wouldn’t boot).

So. What now? I’m out of reasonable ideas. I have opened a Microsoft support case regarding the SCVMM 2008 R2 P2V crash on an EFI machine, but i’m not sure i’ll get a quick out of this. If anyone has any ideas on how to get this fixed, i’d be thankful for any replies.

If i ever get a solution that does not include reinstalling everything from scratch, i’ll of course post it.

Update: Here’s the official statement:

There are no workarounds for moving a Windows system with an EFI partition to non-EFI architecture. EFI and Itanium are in lockstep. Classic x86 and x64 cannot boot EFI, and there and is no simple switch back to MBR boot.

Es tut mir Leid, das ich keine besseren Informationen für Sie habe aber das Feedback von unseren Development ist sehr eindeutig das keine Kombination von P2V / GPT bzw. EFI zur Zeit unterstützt wird. Mein Vorschlag wäre, unseren Service Request als “Dokumentations-Bug” für Sie kostenfrei zu schließen. Was halten Sie von meinen Vorschlag?

Bunch of idiots. Their agent shouldn’t crash in this scenario anyway and it should be documented that you can’t migrate machines installed in UEFI modes.

A window titled “Windows Installer Coordinator” will pop up behind the IBM i Access 7.1 Installer (hidden until you click on it in the task bar). This “Windows Installer Coordinator” will run indefinitively, without ever successfully installing the application.

Thanks to a helpful guy from IBM Software Support Austria, i now have a solution to this issue. It’s caused by a new feature in WS08R2 RDS.

It’s called Windows Installer RDS Compatibility. If this feature is enabled, IBM i Access 7.1 will not install successfully, and hang at the “Windows Installer Coordinator” window.

To successfully install IBM i Access 7.1 on a Windows Server 2008 R2 Remote Desktop Session host, set the following DWORD registry key to 0:

HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\Windows NT\Terminal Services\TSAppSrv\TSMSI\Enable

It’s possible that not all keys exist – in my case, the TSAppSrv and TSMSI keys didn’t exist yet – you have to create them manually. After creating this key, you can rerun the installation – a reboot is not necessary.

As the project progressed, we’ve found a few extremely irritating and hard-to-debug issues, which needed my involvement to figure out the root cause and get them resolved, without compromising the exam results.

Be aware that most of the debugging and research here was mostly done by our apprentice, not by myself.

There are several key issues with TMG, that we’ve noticed so far:

IP Blocklist Entries

If IP Blocklist Entries are present in Exchange 2010 Edge, enabling E-Mail Policy Integration will cause TMG to reject all further changes, with the following error message:

Windows Could not Start the "Microsoft Forefront TMG Managed Control" service on Local Computer
Error 0x80070057 : Parameter is incorrect

I’ve found this solution in the TechNet forums. You need to remove all IP Blocklist and Allow List Entries.

Extremely slow boot

Forefront TMG 2010 with Exchange 2010 and FPE 2010 installed will boot extremely slowly, requiring up to 30 Minutes to boot. This issue is caused by the coexistence with Exchange 2010.

Again, i’ve found a solution in the TechNet forums.

You need to set the service Microsoft Exchange Transport and Microsoft Forefront TMG Managed Control to Automatic (Delayed Start). This will reduce the boot time to about 3 minutes.

lsass.exe crashes when creating Edge subscriptions

The next issue we’ve noticed is that while the initial edge subscription worked, the second one didn’t. It crashed lsass.exe, which subsequently caused a bluescreen. Not a very nice experience.

Again, we’ve found a solution on the TechNet forums, and this is getting worse by the minute. The lsass.exe crash can be mitigated by removing all except one SSL certificate – not exactly a good approach since a TMG likely has multiple SSL certificates for publishing a variety of services. But it worked. Except that mailflow didn’t.

Outgoing Mailflow doesn’t work with TMG 2010

Of course, stuff wasn’t working yet. While incoming mailflow now worked flawlessly, outgoing mailflow didn’t – mails where stuck in the queue with “Primary Target IP Address responded with 421 Unable to establish connection”.

We’ve tried to look at this, but everything seemed alright – but we couldn’t modify any connectors on the Edge server – TMG prevented this, and thus we had no Verbose logging from the Receive Connectors. Changing the configuration in the Exchange Edge console resulted in the following error message:

Forefront TMG detected changes in Microsoft Exchange Server or Microsoft Forefront Protection configuration, and reapplied the e-mail policy configuration on server

So i’m not supposed to do that. The TMG console didn’t give me the option of enabling Verbose logging. We were stumped.

Luckily, further research showed that one could disable the integration between the Exchange Edge role and Forefront TMG – this was mentioned on this TechNet forums post.

After disabling this integration, i was able to allow Verbose logging. Which didn’t help at all, since the Exchange 2010 HT just wouldn’t show up in them, suspecting a deeper issue.

At that point, we’ve checked the receive connectors that were created by Forefront – and the internal Receive Connector didn’t allow Exchange Server Authentication. After setting that to enabled, we were finally able to send mail successfully using the Exchange Edge services.

Final words

Forefront TMG 2010 still seems to be in Beta. The integration with Exchange 2010 doesn’t work as nicely as it should. I hope these things get fixed soon with Hotfixes for TMG 2010. Until then, we’ve found workarounds for all of these issues.

I’m publishing this article as quickly as i can, because i’m most likely not the only one with these issues.

I have attended TechDays 05, 06, 07, 08 and 2010. While i’ve always had something complain about, there was always something to gain from attending. Not this time.

When reading this, keep in mind that i’ve only attended IT Pro Sessions (with the exception of the Windows Phone 7 Developer Briefing).

There were to many things that have gone wrong.

• The keynote was boring and it didn’t even remotely have anything to do with the job of an IT Pro or a Developer. The keynote speaker also used a MacBook with OS X/Keynote.app. Seriously.
• The food was worse than what I got to eat at my Berufsschule (which wasn’t very good).
• The long lines were still there – waiting 20 minutes in line for bad food isn’t my idea of spending the day. They should’ve solved this problem by now.
• No more English talks. Why? I think we could use some experts.
• All the talks were very basic. No In-Depth stuff. Nothing to learn.
• I don’t want a basic talk about what OCS 2007 R2 can do. We’ve been using this for two years. It’s old news. Talk about Wave 14.
• Giving a basic intro on what SCVMM and Hyper-V are is not an IT Pro track – these technologies have been out since years and everyone that’s interested will already know those basics
• Make sure your stuff works. 75% of the demos did not work. Most of them because of bad internet connectivity. Yeah, i guess moving all the stuff to the “cloud” is a good idea.

The location and the whole ship theme was okay though. The evening event was also nice, and they did have good food there (different catering organization). Not sure what some danish bloke was doing there yelling “in the cloud” about 50 times. I wanted to see some chair-throwing.

Another interesting tidbit: The number of iPhones at the event. I’ve seen more iPhones than WinMo phones.

So, did you attend TechDays? What did you think about it?

The most important thing between the laptop and the desktop is that i’m using BitLocker on the laptop, which might have an influence on things. I’ve always been using BitLocker on the SSD, so it would seem strange that this is now suddenly an issue.

I’ve always been aggressive about SSD firmware updates, after a good backup. I’ve upgraded both the Vertex and the Intel drives to be TRIM capable as soon as the respective firmware was out.

Unfortunately, a few days after using the OCZ Vertex in my new laptop, it started to have serious hickups – during which no IO would take place (perfmon disk queue shooting up to 50). During this time, the HDD light on the laptop is not lit.

I’ve tried to make sure that this issue was related to the SSD, so i ran HDTune benchmark:

This looked bad. Further investigation showed that there was a new Firmware out – 1.5. I’ve upgrade to Firmware 1.5, which supposedly had a Garbage Collection and TRIM support. After upgrading to 1.5, the hickups became much worse – the laptop needed about an hour just to boot up.

After looking at and posting on the OCZ support forum, i was told that i’d need to wait for Garbage collection to kick in. I let my laptop sit for a night, during which it crashed and the subsequent reboot was stuck on a “No harddisk found” message from the BIOS. Things looked bleak.

Further replies on the OCZ support forum requested that i do a sanitary erase, which would reset the disk to pristine performance levels (and delete all the data on it).

Unfortunately, the machine was too slow to run a Windows Complete PC Backup (wasn’t finished after 4 hours in). Fortunately, all the important data on my laptop is backed up using the Client Protection of DPM 2010, meaning all i had to do was reinstall my apps and i’d be good to go.

After reinstalling Windows 7, i installed the most important apps and then reenabled BitLocker protection – during which the hickups started happening again. The laptop would sometimes hang for 20-30 seconds, and then continue on on it’s merry way.

At this point, i went to sleep and let the laptop idle at the boot selection screen, so that the garbage collection could do it’s magic.

And now here we are, 8 hours later. While the read performance using HD Tune is nowwhere near as bad as it was before the sanitary erase, the write performance is stil abysmal.

For Comparison, here’s my Intel X25-M G2:

What now? I think i will RMA the drive. It’s the only choice i have left at this point.

If anyone has a better idea, give me a whirl.

The W500 i had had a 15.4″ 1920×1200 Panel, which wasn’t too great. While the high fidelity was certainly nice, the screen was very, very dark. It could only be used indoors, and required you to darken the room on sunny days.

Today i’ve had the chance to upgrade from the W500 to a T510, which i did. So far, i’m very much impressed with the changes Lenovo has do to this device. The W500 is running Windows 7 Enterprise x64.

• New controls for volume, microphone mute. Much easier to use than before
• New bigger and multitouch capable touchpad. As i prefer the touchpad over the TrackPoint, this is something that helps me tremendously
• Integrated Camera and eSATA connectivity
• Improved connectivity layout

There’s only one thing that i don’t like very much right now – the redesigned keyboard. As part of my job i deal with IBM’s IBM i platform, which still makes use the Function keys – which have all been shifted to the right for one key. So i regularly press F3 instead of F4, but chances are i will get used to it.

There’s one thing that worked very well – moving my Windows installation from the W500 to the T510. I’ve disabled Bitlocker protection, removed the OCZ Vertex SSD from the W500, placed it into the T510, booted it up, Windows installed several new drivers. Then, i installed the Intel LAN drivers from an USB stick, rebooted once more and installed the rest of the necessary drivers from Lenovo’s driver matrix. The whole process was done in less than half an hour, and reenabling Bitlocker protection was a breeze.

Windows 7 automatically reactivated by contacting our KMS servers, and i’ve had to reactivate my Office 2010 Beta manually, which also worked flawlessly.

While this portability is great (and also existed with Vista), it’s something I was able todo with Linux back in 2004 (assuming of course that the kernel had the storage drivers you required).

I’ve been using ThinkPads exclusively since 2004 – my first ThinkPad was an R51 and my first new laptop (my first laptop was a Compaq Armada i’ve bought used for 50.- CHF). When Lenovo took over the brand, i wasn’t to sure what to think of it, but having gone through several iterations of ThinkPad devices now (R51, T60, W500 and now the T510) i can see that Lenovo is commited to provide further well built, high performance devices.

Both the T60 and the W500 are still in service, neither of them are broken. The T60 is used by my apprentice and around 3 or 4 years old. We’ve replaced the Mouse and Keyboard to mitigate the wear and tear of several 40 hour work weeks on the device, but aside from that it stills works great.

Interestingly, the Hotline wasn’t reachable either – busy signal, Swisscom text “Leitung gestört” or simply “Call Failed”.

Lasted from 19:33 to 20:30, but it looks like everything is back online now.

As a disclaimer, i realize that Debian in any version isn’t a supported OS on Hyper-V R2 – i just want to tell of my experiences with this unsupported configuration.

The hardware, an aging IBM xSeries 306m with a Pentium 4 CPU wasn’t getting any younger and after a drive failure about half a year ago that lead to a system crash (No data loss though – it just crashed the machine, that’s Software RAID for you), it was finally time to modernize this.

The plan is to consolidate all our DMZ workloads (ISA, OCS Edge, XMPP Gateway, Exchange Edge) on Hyper-V 2008 R2 and doing the trickiest part first seemed like a good idea.

So i created a new VM using SCVMM 2008 R2, selected Other Linux 32bit as the guest OS, inserted a Debian 5.0 netboot CD and that’s where the problems already started. While the installation worked well in general, the Framebuffer used by the Debian installed is awfully slow. So it took me about half an hour just to get the install done (on a 5GB partition of the 80GB VHD).

After finishing the installation, i formatted the rest of the disk appropiately and then used rsync to transfer the machine contents over. A short bit after reconfiguring Grub, i could choose to boot either the transferred OS with it’s kernel, or the Debian 5 rescue system i installed alongside.

Booting the transferred system worked well enough, but the tulip driver wasn’t compiled into that (custom) kernel and building the module failed. So i read up a bit, and realized that the newest kernel (2.6.32.8) shipped with experimental Hyper-V VMbus drivers, that allowed synthetic NICs to be used.

I tried to compile the kernel after chrooting into the old installation, but it failed because gcc was too old. Not to worry, i compiled it in the rescue system, but couldn’t install the dpkg that make-kpkg created. So i installed it manually, which worked pretty well.

One reboot later, i was back in business with the extremely verbose Hyper-V drivers cluttering up dmesg, but the Synthetic NICs showed up as seth0 – seth2. After quickly changing all the necessary configuration files, everything was working.

After a bit of more testing, i disconnected the physical machine from the network and plugged the VM into the production VLANs.

I tested everything thoroughly and didn’t find any issues. Sent out an information mail and continued on my merry way.

Half an hour later, i decided to do a quick systems check again – and i realized that the external interface (seth2 in this case) wasn’t working anymore. tcpdump showed no packets being received and other machines in the same VLANs didn’t see any answers to their ARP requests either. So i rebooted the VM, and everything was working again. No error messages of any kind, neither in dmesg nor in the system logs or on the Hyper-V host.

Hoping this was just a fluke, i waited until it happened again – which it did, roughly 10 minutes later. So i decided to skip on the synthetic devices and go with emulated NICs and the tulip driver.

Everything came back up, but i couldn’t ping any devices on the eth0 VLAN from the start, but the other two interfaces worked.

After a few more tries, i arrived at a configuration that has now been stable for 4 hours and 26 minutes, which sounds good so far. For this, i configured a single synthetic NIC that i used as a replacement for the non-working eth0 and three tulip NICs (of which the first was unused).

There are other things that also worry me:

Every reboot of the Linux machine created the following event log entry on the Hyper-V host:

'LINUX' was reset because an unrecoverable error occurred on a virtual processor that caused a triple fault. If the problem persists, contact Product Support. (Virtual machine ID [])

Loading the synthetic NIC drivers logs the following in the event log on the Hyper-V host:

Networking driver on 'LINUX' loaded but has a different version from the server. Server version 3.2 Client version 0.2 (Virtual machine ID []). The device will work, but this is an unsupported configuration. This means that technical support will not be provided until this problem is resolved. To fix this problem, upgrade the integration services. To upgrade, connect to the virtual machine and select Insert Integration Services Setup Disk from the Action menu.

Loading the synthetic NIC drivers also logs all this on the Linux side of things:

VMBUS_DRV: Vmbus initializing.... current log level 0x1f1f0006 (1f1f,6)
VMBUS: +++++++ Build Date=Feb 17 2010 12:37:00 +++++++
VMBUS: +++++++ Build Description=Version 2.0 +++++++
VMBUS: +++++++ Vmbus supported version = 13 +++++++
VMBUS: +++++++ Vmbus using SINT 2 +++++++
VMBUS: Windows hypervisor detected! Retrieving more info...
VMBUS: Vendor ID: Microsoft Hv
VMBUS: Interface ID: Hv#1
VMBUS: OS Build:7600-6.1-16-0.16485
VMBUS: Hypercall page VA=f80c9000, PA=0x36afe000
VMBUS_DRV: irq 0x5 vector 0x35
VMBUS: SynIC version: 1
VMBUS: Vmbus connected!!
VMBUS_DRV: generating uevent - VMBUS_DEVICE_CLASS_GUID={c5295816-f63a-4d5f-8d1a4daf999ca185}
VMBUS: Channel offer notification - child relid 1 monitor id 0 allocated 1, type {32412632-86cb-44a2-9b5c50d1417354f5} instance {00000000-0000-8899-0000000000000000}
hv_netvsc: module is from the staging directory, the quality is unknown, you have been warned.
NETVSC_DRV: Netvsc initializing....
VMBUS_DRV: child driver (f80dc570) registering - name netvsc
VMBUS: Channel offer notification - child relid 2 monitor id 255 allocated 0, type {cfa8b69e-5b4a-4cc0-b98b8ba1a1f3f95a} instance {58f75a6d-d949-4320-99e1a2a2576d581c}
VMBUS_DRV: generating uevent - VMBUS_DEVICE_CLASS_GUID={32412632-86cb-44a2-9b5c50d1417354f5}
VMBUS_DRV: child device (f73a8634) registered
VMBUS: Channel offer notification - child relid 9 monitor id 1 allocated 1, type {f8615163-df3e-46c5-913ff2d2f965ed0e} instance {9d44a66e-4b09-41d5-80d807ae24bf537d}
VMBUS_DRV: generating uevent - VMBUS_DEVICE_CLASS_GUID={cfa8b69e-5b4a-4cc0-b98b8ba1a1f3f95a}
VMBUS_DRV: child device (f73a5a34) registered
VMBUS: Channel offer notification - child relid 1 monitor id 0 allocated 1, type {32412632-86cb-44a2-9b5c50d1417354f5} instance {00000000-0000-8899-0000000000000000}
VMBUS_DRV: generating uevent - VMBUS_DEVICE_CLASS_GUID={f8615163-df3e-46c5-913ff2d2f965ed0e}
VMBUS_DRV: device object (f73a5ee4) set to driver object (f80dc5c0)
VMBUS: Channel offer notification - child relid 2 monitor id 255 allocated 0, type {cfa8b69e-5b4a-4cc0-b98b8ba1a1f3f95a} instance {58f75a6d-d949-4320-99e1a2a2576d581c}
VMBUS: Channel offer notification - child relid 9 monitor id 1 allocated 1, type {f8615163-df3e-46c5-913ff2d2f965ed0e} instance {9d44a66e-4b09-41d5-80d807ae24bf537d}
VMBUS: channel f73aac00 open success!!
NETVSC: *** NetVSC channel opened successfully! ***
NETVSC: Sending NvspMessageTypeInit...
NETVSC: NvspMessageTypeInit status(1) max mdl chain (34)
NETVSC: Sending NvspMessage1TypeSendNdisVersion...
NETVSC: Establishing receive buffer's GPADL...
NETVSC: Sending NvspMessage1TypeSendReceiveBuffer...
NETVSC: Receive sections info (count 1, offset 0, endoffset 1048000, suballoc size 1600, num suballocs 655)
NETVSC: Establishing send buffer's GPADL...
NETVSC: Sending NvspMessage1TypeSendSendBuffer...
NETVSC: *** NetVSC channel handshake result - 0 ***
NETVSC: Device 0xf6552e80 mac addr 00155d031a09
NETVSC: Device 0xf6552e80 link state up
VMBUS_DRV: child device (f73a5e34) registered

So, it works. But not without troubles. I’ve still got the physical machine to fall back on, but i sure hope Microsoft will get this to work better.

These issues are the reason why i decided to deploy my private server using ESXi instead of Hyper-V – because i need both Linux and Windows guests.

I’ve setup a new Hyper-V server on a x3650 M2 (using server core) – i’ve also installed the latest Broadcom NetXtreme II drivers, all the firmware updates, all the best practices you do.

Setting up the machine, transferring VMs from another host (using BITS) worked well and fast, no issues.

And then i installed the DPM agent, started a backup. Two hours later, it was still stuck at “Replica creation in progress”.

I tried reading through the DPM agent logs, through the DPM server logs, looked if DPM created shadow copies (using vssadmin list shadows).

After two hours of fruitless searching (which included restarting everything), i wasn’t any further to a solution.

Well, backup wasn’t working right, but this was just a testing environment, so i decided to do other stuff.

A while later, i ran netstat -t to lookup connections – and also realized that TCP Chimney Offloading was still active. So i disabled it using netsh int tcp set global chimney=disabled. Just a few seconds later, the utilization of the management network adapter jumped to 100% and 5 minutes later, all the VMs were replicated to the DPM server.

So, if you’re having issues with DPM backups being stuck, check the status of your network offloading.

If you’re reading this, this blog is now hosted on Windows Server 2008 R2 Web Edition (Yay NFR promotions!). There may still be some kinks that have to be worked out, because this was quite a rush job. Leave a comment if you find any issues.

Our current cablecom hispeed business line is ADSL2+ with 20/2 megabits. While the upstream is too low for my taste, i haven’t really seen better offers.

I talked with a sales on the phone – for about 200 CHF more, we could get 20/2 SDSL (which sounded strange) and a 20/2 DOCSIS backup line, together with a “Bronze” level SLA. This sounded very attractive to me and i told the sales to send me the offer.

In the written offer, the ominous 20/2 SDSL was downgraded to 4/4 SDSL (which made much more sense). Of course, downgrading our internet connection from 20/2 to 4/4 seemed like a rather bad idea. We have about 30 people working here everyday, and almost all of them really use the internet to do their job. We’ve upgraded from 6/.6 ADSL to the current cablecom connection, because 6 megabit downstream wasn’t fast enough.

So i asked what else they could offer us – for 500 CHF more than we pay today, we could get 8/8 SDSL with a 20/2 DOCSIS backup. That still didn’t sound interesting to me.

I, personally, think 1000 CHF per month would be okay for a redundant 20/20 connection or something in this direction. My current connection at home is 25/2.5 – for 75 CHF a month. It works well enough, and the last failure i had was fixed in three days. Just like the failure we had on our 500 CHF per month 20/2 connection. This should be a telltale sign that something is very wrong with either the pricing or the service level.

The next question i asked if they could do a 20/2 ADSL with a 20/2 DOCSIS backup. Apparently, that’s not technically possible right now, but they might introduce this later this year. That sounds attractive to me.

All in all, i still think that cablecom hispeed business sucks. They can’t be bothered to do a 5 minute fix in a 2 hour time window on Friday evening. Then, they make one ludicrous offer that noone can take serious after the other.

I’m pretty sure that cablecom doesn’t really understand what small businesses need.

As a side note, if you work for an ISP and think you can make us a better offer than cablecom, i’d be very much interested. Send your stuff to l dot beeler at acommit dot ch. We will be moving to Horgen/ZH at Seestrasse 202 in March 2010 and need 32 static IP addresses.

Shortly after installing the line back in 2008, we’ve ran into an issue where cablecom hispeed business blocks GRE packets. After almost three days and speaking with a variety of technicians, they were finally able to resolve the issue.

Now, we’ve run into another, much more grave problem. Since about 15:45, a variety of hosts on the Internet aren’t reachable and of course several other hosts can’t reach us.

Of course this isn’t a clear-cut “my DSL modem has no link” issue – so cablecom currently isn’t even trying to fix the problem. I’ve been on the phone twice, never get any callbacks and don’t get any updates on the state of the problem resolution.

Fact is, some hosts can reach our OWA 2010 and some can’t. Nasty thing is, Swisscom’s GPRS/UMTS IP addresses can’t – this means no push-email for all 35 of our employees. Since we’re working for a rather important project (ERP and POS implementation) this weekend, this is a big issue for us.

It also looks interesting in a tcpdump – some packets just get lost – and from other hosts it works without any issues.

The 77. addresses are cablecom hispeed business, the 217. are my cablecom residential connection. In the first part, we see a TCP connection to port 80. In the second part, we see a ping -t. As you can see, there are a lot of dropped packets.

23:12:12.629457 IP 217.162.252.98.18417 > 77.59.216.227.80: S 4006182815:4006182815(0) win 8192 <mss 1460,nop,wscale 2,nop,nop,sackOK>
23:12:12.629479 IP 77.59.216.227.80 > 217.162.252.98.18417: S 1280362581:1280362581(0) ack 4006182816 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 6>
23:12:15.826736 IP 77.59.216.227.80 > 217.162.252.98.18417: S 1280362581:1280362581(0) ack 4006182816 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 6>
23:12:22.026734 IP 77.59.216.227.80 > 217.162.252.98.18417: S 1280362581:1280362581(0) ack 4006182816 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 6>
23:12:34.026733 IP 77.59.216.227.80 > 217.162.252.98.18417: S 1280362581:1280362581(0) ack 4006182816 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 6>

08:51:49.642995 IP 217.162.252.98 > 77.59.216.227: icmp 40: echo request seq 65
08:51:49.643024 IP 77.59.216.227 > 217.162.252.98: icmp 40: echo reply seq 65
08:52:00.641330 IP 217.162.252.98 > 77.59.216.227: icmp 40: echo request seq 68
08:52:00.641345 IP 77.59.216.227 > 217.162.252.98: icmp 40: echo reply seq 68
08:53:16.641813 IP 217.162.252.98 > 77.59.216.227: icmp 40: echo request seq 84
08:53:16.641829 IP 77.59.216.227 > 217.162.252.98: icmp 40: echo reply seq 84

Cablecom gets 180 CHF per month for 24/7 support. The case has now been open for 7 hours, with no resolution in sight. There’s no escalation path and there are no workarounds – we don’t have redundant connections.

Interestingly, one of our customers who also uses cablecom hispeed business had a similar issue, that lasted for roughly three weeks – one of their IP addresses wasn’t reachable externally, from one minute to the other. Unfortunately for us, all of our public IP addresses are affected by this issue, so we don’t have an easy workaround.

Of course, for some part we’re also to blame. Luckily i’m not one of the higher ups who gambled with non-redundant internet connections and lost.

Have you made negative experiences with cablecom hispeed business? Positive ones? Was support able to fix your issues quickly?

Update: I’ve called cablecom again on Saturday at 09:00. Apparently, these sort of issues are supported on a best-effort base from 9 to 5, and not covered by our 24/7 support contract. We will have to wait until monday – they will not look at this issue further during the weekend.

Update: Monday morning, 11:00. Problem is still unsolved.

--- hor-fw-01.acommit.ch ping statistics ---
20 packets transmitted, 3 received, 85% packet loss, time 19012ms
rtt min/avg/max/mdev = 20.490/21.360/22.585/0.891 ms

Update: Monday morning, 11:36. Problem is now solved. According to the Tech i talked to, the he fixed the issue in 5 minutes. He could’ve done that on Friday, but apparently noone at cablecom felt like doing so. The issue was that cablecom configured our new line for the planned office move and configured load sharing between the new line for the new office and the old line. Since the new line didn’t physically exist yet, half of the packets were dropped.

Thanks to the Tech who fixed the issue, no thanks to cablecom in general for wasting an entire weekend on what could’ve been a five minute fix on Friday evening.

Currently, we’re looking for:

Project Manager (PDF)
Sales (PDF)

Sure, there are loads of MVPs and TAP-Members that have migrated to Exchange 2010 a long time ago, but i’m still proud of this.

At a starting point, i had a Exchange 2007 SP2 machine, with one Mailbox database, no public folders and 35 Mailboxes that used up 25GB of space. Moving this is simple enough, but the issue is that our Exchange isn’t virtualized, and i couldn’t get my hands on new hardware since the current box was only a year old.

Since in-place upgrades are not supported, i needed a temporary server for the migration. I used an HP ML110 from the Lab, which offered enough space to migrate.

Another issue was BackupExec 12.5, which did not support Exchange 2010 yet. Fortunately, Exchange 2010 (and 2007 SP2) can be backed up by using Windows Server Backup. So my goal was to just let WSB backup to a file server, and have BackupExec pickup the files from there. This way, i will get a reliable, clean and supported Exchange backup, and still have it on tape.

To Migration itself was straightforward and easy. There’s already _lots_ of content on the web about Exchange 2010, most of it from the RCs or Beta of course.

I followed the Migration Guide from TechNet, which worked out well enough. Unfortunately, the iPhone does not support Exchange 2010/2007 coexistence, which made it necessary for several people to manually reconfigure their phone.

Removing Exchange 2007 worked without issues, but after moving all the Exchange 2010 data back to the real hardware and removing the temporary server i ran into the issue of moving arbitration mailboxes, which fortunately was already documented widely on the web.

In the end, upgrading from Exchange 2007 to 2010 while keeping the same hardware is not difficult, it just needs a bit more time.

The error message in the IIS Log looks like this:

RdirTo:https%3a%2f%2flegacy.contoso.com%2fMicrosoft-Server-ActiveSync_LdapC2_LdapL15_Error:MisconfiguredDevice_Budget

For the benchmarks, i’ve used the free version of HDtune. I’ve benchmarked four systems, and five different disk configurations. Note that the free version only does benchmarks for disk reads, and it’s a not a very pervasive test. None of these benchmarks are scientific. They should serve as a general indicator of performance, not as a final world on this topic. I don’t have that much clue about benchmarking.

The first system is my computer at home: It has an i7-920 CPU at stock speed, with 3x2GB RAM at 1333 Mhz (which is a slight overclock, but within the spec of the memory i purchased). Attached to it’s ICH10R controller are an Intel X25-M G2 160GB (Firmware 02HA) and a WD1001FALS (1TB, 7×24), running Windows 7 x64.

The next system is my work laptop, which is a ThinkPad W500 with a 2.53 Ghz T9400 C2D CPU, with 4GB of RAM. Attached to it’s onboard controller is an OCZ Vertex 120GB (Firmware 1.40), running Windows 7 x64.

The third system is our Exchange Edge server, on which i dared to install a benchmark utility. It’s an IBM x3250 with two 70GB 15kRPM 2.5″ SAS drives installed, attached to an onboard LSI1064E SAS controller. The system has a Xeon 3040 2.4Ghz Dualcore CPU and 5 GB RAM. It is running Windows Server 2008 x64 SP2.

And the final system is a HP ML110 G5 with a 2.33 Ghz Xeon 3065 CPU, 8GB of RAM and a E200 with the latest firmware (1.78). Attached to that are 4 WD1001FALS drives in a RAID10 configuration. The E200 has a backup battery and 128MB of cache installed. The system is running Windows Server 2008 R2.

Please note that none of these benchmarks are scientific. They were done on real systems, with workload minimized as much as possible, but virus scanners and other mandatory background applications active. Both the laptop and the desktop have not been formatted since Windows 7 RC was installed (i migrated to Windows 7 RTM using Windows.old), but the ML110 was freshly setup and the only application that’s been installed so far is the HP ACU and Forefront Client Security. The Exchange Edge server has been in use since May 2008. As such, the ML110 is the “cleanest” machine out of these four.

Intel’s X25-M G2 160GB on an ICH10R (AHCI Mode)

This is how a graph should look. It’s nice, it’s clean, it’s fast. Intel’s X25-M G2 shows how a modern SSD and storage subsystem should behave. Clean, predictable performance.

OCZ’s Vertex 160GB on an ICH7 (AHCI Mode)

Here’s the OCZ Vertex. It’s running on a machine that’s a lot slower than the one the X25-M is attached to, and it’s storage controller is also quite a bit older. It still shows remarkably good performance. It should also be considered that this Vertex is quite a bit older – it was bought in May 09. It’s still very fast and responsive and a good SSD.

2x IBM’s 73GB 15kRPM 2.5″ SAS Disks on an LSI Logic 1064E SAS Controller

As you can see, this is the performance you get from the server hard disks on an entry-level controller in an entry-level system. It’s not astonishing, but the performance is very well acceptable.

Western Digital’s 1001FALS 1TB on an ICH10R (AHCI Mode)

Here’s how the Western Digital disk behaves on a proper controller. Please note that this is a single disk, not part of a RAID array. The performance is quite good.

4x WD’s 1001FALS 1TB on an HP E200 in RAID10

And here’s how it shouldn’t look. Compare this to the stand-alone disks above, which exhibits better performance. HP fucked up bad on this one, and there’s no fix in sight. Stay away from the E200.

And as a final word: I really don’t have much of a clue about benchmarking. If you see an obvious error here, please state what you think. If possible, i will try to correct it.

Update: As requested in the comments, i upgraded the E200 to Firmware 1.84 and redid the benchmark. It looks roughly the same.

If you’re following the recommended deployment method for Exchange 2007, you’ll already be using a SAN certificate in order to publish AutoDiscovery and OWA. For coexistence of Exchange 2007 and Exchange 2010, an additional name will need to be added to your SAN certificate.

With most CAs, this is a pretty straightforward process that can be done using their web interface, since the private key doesn’t need to be touched. After modifying this, you will get a new .crt file containing the certificate, but no private key (which is correct).

However, importing this into Exchange 2007 using Import-ExchangeCertificate doesn’t work – Windows won’t know which private key is associated with the newly imported certificate. When you try to use Enable-ExchangeCertificate, you will receive the following error message:

Enable-ExchangeCertificate : The certificate with thumbprint 1234 was found but is
not valid for use with Exchange Server (reason: PrivateKeyMissing).

I searched high and low on how to replace a certificate without touching the private key, but i didn’t find anything. So i turned to the community for support – MCSEBoard.de is an excellent Windows community for those who speak German.

Unfortunately, noone knew an easy way either – the suggestion was to use OpenSSL to create a new keystore.

This was rather easy, but i didn’t find any guides on the net on how to do this, so i’m publishing this here in the hope that it will help others with the same issue.

• First, you need to export the key including the private key using the Windows certificate manager. Open an elevated MMC, add the Certificate snap-in and focus on the Computer certificate. Click “Personal”, and then export the certificate with the private key.
• Download and Install OpenSSL for Windows
• Issue the following command: openssl pkcs12 -in mykey.pfx > out.txt
• Open out.txt using an LF-aware text editor, such as Notepad++. Save the PRIVATE KEY part to a textfile called key.pem.
• Save the certificate to a file called cert.crt
• Issue to the following command: openssl pkcs12 -export -in cert.crt -inkey key.pem -out newcert.p12
• Copy the newly created newcert.p12 to the Exchange server.
• Open PowerShell and run the following command: $secureString = ConvertTo-SecureString "blubb" -AsPlainText -Force – Replace “blubb” with the Passphrase you used in the step before
• Run Import-ExchangeCertificate -path newcert.p12 -pass $secureString to import the certificate back into Exchange
• The rest is as usual – use Enable-ExchangeCertificate to enable the certificate.

And that’s it. It might be a bit cumbersome – and i really hope that there is an easier way to to this. If you know, let me know so i can update this page.

See here for the fix and it’s description KB974571

Click here to download the ocsasnfix.exe directly, which will fix the incorrect ASN License data – something which i already guessed about in my previous post about this issue.

During this time, i’ve gained valuable experience, which i’ll try to share here so that others can profit from it. Take all this with a grain of salt, as some observations may simply be my fault. Also, as times changes these things might change too.

Software

• Make sure to install Windows Server 2008 SP2 after installing SBS 2008. Some media may come with SP2 already preloaded. You can use the normal SP2 package that’s also used for Vista and the normal Server 2008
• Do not install SBS rollup updates before completing the configuration wizard. This is extremely counter-intuitive, but is described on the Official SBS blog
• Installing Exchange 2007 SP2 requires you to follow special considerations Here
• Installing WSUS 3.0 SP2, which is needed to support Windows 7, is currently not recommended. I was able to do this without issues on my lab machines, but others have reported issues doing this on machines that were in production. If you’re deploying a new SBS server, this should probably be safe to go. But make sure to test functionality afterward.
• Always use the answer file to deploy SBS 2008. This will make it possible to choose a custom domain name. Read my post about choosing your AD DNS namespace
• Do whatever tasks you can do using the SBS console. Resist of using the normal administration tools as much as possible, as you can break SBS with them easily.
• Ensure that the AV software you install is compatible with WS08 x64. Symantec Endpoint Protection Manager works well – Forefront Client Security on the other hand requires a seperate server running 32bit Windows for management. You may consider deploying FCS unmanaged in smaller environments, and configure FCS using the FCS ADM File

Hardware

• Use servers with the new Xeon 5500 CPUs. Read my x3650 M2 tips to find more about them. Consider using an E5530 or faster CPU. Using two CPUs (for a total of 16 virtual and 8 physical cores) makes little sense.
• Buy enough memory. Lots of it. Really. I mean it. You’ll need lots and lots of memory. I would consider 12GB to bare minimum. In a 3x4GB configuration which makes the most sense for the Xeon 5500 setups, this is quite cheap. Consider more memory if you intend to run SQL Server as, consider bumping the memory to 24GB. Remember that you can only use the first 8 slots in a single socket machine.
• Buy enough disks. A good starting layout is 8x147GB 2.5″ disks. Use a RAID 1 for the OS, another RAID1 for Exchange and Sharepoint, and a RAID10 for Data and WSUS. This is all up for debate of course, and it might make sense to consider other disk layouts.

If you have any additions, think i’m wrong somewhere just send in a comment.