Earlier today, we started a pretty bold move from one data centre to a brand-spanking new one across the city. I say bold, because we had to take ourselves completely offline while we disconnected more than thirty bits of hardware, shipped it across the city, and then reassembled it all.
I didn’t take very many shots from old data center; just this one of Webmaster Matt behind a bunch of cables.
Denis coordinated the whole thing from an impressively comprehensive plan assembled over weeks of careful analysis, multiple mental walk-throughs, and–no doubt–many sleepless nights. Here’s a shot of Denis piecing some of our systems together in the sparkly-clean new data centre.
Chris was a big help. In this shot, Chris is installing the the back half of a rail in for one of the servers. I was able to take the picture because I did all the front screws which are way easier and less time consuming than the ones in the back.
I managed to take this next shot in a rare moment of inactivity from Mike (seriously, the guy’s a machine) who had stepped back for a minute to give Denis some room to connect a few systems into the switch. We actually managed a very high level of parallel activity despite the tight space. I credit Denis’ plan and the well-laid out documents and diagrams which allowed us to get done what he needed without bombarding him with questions.
By early afternoon, it was all coming together nicely. I think it was at this point that Denis said something to the effect of “our backends are up”. This made me giggle. I guess that deep down we never get any older than twelve.
The data centre is still relatively new and so there’s still some empty space (which was good because being able to spread the servers around a bit made things a lot easier). I was tempted to use the space to demonstrate some of my sweet Karate moves. But everybody was busy doing actual work and wouldn’t have been able to fully enjoy the experience.
Our other Chris–the guy who writes the cheques–joined in as well. In this shot, our two Chrises are assembling various bits of hardware while Denis wrestles the systems to life. As as aside, I quite like the portable workstands we used; they’re easy to wheel around, they’re very stable, and they’re height-adjustable to boot. I need to get one of these for home.
By late afternoon, we’d installed every single rail and shelf, and every bit of hardware was in its place. When I left, Mike and Chris were following Denis’ cabling plan while Denis and Matt bent the operating systems to their will. The speed and confidence that Matt and Denis demonstrate while elbows-deep in Linux configuration files is awesome. And humbling.
It was a great day. Frankly, I love playing with hardware. But I had to leave. Minor hockey needed me…
Denis reported late this evening that all was progressing according to plan. Many of our servers are up and running again and the rest should be taken care of tomorrow. I’m not 100% happy that the Foundation’s email server is back up, though…
