On the operation of massively multiplayer online games.
RSS icon Home icon
  • Data Center Pictures

    Posted on April 5th, 2009 Bryant

    Without preamble:

    Rows and rows of closed cabinets.

    Rows and rows of closed cabinets.

    That’s probably a bunch of cabinets that are being rented out individually to different customers. If so, they’re lockable. In my mind, enclosed units like these are cabinets while open units are racks, but the terms get used interchangeably. Each one of these contains the standard 42 units of available vertical space; 1U is 1.75 inches. They’re 19″ wide. The standard for rack design goes back to railway signaling, and is used in telecom, electronics, audio, etc., etc.

    You can see a cable run going above the cabinets at the top left center. Cables over the top is fairly standard. Some people still do cabling under the floor — the old Sun corporate headquarters was like that — but generally that’s reserved for power these days. It’s a pain to do any cable work when you have to go under the tiles to do it. My group wasn’t responsible for server cabling or maintenance, which was a relief.

    Each one of those cabinets probably has a gigabit Ethernet cable or two dropping into it to provide bandwidth.

    Our hunched over tech ops guy is working at a crash cart; the data center provides these so you can hook a monitor and keyboard up to a server for diagnostic purposes. The etymology is fairly obvious. Vendors sell these nifty flat screen monitors that are designed to fit into a rack, and slide out of the way when you’re not using them, but at a thousand bucks per monitor they’re a little pricy.

    Private cage space.

    Private cage space.

    This is the other way data center space is sold: in chunks. We call these cages, because they’re caged in — see the chain link fence surrounding this space? The cage itself is locked, but the racks within don’t need to be. Sometimes closed cabinets are necessary for cooling; I find we geeks tend to like to expose our servers for easy access whenever possible.

    Those tiles on the floor lift up easily to expose the crawlspace beneath. There are lots of power cables there. Some of the tiles are perforated; that’s for cooling. There is serious math behind both the design of individual perforated tiles and the layout of perforated and non-perforated tiles within a data center, which is both neat and one of many reasons I never want to host servers myself. In practice sometimes the data center gets it wrong, alas, so I have to know something about it. Or at least be able to detect uncertainty on the part of my vendors.

    The little servers in rack 002.005, one in from the left, are 1U servers. It looks like someone spaced them out for the sake of making the rack look fuller or for cooling. I’d stack ’em right up next to each other like the five 1U servers you can see down on the right, in rack 002.001 — the cooling is generally better that way.

    These days, the amount of physical floor space you buy is determined by how much power you need rather than how many racks you want in your cage. The last time I bought data center space, I wound up deciding that we were going to buy enough power to fill our racks halfway full of 1U servers; that was the most cost effective model given pricing. That might be one reason to spread your servers out like these guys, so it doesn’t look like you’re wasting space, but I don’t know. Then again, I wasn’t there, so I’ll quit speculating.

    That far right rack has all the network devices in it. You can tell because there’s a little nest of cabling there. The third device from the bottom in the far left rack with all the vertical panels is a storage unit. Each one of those panels can contain a hard drive. That might be a tape backup unit right above it; I’m not sure. Lots of nice IBM hardware, though.

    I do spend a certain amount of time peering curiously at other cages when I’m in a data center, trying to figure out what everyone else’s setup is like, yes. I’ve shared data center space with Blizzard. That’s a cool setup.

    A rather nice bit of cabling work.

    A rather nice bit of cabling work.

    Here, we’re looking at a stack of Cisco Catalyst switches, which are real workhorses. Those cables are running off to individual servers, while the switches are probably connected to a router in the same rack. The cables all run downward, which makes me think that this particular installation runs cable under the floor, so there you go.

    All the cables run through cable guides, which prevents them from turning into disorganized spaghetti. They’re also all labeled, so you can look at the cables here and know which one is going to which server without tracing it through the floor to the end. They’re color-coded for good measure: if you’re at a server, you know by looking at the cables which segment or segments of the network that server is on.

    A not so awesome cabling job.

    A not so awesome cabling job.

    This cabling is sub-par. Sorry to whoever took the photo! It’s good that everything’s labelled, but the cables aren’t running through any guides, which means they’re going to get tangled up. Also, the labels are too big and I suspect they’ll get in the way. The idea of having removable labels is good, because it makes it easier to update, mind you. This isn’t tragic cabling, it’s just not great.

    A real mess.

    A real mess.

    That’s really bad cabling. Ow.

    Oh, and yeah, I’d said something about a mildly amusing story. Welp, I found all these photos via Flickr’s Creative Commons search, which means I’m sure it’s OK to reproduce them here. It turns out that people aren’t always so careful about rights. Back at AltaVista, we originally kept our servers in this fine building, inches away from the most desirable retail space in Palo Alto. Not entirely cost-effective. Sometime thereafter, we moved most of our servers to a Level 3 facility, which was generally pretty good.

    We had a lot of servers for the time, and because AltaVista was still tightly coupled with Compaq at that point, we had nice new ones. Our racking and stacking was, modestly, top-notch. Our cage in the Level 3 data center was very pretty. Pretty enough so that Level 3 decided to photograph it and use it on the cover of one of their stockholder reports without asking us. Our sales rep was fairly embarrassed.

     

    5 responses to “Data Center Pictures”

    1. Rick J

      I have seen cable rat nests worse than that one.

      In fact, I had to untangle a cable rat nest worse than that one.

    2. Bryant

      That one isn’t too bad. But it would horrify me to find it in a rack I owned. I couldn’t find anything really awful under Creative Commons…

      http://www.vibrant.com/cable-messes.php has a bunch of ungodly ones.

    3. flit

      Someone you know tried to hire me at a place with a cable mess worse than any of the ungodly ones! I guess I would have had days of fun if I’d taken him up on it. 😉

    4. Hi~
      I was wondering if I could use (or if you have it somewhere for purchase) the top image of the IT admin in the datacenter located: http://cogs.innocence.com/2009/04/data-center-pictures/

      thanks!!
      peter

    5. Bryant

      Hey, Peter — it’s not mine; it’s from this URL: http://www.flickr.com/photos/sterlingely/1418364/

      It’s under a Creative Commons license, but it looks like it’s only available for non-commercial use, so if it’s for F5 I can’t help you. Sorry!

      (But I love your load balancers.)

    Leave a reply