Load balancer upgrade scheduled Wednesday, Dec 8, 2am GMT for up to one hour

By on December 6, 2010

Bitbucket will be unavailable for up to one hour starting Wednesday, Dec 8th, 02:00 GMT while we upgrade our application architecture for high-availability and fail-over. We expect the downtime not to exceed one hour.

During this maintenance window we’ll deploy two new HAProxy load balancers clustered with Heartbeat. As it stands now we’re load balancing with Apache’s mod_rewrite RewriteMap feature. Repositories are statically distributed across our application servers. If your assigned server has any critical failures, we’re unable to redirect traffic to another server. Luckily that hasn’t happened often since migrating Bitbucket from EC2 to Contegix. With the new application architecture repositories will no longer be bound to a specific server. Should one of our application servers crash, one of the others in the load balancer pool will pick up the slack.

If you check out the footer on any Bitbucket repository page you can see it’s bound to a particular server:

After this change our new load balancers will pass requests to our application servers with the fewest connections and least load instead of a static mapping. In the footer you’ll see different servers handling your repository requests. An added benefit we’re looking forward to is the ability to remove our application servers from the load balancer pool without taking your repositories offline. Overall this will make Bitbucket more reliable and ensure less scheduled maintenance and downtime.

11 Comments

  • Posted December 6, 2010 at 3:22 am | Permalink

    (The image isn't visible here – the URL points to a login page.)

    Hooray for less downtime!

  • Posted December 6, 2010 at 11:22 am | Permalink

    (The image isn’t visible here – the URL points to a login page.)

    Hooray for less downtime!

  • Posted December 6, 2010 at 11:22 am | Permalink

    (The image isn’t visible here – the URL points to a login page.)

    Hooray for less downtime!

  • Posted December 8, 2010 at 12:42 am | Permalink

    <some sitename here>.bitbucket.org – 404

  • Posted December 8, 2010 at 8:42 am | Permalink

    <some sitename here>.bitbucket.org – 404

  • Posted December 8, 2010 at 8:42 am | Permalink

    <some sitename here>.bitbucket.org – 404

    • Posted December 8, 2010 at 8:59 am | Permalink
      • Charles McLaughlin
        Posted December 8, 2010 at 11:03 am | Permalink

        Thanks for pointing this out. We put a temporary redirect in place:

        your_login.bitbucket.org will redirect to bitbucket.org/hosted/your_login

        We’ll restore the true functionality ASAP.

        • Charles McLaughlin
          Posted December 9, 2010 at 2:38 am | Permalink

          This was fixed this morning.

  • Shrawan Patel
    Posted June 8, 2011 at 11:17 am | Permalink

    This post excellently highlights what the author is trying to communicate. Nonetheless, the article has been framed excellently well and all credits to the author. For more information on how to load balance your web servers, please visit ..nhttp://serverloadbalancing.biz/wordpressbiz/, nhttp://serverloadbalancing.info/wordpressinfo/

  • Posted February 8, 2012 at 6:49 am | Permalink

    Thanks for informing us on this! I’ll use the redirect then. Can’t wait to start using this update! It’s most likely to become a much better tool.

    Convert m4a to mp3 http://m4atomp3converter.org