Project NipTuck: Redesigning Bitbucket with a Flick of a Switch

By on November 7, 2012

On October 9, we released a site wide redesign of Bitbucket. The release was the culmination of 4 months of effort, in which we simultaneously ran two versions of Bitbucket side-by-side for a select group of beta users as we converted each page to the new look and feel now found on Bitbucket. At the end of 4 months, we were able to roll an entirely new version of the site by literally turning on a simple flag for all users. No deploy process; no build scripts; no hassles.

Laying the groundwork

Before we started implementing, we laid out some constraints on what we wanted to change:

  1. We’re not going dark: Rather than have the whole team stop everything to work solely on this project, we wanted to still be able to work on the existing design and make changes and improvements.
  2. Reskin, not reconstruct: We wanted to retain the functionality and existing feature set of Bitbucket as much as possible. So, for the most part, we only rebuilt the HTML/CSS for each existing page.
  3. Fail gracefully: Since we wanted to do some alpha testing of the site as early as possible, we needed to present the new view to some users even though we weren’t necessarily finished converting the website. So, if a user viewed a page that hadn’t been redesigned yet, the site would fallback to the old view.
  4. Clean up after yourselves: As is the case with many projects, when you rewrite something, the existing code becomes intertwined with your new code. By the end, you just have a mess of duplicated files that you’re not quite sure you can delete. We wanted to make sure that we implemented a proper system for removing the old code after we released.

Moving all the code

To start our redesign, we moved all the existing HTML templates into a directory called old. We didn’t just blindly move the entire templates directory into a different folder though. Instead, the top-level directory and each first-level subdirectory had its contents moved into the ‘old’ subdirectory. For example, templates/base.html moved to templates/old/base.html and templates/account/email.html moved to templates/account/old/email.html.

Why make it so complicated? One of the main reasons we did it this way was because a majority of the team was still working on the existing website. By only moving subdirectories, our developers were more easily able to see which templates had been reworked and might need to have a change incorporated.

Along with moving the templates, we also built a small Django app for handling the two sites.

Using django-waffle, we wrote a wrapper for Django’s render_to_response function. In pseudocode, the wrapper would:

  1. Check if the requesting user had the ‘niptuck’ flag enabled.
  2. If the flag was enabled, it would then search for the template
  3. If found, we would render the new template.
  4. If not, we would reconstruct the template URL to search for the old template and render the old view.

With the render wrapper in place, we were able to convert each template without affecting the live site. We were also able to fail gracefully as we turned on the flag for ourselves.

This process enabled us to convert views without conflicting with other streams of work.

What about the CSS and JavaScript?

We moved the CSS and JavaScript in the same fashion as our HTML templates. Since we were rewriting the CSS as part of the redesign, there wasn’t much to do apart from moving the CSS and updating the script tags.

The JavaScript was a different matter though. Our JavaScript had accumulated over the years and had grown to the point where it was much like our python backend — too big to change. To resolve this, we created a ‘shared’ directory, in addition to moving our JavaScript files to an ‘old’ directory. The ‘shared’ directory would include all our JavaScript files that were able to run on both the existing and the new site. The theory was that we would try and keep the ‘shared’ directory as large as possible and only create new files if they were absolutely necessary.

In practice however, it turned out that this was not a great solution. In almost all cases, trying to maintain two different views with the same file was just too unwieldy. If done again, I would suggest copying the JavaScript to a different location, even if only minor changes are made.

Caveats and exceptions

What’s a good system without a few pitfalls and exceptions? We encountered a few minor issues when we implemented this redesign, mostly related to views and templates provided by third party apps.

Third party views

In Bitbucket, we use Django’s built-in user authentication, as well as the Social Registration app. Both of these apps have their own views that are given the location of the template in their parameters. Because of this behavior, we are not able to replace the call to the render method like we do with our custom views.

To get around this, we wrote a view wrapper that would wrap the execution of the view with our own and manipulate the provided template name.

Default templates

These third party apps also present additional issues since they provide default templates. Therefore, our check for the template’s existence always resolves to ‘True’. This problem only occurred if the template had not yet been converted AND if the user has the flag turned on. Since it only affected our alpha testers, we decided that this issue was a worthwhile compromise.

Feedback and iterations

You never get things right the first time, so we also had a solid process in place for feedback and redesign. We started releasing the new look to the rest of Atlassian’s staff in August, months before the official release in October. This meant that we received large amounts of feedback from users who interacted with Bitbucket on a daily basis. We also ran Friday afternoon demos to gather feedback from the team about new changes made during the week.

With all this data, we could identify which views needed work and iterate on them again. Most of our pages went through multiple incarnations and in some extreme cases were removed entirely. All this occurred without ever affecting the old site.

Deploying the new look

By using this system, we not only were able to have both designs run concurrently, but it also made the deployment process seamless. To deploy the new look, we simply needed to turn on the waffle flag to activate the new view for all users. There was no need for a deployment script, which in many cases could be flaky.

After the deployment, we then monitored our Sentry platform in order to find any errors. We also watched our support, issues, and social media channels to catch any UX issues that we may have missed.

The code

The templating code is now available for open-source use at bnguyen/niptuck-templating. You should be able to deploy it directly into your Django app.

  • http://twitter.com/briangonzalez__ Brian Gonzalez

    How did you deal with inline javascript (anti-pattern, I know) issues, if any?

    • Anonymous

      In the majority of cases we were able to move old legacy inline js into a proper file structure. It did require us to rewrite some sections of code (i.e. diff rendering), but it never turned into a major issue.

      For the cases where we couldn’t do that we just moved all the inline js into one file where we could manage it better. We ended up with only 35 lines of inline js so it worked out reasonably well.

      • http://twitter.com/briangonzalez__ Brian Gonzalez

        Wow, kudos to your past engineers, right?

        • Anonymous

          Yeah, definitely.

  • Pingback: How We Redesigned Bitbucket without a Deploy Process or Build Script | My Daily Feeds

  • Pingback: Innovation week launches several new features | Bitbucket Blog