It’s no secret anymore that Git is gaining traction within professional teams. 33% of respondents from a Forrester Enterprise Study last year indicated that 60% or more of their source code is managed by Git-based systems. Git surpassed Subversion for the first time in the 2014 Eclipse Community Survey. Git is becoming increasingly popular because of its easy branching model, flexible workflows, and distributed architecture.
At Atlassian, we’re committed to supporting professional teams making the switch to Git with Bitbucket. So, we’re announcing new capabilities today that will be available soon to help you use Git at massive scale:
Git mirroring to increase performance for distributed team members
Git large file storage support to allow collaboration on all file types of any size
Projects to keep your repositories organized
Build status for tighter integration between Bitbucket and Bamboo or any other Continuous Integration vendor
Our goal is to make it easier for professional teams to collaborate and deliver software faster. We’ve already added active-active clustering to ensure continuous uptime for source code management systems, a free business-ready product for small teams, and the first marketplace that allows for the discovery and distribution of third-party add-ons.
Organizations of all sizes from large enterprises such as Samsung, Splunk, Netflix, and Barclays Capital to small startups like Pinger, Metromile, and Kaazing are using Bitbucket today. Our JIRA Cloud customers picked Bitbucket as their #1 Git solution. One in three Fortune 500 companies trust Bitbucket and are using it everyday to build and ship software.
We hope you do as well.
Bitbucket is built for professional teams
Git was not originally designed for professional teams that are agile, distributed, and need secure and extensible workflows. Bitbucket makes it easier for a professional team to use Git:
Features like pull requests, inline comments, and permissioned workflows make it easy for teams to collaborate.
It’s the only Git solution that massively scales with active-active clustering, mirroring, and flexible deployment models.
Teams have access to integrations with the tools they most use, such as JIRA, which provides traceability across issues and source. Also, developers can create custom functionality or install a rich set of third-party add-ons available in the Atlassian Marketplace.
For larger organizations who need Premier Support and Strategic Services to get the most out of Bitbucket, we have already added Atlassian Enterprise.
Bitbucket: a unified brand for professional teams
To make it easier for you to find a collaborative code management solution that best meets your needs, we’ve unified our Git products under the Bitbucket name. With Bitbucket, you now have a range of options that can be adopted by teams of all sizes and requirements: Bitbucket Cloud (previously known as Bitbucket), Bitbucket Server (previously known as Stash) and Bitbucket Data Center (previously known as Stash Data Center).
Get started with Bitbucket: Git your way
We have a solution for teams of all sizes and needs – collaborate on code either self-managed or in the cloud, use Git via command-line or SourceTree. If you’re new to Git – head over to “Getting Git right” or if you’ve already made a decision to switch to Git, Try Bitbucket today.
Note: If you’re an existing customer of Stash or Bitbucket and have more questions, please visit the Bitbucket Rebrand FAQ.
Two-step verification secures your account by requiring a second component, in addition to your password, to access your account. That second step means your account stays secure even if your password is compromised. Bitbucket’s two-step verification implementation is built on the Time-based One-time Password Algorithm (TOTP), to ensure compatibility with mobile apps like Authy or Google Authenticator.
You will find the two-step verification (optional for you to use) in your Bitbucket account settings, which will take you through the onboarding process. More information can be found here.
The Bitbucket team has been hard at work adding features to make your day more productive and Bitbucket even more wonderful to use.
Et voilà… Snippets API, wiki edit restrictions, and relative links in READMEs and Snippets.
Snippets becomes even more of a first-class citizen among Bitbucket resources with the addition of APIs to create, rename, update, and delete Snippets — and more. See documentation here. We just made it easier for you to update your snippets from any third party application.
Wiki edit restrictions
Wiki editing is now restricted by default to those users with write permissions to the parent repository to prevent spamming on wikis. There is a new setting in the repository admin to toggle between this behavior and the previous behavior, where all users with any repo access could edit the wiki.
Relative links in READMEs and Snippets
A fix, really. This answers the very popular issue (#6513) on our public issues board.
Now you can use relative links in READMEs for both files and images. This also makes it possible to link to files from issues, changesets, and pull requests. Images can also be embedded in Snippets. Both “.” and “..” path notations are supported, and you can now start a link with “/” to make it relative to the repository root.
We owe you an explanation for the recent service interruption. On Wednesday, our users were unable to push or pull over SSH or HTTPS, access the GUI or API, or complete important tasks on time. We violated the cardinal rule of SaaS – “don’t have downtime” – and we’re very sorry for all the trouble we’ve caused. We know that you rely on Bitbucket for your daily work and when our service isn’t working correctly it affects your productivity.
I’d like to take a little time to explain what happened, how we handled it, and what are we doing to avoid a repeat of the situation in the future.
What actually happened?
At 11:50 UTC, Wednesday, September 2nd, we noticed a drop in the webhook processing rate. Our Site Reliability Engineers (SREs) escalated the issue to the Bitbucket team. We noticed at the same time the load balancer queues for every other service – SSH, Git over HTTPS, Mercurial over HTTPS, the API, the GUI – began growing very, very quickly. By 12:10 UTC, all services began failing for users.
Response times soon skyrocketed for Git over HTTPS.
Initially, the issues seemed unrelated. Application nodes all showed a reasonable load for that time of the day, and our RabbitMQ processes had not thrown any alerts. As our investigation later revealed, we had experienced severe resource exhaustion on the cluster serving both Redis and RabbitMQ, and the kernel’s out-of-memory killer began terminating both sets of services. This led to a split brain in RabbitMQ, as individual RabbitMQ instances were unable to communicate with each other and select a consistent master. Additionally, the application nodes’ Celery workers were unable to publish their background tasks consistently to RabbitMQ, thus blocking normal processing. With application processes blocked, the load balancers’ queues backed up, and soon the HAProxy daemon handling SSH traffic crashed outright.
By 12:30 UTC, we had focused our attention on the load balancer queues. Restarting haproxy-ssh did not work at first, as the kernel’s SYN backlog was still full, but once enough broken connection requests had timed out we’re able to restore haproxy-ssh at 12:47 UTC. By that point, RabbitMQ had stabilized to the point that Celery workers were once again able to publish. Our backlog of connection requests started to come down between the load balancer fix and the resumption of Celery workflow.
With traffic moving again, we started looking for a root cause. There were a few false leads: network changes, software deployments, back-end storage, and new RabbitMQ/Redis hardware. By 14:00 UTC, we came to a consensus that RabbitMQ’s failure was the reason we had seen so many problems. Unfortunately, Celery consumers were still unable to connect reliably to the queues, so the queues began growing very, very quickly, adding roughly 250,000 new tasks from 13:31 UTC to 13:59 UTC.
We spent the next several hours attempting to get Celery consuming again. However, the sheer size of the task queues, combined with the backlog of other services and our normal level of traffic for a Wednesday, meant slow going. At several points, consumer workers lost their ability to communicate effectively with their master processes. This kept the queues from being consumed consistently – workers would take a few tasks, run them, then die, all within the space of a few minutes.
Once Celery was fixed, it began handling a sizable message backlog.
By 19:27 UTC, we had managed to troubleshoot Celery enough to get it to consuming tasks consistently, though slowly. We still had two million backlogged tasks to process, and even with a relatively quiet back end that takes some time. By 21:00 UTC, we had managed to reduce the backlog to about 3500 tasks, and we resolved the incident as soon as we were confident the backlog wouldn’t go back up.
What are we doing to keep this from happening again?
There are a few actions we’re taking immediately:
We’ve migrated Redis to new hardware, separate from RabbitMQ, so that we can avoid resource contentions like this one in the future.
We’ve corrected a flaw in a script to truncate old keys in Redis, which could have helped us notice the Redis cluster problems sooner. This script has already purged about 100,000 dead keys, and it’s still going.
Our developers are re-evaluating their usage of Redis and RabbitMQ, and they are preparing adjustments that should reduce unnecessary traffic to either cluster.
Last but not least, the outage exposed some holes in our monitoring, especially centered around RabbitMQ and Celery. We’re upgrading our monitors to improve problem detection, and we’re rewriting SRE runbooks so that there’s a much clearer plan of action.
Thank you for using our service. We know that many of you rely on it. We truly strive every hour of every day to keep Bitbucket running reliably, so you can get stuff done and create amazing software.
We introduced a new way to manage teams in 2011 called groups. Groups allow you to administer your team members and provide access to specific repositories. The feedback from our users was that the groups management UI is not optimal. We listened, and we completely revamped groups management.
The new groups list makes it easier to find the most important information. You can see which groups can manage the team and how many members and repositories are in each group. You can also add new team members directly from this page to multiple groups:
The group details page now has a better display of which permissions are set for the group, with descriptions to state exactly how that permission is used. We’ve also added a list of all the repositories the group can access. The entire page has been designed with efficiency in mind and makes it much easier for large teams to manage group members:
This implementation also fixes some performance issues that were affecting customers managing groups with very large numbers of repositories. Please leave a comment if you have any specific feedback about groups management.
We launched Atlassian Connect for Bitbucket at AtlasCamp, July 2015. Atlassian Connect for Bitbucket provides an integration architecture that embeds your add-ons right within the Bitbucket UI creating a more unified workflow. We re-engineered Bitbucket into a platform that helps remove the interrupt-driven workflow and brings all the information to ship software in one place. Many developers have already built add-ons for Bitbucket and we know some of you are in the process of building one, too.
We’re pleased to announce that we are running a two-day Atlassian Connect for Bitbucket bootcamp on Tuesday, September 29 through Wednesday, September 30 at our San Francisco office. This is your opportunity to get our help in accelerating your integration and bringing it to life. You’ll have the unique opportunity to get in-depth information from our Bitbucket team as well as 1:1 advice on building kick-ass integrations directly from Atlassian engineers. Tuesday night you’ll also get networking time with the team over beer and pizza. Why not come hang out with the Bitbucket team for a couple of days and make your integration a reality? Here’s an outline of some of the topics we’re going to cover to help you get going on your integration:
Technical training sessions on:
Overview of Atlassian Connect for Bitbucket
Bitbucket UX and design considerations
Security best practices
Bitbucket REST APIs
Hack time & 1:1s with Bitbucket engineering
Evening networking with the Bitbucket team over beer and pizza
More hack time
Demos in the afternoon
We have limited space for this event so you’ll need to pitch us your integration idea. We’re accepting and reviewing applications until September 18th. Learn more about Atlassian Connect for Bitbucket and send us your application soon. Experience with Atlassian technology is a plus, but the best Bitbucket integration ideas will be accepted regardless of prior work. As a bonus, the add-ons you hack on during the bootcamp are also eligible for Atlassian’s Codegeist competition, where we’re giving away prizes totaling more than $100,000.
We recently added four new features to make it easier for you and your team to collaborate on code using Bitbucket:
File-level code comments
In addition to comments on images and binary files, you can now leave comments at the file level for code and text files. This makes it easier to have a high-level discussion that applies to the entire file.
Improved commit message handling
The commits history, compare view, and pull request pages now display full commit messages when listing commits. Commit message text after the first line is displayed in gray text, providing better context. And you can mouse over the commit message to see it in full. No need to click on the individual commits to see the full commit messages.
Pull request links in the terminal
The Bitbucket CLI provides links (as URLs) right in your terminal to create and view pull requests when creating and updating remote branches with Git and Mercurial. Quickly copy and paste the URL in the browser to update the pull request.
Pull request form auto-fill
The create pull request form now automatically fills in the description field based on the incoming branch’s commit messages.
For branches with one commit, the commit message is the title. For branches with multiple commits, the branch name is the title, and the commit messages pre-fill the description. This saves you the hassle of typing a title and a description for every pull request.
This guest post is written by Ori Pekelmen, VP of Inbound Marketing at Platform.sh. Ori is a start-up guy, with a tech background, who enjoys evangelizing new technologies, organizing events such as Paris DataGeeks, and helping tech teams succeed.
What if you could go beyond code review in a pull request and actually play with the new feature you’re working on, without the hassle of deploying to your machine? What if you could easily share a feature spike with your clients for early feedback, before it’s actually shipped? The new Platform.sh integration with Bitbucket makes it simple to stage pull requests and feature branches on-the-fly without any additional effort. Interested? Read on.
Platform.sh is a second generation PaaS powered by a highly available grid of micro containers. You push code, and we automatically put everything into production. We deploy databases and configure the lot with no sysadmin work, none, zero. It’s fully automated and happens almost instantly. Platform.sh removes the middle-man between your code and your deployment.
One of the earliest features we implemented was a simple Bitbucket integration. We love the Atlassian suite of tools and use it quite a lot internally since we like dogfooding as much as anyone else.
A pleasant surprise
But we felt the integration could be better. Our developers wouldn’t get feedback inside the Bitbucket UI on how the deployments went, and switching tools constantly made them feel they lost context.
In March 2015, Fred Plais, our CEO, and I went to San Francisco for a series of meetings, one of which was with the Atlassian Bitbucket Team. We showed them “screen captures” of our ideas behind a good integration between Platform.sh and Bitbucket. There was some grinning around the table that we didn’t immediately understand. A couple of weeks later the team got back to us with an unexpected question, “The screen captures you showed us were precisely the kind of integration we’ve been working on. Can you be ready in 6 weeks?”
Two months ago, and on time, Platform.sh was fortunate enough to be among the first Bitbucket Connect Add-Ons, and shipped https://platform.sh/bitbucket.
What? There are people in 2015 that have feature branches without a live test cluster?
It’s never been easier to deploy an application directly from a Git repo, never before was collaboration, testing, and development so seamless.
And judging by our client’s response, we got something right. Some of our clients just immediately got used to the idea that every branch becomes clickable in the Bitbucket UI, giving access to a staging environment, just for that branch.
“I’ve been managing Bettrack’s code with GIT since day one […] when I installed the Platform.sh add-on on Bitbucket, I was […] blown away by how seamless the integration is. This integration just saves me so much time! When I make changes in my code, I can see the outcome right away. I decide when I want to go live, and that’s as simple as pressing a button. That’s just so powerful! The need to react to market forces is very important and Platform.sh with Bitbucket have dramatically improved not just the site but Bettracks as a business”, said Scott Hooker, owner of Bettracks, a betting website for football fans.
This is how feature branches should always have been, testable but in isolation. This integration simply makes the idea of feature branches, feature complete.
Follow the simple wizard that allows you to setup a Platform.sh project, which will be integrated with your Bitbucket repository.
From there, there’s nothing much to do. Every Git branch gets a clickable staging URL.
All this is possible because Bitbucket’s Atlassian Connect framework provides much deeper integration than what is offered by standard REST APIs and webhooks.
From the Platform.sh side you just add a simple YAML file at the root of the repository and your application is ready to be deployed. Having the deployment configuration checked into your repository not only makes it simple to deploy, but it also means that it is tracked over time as your application evolves. And the integration works the other way around, too. You can log in to Platform.sh with your Bitbucket account to get detailed feedback on deployments and the state of your environments.
You can even manage a micro service oriented architecture as a single entity (with every service as a separate Bitbucket private repository) and still get “magic” testing environments for every branch.
And Platform.sh is not just for staging.
Robust hosting and batteries included
Platform.sh is not a makeshift solution. It is a robust, mature, hosting environment. Platform.sh enterprise includes white glove support and 99.99% guaranteed availability for the price of traditional managed hosting.
And it is not only extremely scalable, but also very versatile. By adding a single property to a configuration file, you can immediately get an instance of Postgres, MariaDb, Redis, Solr, ElasticSearch, etc. with no external add-ons and no external points-of-failure. You can manage your entire infrastructure right there, in your Bitbucket account. And when you backup your cluster, you get a consistent snapshot of everything.
Simple is more
Deep integration with code management and hosting is a big leap forward. It allows even small and medium sites to go full-on-continuous-everything like a Google or a Facebook which translates to reduced lead times with huge cost savings.
Platform.sh and Bitbucket integration gives you all the DevOps features with none of the hassles. You are not locked into complex software and architectures that might not exist in a couple of years. You run your stack unmodified, and a couple of YAML files makes it “continuous-everything on the cloud.”
Currently Platform.sh supports any PHP application. If you are using Nodejs, Ruby or Python, stay tuned on Twitter for some exciting news in the future. If you have any feedback about the add-on, let us know by leaving a comment on this blog post.
This guest post is written by Alexander Kuznetsov, co-founder of StiltSoft. Alexander has seven years of experience as a software developer, including five years in developing add-ons for Atlassian platforms. He’s also the runner-up of 2012 Codegeist, Atlassian’s add-on development competition, for the add-on he built called, “Awesome Graphs for Stash.”
We at StiltSoft, Atlassian verified vendor and expert, create add-ons for various Atlassian applications. Our team is always eager to provide developers with the handy tools they need so they can focus more on the important stuff, deliver on their work commitments with fewer efforts and enjoy the process at the same time.
One of our products is Awesome Graphs for Stash. It provides graphs and charts to visualize the contribution statistics in Git repositories, evaluate a team’s performance and get useful data for planning and analysis.
We have watched Bitbucket evolve at a rapid pace and couldn’t wait for the opportunity to make Awesome Graphs available in the cloud. We were very excited when he heard about Atlassian Connect for Bitbucket. The Atlassian Connect framework made it possible to build the add-on and make Awesome Graphs available for 3 million developers on Bitbucket.
Awesome Graphs for Bitbucket
Our goal currently is to make the graphs and charts of Awesome Graphs for Stash available in Bitbucket so that Bitbucket developers can visualize the statistics of their repositories, keep track of what the contributors have done, get analytical data for running retrospectives and plan their future work while making informed decisions.
It is always very inspiring to watch your projects evolve by checking graphs from time to time. You can track your progress, see the work that has been done and feel accomplished.
We released the first version of Awesome Graphs for Bitbucket in June with Commits and Punchcard. That’s only the beginning and more graphs are coming in the next couple of months.
Why should you use Awesome Graphs?
The Commits tab shows the number of commits made over the last year grouped by week on a given repository. The interactive scatter chart below the bar chart displays detailed daily commits for the week selected on the bar chart. This allows you to get a summarized history of each repository and understand when most of the work was done.
The Punchcard tab helps you figure out which hours of the week have been the most productive for your team by showing commit frequency on each day of the week and time starting from the first commit in the repository. This helps you plan your day’s activities so that you are not distracted and completely focused during the most productive hours. Most importantly, you can use the insights to improve the performance and efficiency of your team.
Wanna see it in action?
You can try Awesome Graphs for Bitbucket by installing the add-on from the Find new add-ons section of Bitbucket. You can start using it as soon as the installation is completed. Navigate to the Graphs section on the left-hand sidebar to view commits and punchcard graphs of the current repository.
Bitbucket just added different levels of access to its APIs, enabling add-on and integration developers to request read, write, or admin access to repos, issues, wikis, and snippets. For Bitbucket users this means that add-ons and integrations now only get the level of access they actually need.
Our previous OAuth1 implementation provided only all-or-nothing access to resources. As a developer building integrations or add-ons, you had less control. Additionally, some operations that were not possible via OAuth 1, such as cloning, are now possible via OAuth2, enabling a new set of add-on and integration functionalities.
How do I use OAuth2?
OAuth2 is available within webhooks, REST APIs, and Atlassian Connect for Bitbucket. When you register your application (add-on or integration) with Bitbucket you become an “OAuth consumer”. OAuth 1 consumers that have been previously instantiated have been grandfathered into all-access. However, all new consumers will now need to specify the scopes requested, with a much more granular set of scopes available now for both OAuth options.
How does it impact me if I am a Bitbucket user?
You have complete control and visibility over what levels of access are granted to resources and repositories when you plan to use a certain integration or add-on. You may choose to grant access via a page similar to the one shown below:
You can also revoke access easily via “OAuth” under “Access Management” on your Settings page as shown below:
Please upgrade to OAuth2
We encourage all developers of Bitbucket integrations and add-ons to upgrade the authentication mechanism they’re using to scopes and OAuth2 wherever possible.