Introducing automatic concurrency control for Bitbucket Deployments

By on May 15, 2018

Last month, we announced the general availability of Bitbucket Deployments, a new feature within Bitbucket to help you keep track of the status of your shared deployment environments. With Deployments in Bitbucket, your team has every capability they need, from code hosting, code review, built-in CI/CD and now deployment tracking, to build and ship great products from within Bitbucket.

But there’s still a lot of uncovered potential for Deployments. Imagine, for example, that you push some code which triggers a pipeline that deploys to production. Halfway through, your colleague pushes more changes that also trigger a deployment – two deployments are now running! What will your production environment end up looking like?

We’ve received feedback from teams that it’s important only a single deployment is in progress to an environment at any given time. This gives teams confidence about the status of their environment and eliminates any uncertainty about whether deployments are running in the right order. We’ve listened to your feedback and realise how important it is for Bitbucket to just do the right thing.

So we’re excited to announce that Bitbucket Deployments is the first CI/CD tool to automatically enforce deployment concurrency control. Simply by adding deployment tracking to your pipeline, we will automatically ensure that only one deployment is in progress per environment at a time.

This is amazing! There’s nothing more frustrating than a 35min build failing because the environment is still updating”

– Ken Greeff, Software Engineer at Realhub Systems

How does the concurrency control work?

Deployment concurrency control requires you to first enable Bitbucket Deployments for your repository. With deployment tracking enabled, Pipelines will then check whether there is already a deployment in progress before starting a new one to the same environment. If there is already a deployment in progress, later deployment steps will be paused. You can then manually resume the deployment once the in-progress deployment has completed.

This feature is hugely valuable to teams who deploy regularly – by letting the in-progress deployment to finish executing before allowing a later deployment to be run, the status of your environment will always be known and your deployment won’t fail due to unavailable resources. Keep in mind that Bitbucket Deployments tracks the status of shared environments in the repository, and that developer’s individual test environments don’t need to be tracked.

We’re super excited to share this addition to Bitbucket with you. If you have any feedback to share, tweet us @Bitbucket. Happy deploying!

File search improvements in code aware search for Bitbucket Cloud

By on May 10, 2018

Last October, Bitbucket Cloud launched code aware search: a semantic search that ranks results by relevance, starting with the function or type definition first. It was built this way to save time manually combing through results when looking for things like classes, interfaces, structs or enums. We’ve continued to expand it by adding a code search API so developers can make code search work for their unique needs.

Since launch, code collaborators could efficiently find the code they were searching for, but results for a file search would show usages of the filename in the code and not the actual filename. This caused a lot of effort to find the file. To fix this, we’ve launched improved file search support which now includes filename matches, extending the syntax-aware code search.

What’s new with file searches? 

#1 Search results take into account filenames and paths

Files can now be found just by searching their filename or parts of the path.

file search bitbucket

If a search matches both in a file’s path and content, those results are shown first. We aim to match the developer’s search intent with smart ranking by boosting search results based on matches in the file’s name, path or code.

#2 A path modifier to refine results

A developer searching for a common phrase or name would get served overwhelming results that they would have to click through page after page to find the right result. To make this easier, a developer can now enter the path modifier, which can be used to limit the search to certain paths or exclude them.

A lot of people will have checked in node_modules and so it’s a very common match. The new path modifier supports a NOT option so the search could be updated to path:package.json -path:node_modules lodash to exclude those matches.

Bonus: A similar filter can be achieved using path anchoring, searching with path:/package.json lodash will only return results for package.json files at the root of the repository.

Try Bitbucket Cloud’s file search

If you’re ready to give our robust and relevant search a try, sign up for a Bitbucket Cloud account, create a repository, and add your code.

If you’re already a Bitbucket customer, you can find code search with improved file search from your sidebar and read further documentation on it here.

Watch this! New look, watch repos, and unicorns in Bitbucket 5.10

By on May 3, 2018

Does it ever feel like the codebase you work in (or its core dependencies) can change in the blink of an eye? As soon as you’ve moved to the latest version, a new stable release comes out. It’s even more frustrating when it happens inside your company. Sure, it might come up during daily standup… but that doesn’t always capture the downstream impact.

Now, you can watch repositories to get an email update with the latest commits or pull request updates. We’ve even added a fresh new look and a little something extra, just for fun. Read on to learn more, or get started with Bitbucket Server 5.10 now.

Download Bitbucket Server 5.10

Staying up-to-date with repositories

Staying on top of the latest changes across multiple repositories can be harrowing for the most seasoned developers. Monolith apps can get hundreds of pull requests in a week, and a team across the globe might make a change to a shared dependency. There has to be a better way for developers and managers to get the latest updates without having to dig through pages of pull requests or commits, right?

With the new watch repository functionality, you can receive email notifications for the repos you’re interested in (and only for those). Most developers are familiar with variations of this feature, but we wanted to offer refined control for the types of email updates you might receive.

Bitbucket Server’s watch emails can be customized to include commits to the default branch only, or all branches as well as pull request state changes, or all pull request activity. These updates can either be batched or sent immediately. Each option helps developers optimize for different goals. Batched emails reduce your inbox clutter, while immediate email updates make sure you know what’s going on as soon as it happens.

A new look for Bitbucket Server

We launched our bold new brand this past September and today we’re introducing a fresh look for Bitbucket Server and Data Center. You may have already encountered it in Confluence server. It is the same Bitbucket workflow you know with a crisp, beautiful UI. Our new look is based on the Atlassian design language and includes updated colors, typography and icons.

Download Bitbucket Server 5.10

Improving developer productivity is near and dear to us at Atlassian – especially since we dogfood all our products. Our devs are loving how Bitbucket Server 5.10 surfaces repository updates in a more intuitive fashion so they never miss a beat, and we hope you’ll love it too.

Btw, if you want to add a little fun to your code review, we’ve also added support for the unicorn emoji (🦄)! Because y’know… unicorns.

Interested in the details of our latest release? Read more in our release notes.

 

Bitbucket, uninterrupted: app diagnostics and better workflows in Bitbucket Server 5.9

By on

Bitbucket Server is the convergence of individual work and team collaboration. Administrators ensure the git server availability, enabling developers to complete deployment cycles. Those teams operate independently but share common goals like, automating and simplifying repetitive tasks.

In Bitbucket Server 5.9, there are improvements for both admins and developers. Bitbucket admins can identify and track causes of degradation, like third-party or custom apps, with a new diagnostics tool and UI. Pull requests and code searches reflect developers’ workflow expectations and offer greater continuity when moving between the command line and Bitbucket Server. Now, you can search commits or edit commit messages when merging. Keep reading to learn more or get started with Bitbucket Server 5.9 now.

Download Bitbucket Server 5.9

Understanding Bitbucket performance

It’s 11pm and you get a Pagerduty alert. Your stomach drops in anticipation of major performance degradation or outright system failure. For Bitbucket Server administrators, we never want to be the source of your on-call nightmares. Performance degradation of a git server can impact hours of productivity or an important release, but issues caused by third-party apps frustrate even the most patient admin.

With Bitbucket Server 5.9, we’ve surfaced diagnostics tools for administrators within Bitbucket UI. Admins can get an overview of Bitbucket system health with a summary of alerts & events for the last 30 days. Diagnostics lists events of different severities: an error, a warning, or simply worth noting. Using this new information, admins can proactively address issues which may lead to more serious problems or tackle system errors before users begin to report them.

Diagnostics also supports event filtering by node, date, or time. For persistent issues or on-going anti-patterns, this provides a level of traceability so admins aren’t stuck searching HipChat to create a timeline or identify a problematic node.

Scaling and improving search

For developers, we’ve made search a little smarter. Navigating to the commit view has previously been tricky, making it hard to jump from an IDE blame into Bitbucket to get context. Now, you can search for code, commits, or repositories. But commit search isn’t the only improvement. Have you ever looked used a period or an underscore in search, hoping to find a file or a variable name? Previously, these punctuation marks were stripped, making it frustrating to find the right result. Results now surface more relevant code with respect to these punctuation marks.

But beneath each code or repository query lies Elasticsearch. With the bundled instance for server and clusters for data center, Elasticsearch is critical to developer productivity, but its performance is an admin responsibility. As of Bitbucket Server 5.7, we’ve added support for Elasticsearch 5.5, allowing admins to upgrade from an end-of-life version.

Pull request enhancements

Bitbucket Server is home and hearth to pull requests and git workflows. We should know since we wrote the handbook. To make your software development team’s workflow more effective:

What used to take clever workarounds is now a click away.

Download Bitbucket Server 5.9

Empowering administrators’ and developers’ common tasks in Bitbucket Server help software and IT teams gain the stability and performance needed to grow. Want to dive into the details of our latest release? Read more in our release notes.

Bitbucket Cloud gets a new source browser experience

By on May 2, 2018

As part of our ongoing efforts to improve Bitbucket Cloud for users we’re always looking at ways to help you code smarter and ship faster. With this in mind we’re proud to unveil our latest improvement, a new source browser experience designed to ensure browsing and viewing source code is more effortless and consistent in the product than ever before.

And while clarity and simplicity were at the heart of the new design, we focused on increasing performance to ensure a fast, efficient browsing and viewing experience. Let’s take a look at some of the improvements.

The new source browser experience

 

Combined repository overview and source tabs 

The repository navigation will no longer lead with the Overview; instead key components of the Overview were combined with the Source browser in order to place focus on primary user actions. No longer will you have to toggle between multiple views in order to view your README and source files.

New right sidebar

We moved much of the information previously found in the Overview into expandable cards in a new right sidebar alongside build statuses. This new experience offers important information about the repository when needed but is also collapsible so it can get out of your way when you need to focus. Best of all, this new pattern opens the door for a range of new information to be added in the future, and we’re working on add-on extension points so the Bitbucket community can develop and share their own too.

 

Helpful directory 

We’ve supercharged the directory listing for the files in your repository. Navigating between directories and files happens entirely on the client, so the experience should feel much snappier as a whole. “Last modified” information now links directly to the corresponding commit and, even more exciting, we now include information about the most recent commits for both files and entire directories. That large block of whitespace that was once at the top of your repository’s root directory list is now full of useful information.

Single Page Application

The new source browser now operates as a single page app which creates a more responsive browsing experience. Whether you’re navigating to a new file or directory, or switching between the various source, diff or commit history views, you’ll notice that the transitions are faster and snappier than ever.

We’re excited to release this new and improved experience in the wild and into your hands. It’s a taste of things to come as we continue to add more wonderful features to the new browser!

 

Try Bitbucket free

 

Renewing our certificates, plural

By on April 25, 2018

The certificate behind https://bitbucket.org is up for renewal. We’re planning to switch to our new TLS certificates – one RSA, one ECDSA – at around 00:05 UTC on Friday, 27 April 2018. (SSH traffic will not be affected by this.)

What does this mean? After we deploy our new certificates, some Mercurial users who connect over HTTPS may see a warning or error message about “unexpected fingerprint” or “certificate not verified”. If you see a message like that, check if the error message matches the new fingerprints below:

RSA

SHA1:23:40:D5:EB:C5:02:31:DA:FF:93:D1:4F:5E:C3:A7:D8:32:4F:E9:82

SHA256:BD:0C:FF:27:A2:B4:5D:D6:BC:AE:DA:0A:95:A5:10:29:71:19:4A:5B:97:73:49:7C:1B:14:43:E5:EA:E8:1D:53

ECDSA

SHA1:70:A7:44:EA:A9:EA:E8:69:16:EA:12:00:35:A9:A6:0E:AE:38:8E:F8

SHA256:32:12:90:9A:70:64:82:1C:5B:52:CC:C3:0A:D0:79:DB:E1:A8:62:1B:9A:9A:4C:F4:72:40:1C:A7:3A:D3:0A:8C

If it does match, then you can pin that fingerprint in your ~/.hgrc or Mercurial.ini as follows:

Mercurial versions <=3.8:

[hostfingerprints]
bitbucket.org = FINGERPRINT_HERE

Mercurial versions >= 3.9:

[hostsecurity]
bitbucket.org:fingerprints = FINGERPRINT_HERE

(adding the actual fingerprint in place of “FINGERPRINT_HERE”, of course.)

If you’re using Python 2.7.9 or later, then you may be able to remove the fingerprint from ~/.hgrc or Mercurial.ini entirely. Please see https://www.mercurial-scm.org/wiki/SecureConnections for more information.

Most users should not notice a difference once we make the switch – your system will use ECDSA if it can, and RSA otherwise, and your system will verify the certificates as it needs to. We’ve been running dual RSA and ECDSA certificates on all bitbucket.io hosted pages and on api.bitbucket.org since June 2017, and it has not caused any support tickets, monitoring failures, or tweets.

Happy coding!

Bitbucket Deployments is now available to all Bitbucket Cloud teams

By on April 17, 2018

With more teams adopting CI/CD and deploying code on a regular basis, keeping up-to-date with what has been deployed has become more difficult. Teams need a single place to track their deployments so they can build and release their software quickly with confidence.

In December, we announced our early access program for Bitbucket Deployments, a new way to track, view and monitor deployments from within Bitbucket. Today, we’re excited to announce that Bitbucket Deployments is available to all Bitbucket teams, with a beautiful new design to top it off.

bitbucket deployments

Track your deployments

The Bitbucket Deployments dashboard automatically tracks all the deployments you run through Pipelines, giving your team visibility over what’s running in each environment and the status of each deployment. There’s also a complete history of all deployments to each environment, shown by clicking on the history icon on each.

Environments in Bitbucket Deployments are configured out of the box as teststaging, and production, and your team can choose to use one or more of these environments as required by your workflow.

This improved visibility is already helping teams dramatically improve their CD workflows, like the team at Fran’s Chocolates in Seattle, WA:

Before adopting Bitbucket Pipelines, our delivery process was like the wild west. There was nothing to stop broken code from being deployed or see what had been deployed where. Bitbucket Deployments was the final step in getting transparency and control over our deployments.” – James Sweeney, CTO at Fran’s Chocolates

Preview and promote deployments between environments

For many teams, an important part of their workflow is a manual review of changes that will land as part of their deployment. Bitbucket supports this workflow by allowing you to configure a manual deployment step which will pause the build before doing the deployment. Any build artifacts that are configured by the build will be kept for up to a week so that you can run your deployment step at a later point in time.

Where a manual deployment is available, Bitbucket Deployments shows a Promote button on the dashboard. This gives your team control over when to promote a build to a new environment, and lets you preview the specific commits and code changes before hitting Deploy.

bitbucket deployments

A complete record of what code was deployed when

By clicking on the history icon, you can see a history of all earlier deployments to an environment. Clicking on any deployment on the dashboard will load a summary showing the changes that were deployed, along with information about when it was deployed and by whom.

With a detailed history of past deployments, investigating problems becomes much faster and simpler. Your team can confirm the cause of an issue by inspecting the code and commits that were deployed, all without leaving the deployment dashboard.

A new design to better support your workflow

Along with many functional improvements, we’ve done a complete visual redesign of the deployment dashboard for this release. The initial state is clean and shows the most relevant status information. A deployment workflow bar across the top clearly communicates the flow of builds through your deployment environments.

For teams with a manual promotion workflow, we’ve highlighted and expanded the promotion buttons to make it easier to perform this common action.

We’ve also revised the environment history list with the aim of speeding up the most frequent interactions. You can now expand deployments in the environment history and quickly see the commits that were included in each. This makes the task of scanning through recent deployments to find the cause of a bug much quicker.

bitbucket deployments commits

This is just the beginning…

We have a ton more exciting things in the pipeline for Bitbucket Deployments, including deployment concurrency controlrerunning failed deployments and Jira Software integration. We’d also love to hear your feedback about how you’re using Bitbucket and how we could make Deployments better.

To get involved, follow @Bitbucket on Twitter for updates and shoot us a tweet with how Deployments is working out for you.

Try Bitbucket Deployments

 

Bitbucket code search API is now available

By on April 5, 2018

Last year we shipped the highest requested feature for Bitbucket Cloud – code aware search, and we’re delighted with your feedback and responses.

We’re excited to announce that we’ve published the Bitbucket Cloud code search API delivering the same love to machines, and opening up code search for your needs.

All code search features and more

Our goal for the code search API was simple – to give consumers access to all search features they already know from the Bitbucket UI. That includes account scopes, powerful query language and modifiers. So how would the search for the common class QueryBuilders on the Elasticsearch repo look like?

You can get same results by making a request to:

https://api.bitbucket.org/2.0/teams/%7B6f461d1e-a3dd-433b-a0e3-7a69daf6ea47%7D/search/code?search_query=repo%3Aelasticsearch%20QueryBuilders

where search_query is just urlencoded version of:

repo:elasticsearch QueryBuilders

Match highlighting

We haven’t forgotten about search result highlighting. With the new code search API, consumers will get access to that information as well.

and this is how it’s going to be represented in the response body:

Match highlighting


{
  "content_matches": [
    {
      "lines": [
        {
          "line": 39,
          "segments": [
            {
              "text": " */"
            }
          ]
        },
        {
          "line": 40,
          "segments": [
            {
              "text": "public abstract class "
            },
            {
              "text": "QueryBuilders",
              "match": true
            },
            {
              "text": " {"
            }
          ]
        }
      ]
    },
    {
      "lines": [
        {
          "line": 728,
          "segments": []
        },
        {
          "line": 729,
          "segments": [
            {
              "text": "    private "
            },
            {
              "text": "QueryBuilders",
              "match": true
            },
            {
              "text": "() {"
            }
          ]
        }
      ]
    }
  ]
}

For a full list of the capabilities and search query considerations with code search in Bitbucket Cloud, check out our documentation. If you’re ready to use a fast and relevant code search, sign up for a Bitbucket Cloud account, create a repository, and index your code. If you’re already a Bitbucket customer, you can start using code search API today!

What will you build?

If you have feedback about code search or anything else Bitbucket related, hit us up on Twitter – we’d love to hear what you’ve got planned for the API.

Meet Bitbucket Cloud’s new chatbot

By on April 4, 2018

Bitbucket Cloud’s new chatbot features a wide range of notification types, plenty of interactivity, and some smart configuration features that will supercharge your team’s development workflow. The bot is available today for Slack, and is coming soon to Atlassian Stride and other leading chat platforms.

Smarter, by default

Debates have sparked weighing the benefits of real-time communication and productivity costs. Being constantly inundated with notifications—both automated and human—can take their toll on any team trying to focus. By analyzing the usage of Bitbucket’s existing HipChat, Slack, and Stride integrations, we’ve carefully balanced the default notifications to provide an out-of-the-box experience customized for your team.

Wait, “an out-of-the-box customized experience” – how does that make sense? Well, when you create a chat subscription, Bitbucket performs an automated analysis of your repository and configures notifications tailored to your usage patterns.

By default, your team will be notified about important events like pull request activity and commit comments across the entire repository. However we try to be a bit smarter about less critical notifications: you’ll be notified about pushes, merges, and builds only for your primary branch—typically master or default, depending on whether you’re using Git or Mercurial.

Bitbucket detects if you’re using a Gitflow branching strategy, and will automatically notify your team when a new feature branch is created. And these are just the defaults! All notifications are fine-grained, configurable, and able to be bound to specific branches or branch patterns. Different notifications from the same repository can be configured to be sent to different channels—for example, you might send pull request events to #dev and builds or deployments to #ops—so you can tailor each repository subscription to suit your team’s needs.

Complete tasks from your channel

Bitbucket’s new chat notifications are more than just informative – you can now perform key Bitbucket tasks without leaving your channel. With a click on the notification, any member of the team (with appropriate permissions) can create a pull request from a newly pushed branch, re-run a failed build pipeline, or reply to a pull request comment.

But our favorite new feature has to be the ability to “nudge” reviewers, to gently remind your teammates that you needed that code reviewed yesterday. You no longer have to passive-aggressively click the approve/unapprove button to ping your team about it.

The notification is updated with the latest status, so the rest of the team stays in the loop with what’s happening with your builds and PRs.

Context, when you need it

Alongside notifications, Bitbucket provides contextual information from your repositories, if and when you need it. If you mention a pull request, Bitbucket will show you a summary of the salient details, highlighting the outstanding work—missing approvals, failed builds, and uncompleted tasks—required to get it merged. This helps ensure your code flows smoothly from development to production. The message buttons also work in Slack’s mobile app—so you can merge the second you get that reviewer approval or green build, even if you’re on the move!

Inline code snippets

Developers are often sending file references to each other to share code examples, or to indicate the area to make a particular change. If granted permission, Bitbucket will respond to the mention of a source url by inlining the relevant code into Slack at the specified commit. If you include a line number in the URL, Bitbucket will center the snippet on that line, saving you a trip out of your chat room.

Your personal Bitbucket bot is a click away

Bitbucket Cloud for Slack is available today, coming soon to Stride, and will eventually support other leading chat platforms.

Get started with Slack now:

If you’re not using Stride or Slack, tell us which chat platform you’re using by clicking on the “give feedback” button in Bitbucket and help us prioritize our roadmap for future integrations. Happy chatting!

Take me to Bitbucket

 

Speed up your build with parallel steps in Pipelines

By on March 27, 2018

When we built Bitbucket Pipelines, one of our goals was to make a tool that developers love. And if there’s one thing developers love, it is getting their builds finished more quickly. Last year, we added dependency caching and detailed timing information to help speed up your builds.

Today, we’re excited to share that parallel steps are now available in Pipelines to speed up your builds even further, allowing you to run groups of tests at the same time and get your testing done faster.

The team at Paper Interactive, one of our early access customers, is already seeing the benefits of this:

Parallel steps in Pipelines has cut our CI time by two-thirds, saving our developers hours every week.” – Shane Fast, Co-founder and CTO at Paper Interactive

parallel steps bitbucket pipelines

Simple configuration

Configuring parallel steps in Pipelines is simple – just add a set of steps in your bitbucket-pipelines.yml file inside a parallel block. These steps will be started up in parallel by Pipelines so they can run independently and complete faster.

We assume you already know how to split up your tests into batches and run each batch via the command line. How you split them up is up to you – it could be unit vs integration tests, or separating a large number of similar tests into batches of even size. Some test runners can automatically split tests into batches for you, or you can just manually split tests into several test suites to see what performance benefit you will get.

Here’s a full Node.js application pipeline demonstrating an initial build step, a set of parallel testing steps, followed by a deployment step at the end:


image: node:9-alpine
 
pipelines:
  default:
    - step:
        name: Build
        script:
          - npm install
          - npm package
        artifacts:
          - dist/**  # copy these files to later steps
    - parallel:
        - step:
            name: Unit tests
            script:
              - npm run test/unit
        - step:
            name: Integration tests
            script:
              - npm run test/integration
        - step:
            name: Browser tests
            script:
              - npm run test/browserstack
    - step:
        name: Deploy to test
        deployment: test
        script:
          - npm run release --version $BITBUCKET_BUILD_NUMBER
          - npm run deploy/test

Saving your team time

The steps you configure to run in parallel will kick off at the same time in our auto-scaling build cluster, and will run to completion before the next serial step runs. It is primarily designed for large suites of automated tests, but can also be used for large compute tasks that can be parallelized.

Running steps in parallel gives you feedback faster. This saves valuable developer time that would otherwise be wasted waiting for the build. Pipelines will still bill you for all the minutes needed to execute all your parallel steps, so there’s no change to the cost of the build. (And if you run out of minutes, it’s only $10 to buy 1000 more for the month – crazy cheap compared to the developer time you’ll save.)

We’ve kept our existing limit of 10 steps per pipeline, as based on our data our customers don’t currently need more than this. We also expect running steps in parallel to have decreasing benefits beyond 10 steps, as the fixed overhead to start and stop the build starts becoming the limiting factor. If you are using Pipelines and find this limit affecting what you can run, please raise an improvement request and we’ll consider increasing this in future.

To learn more about parallel steps, see our Pipelines configuration guide.

If you have feedback about parallel steps or anything else Bitbucket related, hit us up on Twitter. Happy (parallel) building!

Try Bitbucket free