Git Large File Storage (Git LFS) now in Bitbucket Cloud

By on July 18, 2016

In recent years software teams across all industries have adopted Git thanks to its raw speed, distributed nature and powerful workflows. Additionally, modern software teams are increasingly cross-functional and consist not only of developers but designers, QA engineers, tech writers, and more. In order to be successful these teams need to collaborate not just on raw source code but on rich media and large data.

It’s no secret though that Git doesn’t handle large files very well and quickly bloats your repositories. We are therefore excited to announce that, following Bitbucket Server’s lead earlier this year, Git LFS is now available in beta for Bitbucket Cloud to improve the handling of your large assets. So even if your files are really, really large, Bitbucket Cloud allows your team to efficiently store, version and collaborate on them.

Why should you care about Git LFS?

Git was optimized for source – it’s easily merged and compressed and is relatively small, so it is perfectly feasible to store all history everywhere, but this makes it inefficient and slow when trying to track large binary files. For example, If your designer stores a 100 MB image in your Git repository and modifies it nine times, your repository could bloat to almost 1 GB in size, since binary deltas often compress poorly. Every developer, build agent, and deploy script cloning that repository would then have to download the full 1 GB history of changes, which may lead to drastically longer clone times. Just imagine what would have happened if your designer made 99 changes to that file.

A common solution to this inherent flaw in Git is to track these large files outside of Git, in local storage systems or cloud storage providers, which leads to a whole new set of problems. Separating your large files from your repository will require your team to manually sync and communicate all changes to keep your code working.

With the addition of Git LFS support, you can say goodbye to all these problems track all your files in one place in Bitbucket Cloud. Instead of bloating your Git repository, large files are kept in parallel storage, and only lightweight references are stored making your repositories smaller and faster. The next time your team clones a repository with files stored in Git LFS, only the references and relevant large files that are part of your checked out revision will get downloaded, not the entire change history.

For those interested in a longer explanation of how Git LFS works and how to migrate your existing repository, watch this presentation by Tim Pettersen, Atlassian Developer Advocate, on Tracking huge files with Git LFS.

When is Git LFS right for you?

Generally, if you want to use Git to easily version your large files, Git LFS is the right choice. To call out just a few cases in which Git LFS will make your life easier, here’s a short list:

  • Game developers working with large textures, 3D, audio and video files
  • Mobile developers catering for higher and higher display resolutions
  • Web developers building pages with rich media
  • Software teams handling checked in dependencies
  • Researchers working with huge data sets
  • Multimedia producers and designers
  • QA engineers using database snapshots for functional tests
  • Technical evangelists who store their presentation slides in Git

Git LFS support is built right into SourceTree

If you’re like us and use SourceTree, tracking files with Git LFS is one click away. Simply right click on any file type you’d like to track and select the “Track in Git LFS” option. SourceTree will do the rest for you.

Group 3

Group 2 Copy

What you will get with the Git LFS Beta

  • Track everything in one place – Track your large assets alongside your source code and stop worrying about manually versioning them in a separate storage system.
  • Store, version and share large files – Version all your large files with Git, no matter how large they are. With Git LFS you can store huge audio files, graphics, videos or any other binaries efficiently. During the beta period, you get 1GB of storage for up to 5 users. If you are a paid team your storage is based on your user tier.
  • Free up your repository – Store more in your Git repository. Git LFS stores your large files externally and keeps your actual Git repository lightweight.
  • Clone and fetch faster – Only download what you need. Git LFS will only fetch those large files that you check out and not the entire history of changes.
  • Just do git – No need to learn anything new. Git LFS seamlessly integrates into your Git workflow and does not require you to learn a new workflow, new commands or use new tools.

Git going right away

If you’re ready to get started, sign up for a Bitbucket Cloud account. If you’re already using Bitbucket Cloud, enable it in one click on the repos you would like to try it out on by heading to the “Git LFS Beta” section on the left sidebar.

Sign up for Bitbucket

29 Comments

  • Mathias Brodala
    Posted July 19, 2016 at 12:10 am | Permalink

    Why is there a separate “Track in Git LFS” in SourceTree if this feature supposedly works in the background? Thus adding the file to the index as usual should be enough, right?

    • Rahul Chhabria
      Posted July 19, 2016 at 10:23 am | Permalink

      That’s correct! We wanted to surface this functionality in SourceTree in the event you were starting with SourceTree.

    • Tim Pettersen
      Posted July 19, 2016 at 6:24 pm | Permalink

      Hi Mathias,

      Git LFS works pretty much automatically, but you do have to initially tell it which files or file patterns you want to track using LFS. If you wanted to track all mp4s with LFS, for example, you could invoke the track command via your shell:

      git lfs track *.mp4

      Selecting “Track in Git LFS” in SourceTree will mark just that file as being tracking in LFS. It’s the equivalent of running:

      git lfs track

      Once your files are tracked, you can stage them using ‘git add’ and check them out as normal.

      Hope this helps!

      cheers,
      Tim

      • Mathias Brodala
        Posted July 21, 2016 at 12:23 am | Permalink

        Thanks a lot, I must have missed this (obvious) step. 🙂

  • Joe Taylor
    Posted July 19, 2016 at 1:29 am | Permalink

    How do the paid tiers for LFS work?

    • Rahul Chhabria
      Posted July 19, 2016 at 10:22 am | Permalink

      Hey Joe, Please checkout our storage policy here: https://confluence.atlassian.com/display/BITBUCKET/Storage+policy+for+Git+LFS+Beta+with+Bitbucket. It breaks down the allocations by tier. Thank you!

      • Joe Taylor
        Posted July 19, 2016 at 12:34 pm | Permalink

        Thanks Rahul, can you buy extra data packs if you exceed your limit like you can on GitHub? At the moment over on GH our team has 5 seats and is using about 400gb worth of LFS storage

        • Szilard Szasz-Toth
          Posted July 19, 2016 at 4:07 pm | Permalink

          Hi Joe, At the moment you can’t buy extra storage packs but you can upgrade to the next tier to get more storage during the beta. We have plans to introduce better ways to get more storage in the future for the General Availability of LFS.

        • Szilard Szasz-Toth
          Posted July 19, 2016 at 4:32 pm | Permalink

          Hi Joe, at the moment you cannot buy extra storage packs but you can upgrade to the next tier to get more storage during the beta. We have plans to introduce better ways to get more storage in the future for the General Availability of LFS.

  • zdm
    Posted July 19, 2016 at 2:56 am | Permalink

    When support for mercurial large files will be added?

    • Rahul Chhabria
      Posted July 19, 2016 at 12:32 pm | Permalink

      Thanks for writing in. We do not have any plans to extend the beta to Mercurial at this time.

      • Etienne
        Posted July 21, 2016 at 1:45 am | Permalink

        Really?! The issue about the lack of support for large files for Mercurial exists for years, come on… Don’t you listen to your community sometimes?

        And wasn’t Bitbucket at the origin made for Mercurial? Shouldn’t it still be your priority over additional CVS?

        • Posted July 21, 2016 at 5:54 pm | Permalink

          Thanks a lot for your feedback Etienne! We are still dedicated to Mercurial and our integrated CI tool Bitbucket Pipelines just recently added support for it. For LFS we decided to focus on Git for the Beta as Atlassian is one of the top contributors to the public git-lfs extension and lots of customers have been asking for Git LFS. That doesn’t mean that we might not consider adding Mercurial support in the future, though.

        • rewritten
          Posted July 25, 2016 at 1:29 am | Permalink

          Etienne, take into account that large file support in git is open source (just like git itself and mercurial). The actual git-lfs extension has been developed since three years ago by some key people (mostly from github, but also the original writer of atlassian’s SourceTree). It remains to be seen if anything like this is even possible with mercurial, and whether there is enough need so people will dedicate their time to it – it’s all about open source in the end.

          • Robert
            Posted September 20, 2016 at 8:09 am | Permalink

            Mercurial has included the largefiles extension since 2011.

  • Chris
    Posted July 22, 2016 at 11:56 pm | Permalink

    When will LFS be added to Sourcetree 1.7.x?

    • Posted July 31, 2016 at 10:16 pm | Permalink

      Hi Chris, SourceTree offers Git LFS support on Mac since version 2.2.2 and on Windows since version 1.8.2.

      • Chris
        Posted July 31, 2016 at 10:23 pm | Permalink

        Hi Szilard.

        I was specifically asking for this under the 1.7.x (win) branch.

        The 1.8.x branch contains the UI changes which many users find unusable, and a lot of people are remaining or have downgraded back to this version of the software.

        My question is to when will the LFS feature be back-ported to 1.7 as a lot of users want the new functionality without all the interface changes – which make the 1.8 version of software very difficult to use (in a UX sense) for regular use.

        • Rahul Chhabria
          Posted August 2, 2016 at 7:24 pm | Permalink

          Hi Chris,

          Thanks for writing in. LFS will not be back-ported to SourceTree for Windows version 1.7. Sorry for any inconvenience.

          Best,
          Rahul

          • Chris
            Posted August 2, 2016 at 7:31 pm | Permalink

            I find that very disappointing.

            You can have either a working, usable system with a usable interface (1.7.x).

            Or you have an unusable system with LFS (1.8.x).

  • Posted August 3, 2016 at 10:25 pm | Permalink

    I can’t seem to find code search in BB cloud… help?

    • Posted August 3, 2016 at 10:46 pm | Permalink

      Hi, code search is not yet available in Bitbucket Cloud but you can follow this ticket to get the latest updates on our progress:

      https://bitbucket.org/site/master/issues/2874/ability-to-search-source-code-bb-39

      Thanks,
      Szilard

      • Posted August 3, 2016 at 11:09 pm | Permalink

        Is that the right ticket? It looks like it was opened in 2011.

        • Posted August 4, 2016 at 5:42 pm | Permalink

          Yes, that’s the right ticket. As Dan pointed out in the recent update we had to finish up a few things but have a team focussing on search inside of Bitbucket Cloud. We will post all future updates on that issue to keep everyone informed on our progress.

          • Posted August 5, 2016 at 11:53 pm | Permalink

            It’s such a huge hole in the product. Glad there’s a team on it.

  • Khambadkone
    Posted September 21, 2016 at 11:37 am | Permalink

    Can you explain how this would work with the 2GB limit of Bitbucket repositories?
    we have a single game project that’s at 10GB, team of 4.

    • Posted September 22, 2016 at 9:57 pm | Permalink

      Hi, LFS storage is separate from your 2GB repo storage. LFS files are stored outside of your Git repo, so you can still store data in your Git repo up to 2GB and store anything else beyond that in LFS.

      Plans on the free 1-5 user plan currently get 1GB of free LFS storage.

      You can read more about the current LFS storage policy over here:
      https://confluence.atlassian.com/display/BITBUCKET/Storage+policy+for+Git+LFS+Beta+with+Bitbucket

      Thanks,
      Szilard

      • Khambadkone
        Posted September 22, 2016 at 10:05 pm | Permalink

        Thanks.

        Is there anyway to increase the LFS storage without changing the core Altassian plan.

        Mahesh Khambadkone
        Co-Founder – Games2win

        *We’re Hiring! *

        • Posted September 22, 2016 at 10:09 pm | Permalink

          Unfortunately, at the moment you can only move to the next user tier to get more LFS storage.