TLDR: Made a set of scripts automating the migration of Bitbucket repositories from Mercurial to Git.One fine day, my favorite Bitbucket repository hosting service announced that it would soon stop supporting Mercurial repositories in favor of Git, after which all Mercurial repositories would be deleted.
I have many private Mercurial repositories on Bitbucket, I didn’t want to lose them, just like their commit history. Moving to another hosting is also not an option - I'm used to Bitbucket. Bitbucket themselves, oddly enough, were not able to make an in-place converter. They did not even write any step-by-step instructions for converting, sending everyone to the forum of their community - with a rather Jesuit wording (We are happy to support your migration, and you can find a discussion about available options in our dedicated Community thread) - they say We are glad to move your repositories under Git, and how to do this - discuss it yourself on the forum. However, in the same post they left links to a pair of hg-fast-export and hg-git converters .The first one is a separate Python script, the second is a plugin for working with Git repositories directly from Mercurial. Google also found a number of examples of how others solved the task of dragging and dropping Mercurial under Git: lo , behold , and lo . They also used either hg-fast-export or hg-git. The second was already a proven solution for me - I used to use hg-git sometimes when I committed to Github from Mercurial - until I realized that in Git, oddly enough, it's better to commit from Git. So for my purposes, I chose hg-git.HerePowershell script for automating repository migration is provided, which became the starting point of my solution, but for Linux I had to rewrite it to Bash. In addition, the original script only migrates to the local Git repository, and I added the ability to push the contents of the converted Git repository to the Bitbucket repository there. Here's what the resulting script looks like:$ ./convert_repo.sh lebedevsergey advertisements_parser
2a51eee7ade0
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 5 changes to 5 files
updating to branch default
5 files updated, 0 files merged, 0 files removed, 0 files unresolved
Initialized empty Git repository in /home/serge/project/_probes/hg2git/src/advertisements_parser-git/
pushing to ../advertisements_parser-git
searching for changes
adding objects
added 1 commits with 2 trees and 5 blobs
error: Could not remove config section 'remote.origin'
Branch master set up to track
Now I had a ready-made solution, but it required manual actions - before launching it, you had to first create a Git repository on Bitbucket so that there was a lot to push the result. Looking at my 100,500 repositories to be converted, I realized that I would process them for a long time, I would like the script itself to create Git repositories for the converted ones, and even better - it would receive a list of my Mercurial repositories and work on it. This clearly went beyond the skill of pure Mercurial, here the functionality of the Bitbucket itself, or rather the Bitbucket API , was required .There are ready-made libraries for working with the Bitbucket API for various programming languages, in particular for Python - apparently the official SDK(not to be confused with the long-abandoned, but once no less official SDK lying on Bitbucket itself ). There are also Bitbucket API clients for Java , NodeJS, and PHP . The last one I chose.However, when it was studied in detail, it turned out that it was not implemented to obtain a list of user repositories and create a new repository. Fortunately, the creators of the library provided the possibility of expanding their classes by making private methods protected, so that by inheriting the class of their API client, you could add your own methods that do what I need:class ExtendedClient extends Client
{
public function repositories()
{
return new ExtendedRepositories($this->getHttpClient());
}
}
class ExtendedRepositories extends Repositories
{
public function listWorkspace(string $workspaceName, array $params = [])
{
$path = $this->buildRepositoriesPath($workspaceName);
return $this->get($path, $params);
}
public function create(string $workspaceName, string $repoName, array $params = [])
{
$path = $this->buildRepositoriesPath($workspaceName, $repoName);
return $this->post($path, $params);
}
}
After that, I wrote a couple of PHP scripts and another Bash script to run them, and after an hour I could already admire a computer sticking in for me:Checking repository: mysett
Trying to create Git repository: git_mysett
Created Git repository: git_mysett
20abecfb36fe
applying clone bundle from https://api.media.atlassian.com/file/4d5980dc-148f-400c-97f7-8067506778a5/binary?client=403e8d2f-6661-452a-8307-5c68f82c1a13&token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3MiOnsidXJuOmZpbGVzdG9yZTpmaWxE2bUg7aRIzuKbjwOn5SF8IbGEVs33WHDVb-JYto
adding changesets
adding manifests
adding file changes
added 27 changesets with 56 changes to 41 files
finished applying clone bundle
searching for changes
no changes found
updating to branch default
35 files updated, 0 files merged, 0 files removed, 0 files unresolved
Initialized empty Git repository in /home/serge/project/hg2git/src/mysett-git/
pushing to ../mysett-git
searching for changes
adding objects
added 27 commits with 82 trees and 50 blobs
error: Could not remove config section 'remote.origin'
Counting objects: 159, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (133/133), done.
Writing objects: 100% (159/159), 12.95 MiB | 7.95 MiB/s, done.
Total 159 (delta 0), reused 159 (delta 0)
To git@bitbucket.org:lebedevsergey/git_mysett.git
* [new branch] master -> master
Branch master set up to track remote branch master from origin.
Checking repository: jazzz
Trying to create Git repository: git_jazzz
As a result, the conversion took about an hour to work, after which I am now browsing the converted Git repositories in the background to make sure that nothing is lost. And only after that I delete the original Mercurial repositories with my hands. Of course, nothing prevents adding a removal from the Bitbucket Mercurial repository after the conversion to the script, but in my opinion, this is just the case when excessive automation can damage.A ready-made set of scripts for automating repository migration is located hereP.S .:Taking this opportunity, I will raise the rhetorical question - why is Git, which sometimes works with rather unobvious logic, so much more popular than the much more logical Mercurial, in which you just commit, create branches, and if something goes wrong, roll back commits without thinking about what’s under the hood of the version control system, and about the pointer to Head, but just doing what you need? In my opinion, this is unfair, because the obvious is better than the non-obvious, and it is very sad that Bitcurket, one of the pillars of resistance to the mainstream in version control systems, stops supporting Mercurial.