Heptapod 0.40.0 released
Heptapod 0.40.0 (final) was released today. This release saw us reaching an important milestone, as we stopped mirroring Mercurial repositories to Git by default. More about this below.
We're also jumping from GitLab 15.11 to 16.2 and bumping Mercurial to 6.5.
The release candidate was installed on foss.heptapod.net a week ago, the final version will be installed on heptapod.host on 2023-12-05.
Heptapod 0.40.0 can be installed as a Docker image and from source.
The full changelog is available as usual alongside the sources.
Many thanks to all the people involved to make this happen!
End of the mirroring to an internal auxiliary Git repository
Background
In its early prototype, Heptapod was fully relying on converting the Mercurial repository to Git. We then introduced HGitaly, our internal Mercurial content server. HGitaly is designed to be as transparent as possible for other GitLab components, which are meant to interact with Gitaly, GitLab's internal Git handling system. In practice, this means that HGitaly implements the same gRPC protocol as Gitaly.
We have been since then implementing the needed gRPC methods one after the other, first limiting Mercurial content description to changeset hashes, then adding file content and listing capabilities while keeping up to date with evolutions of the Gitaly protocol and planning how we will gradually switch over to using HGitaly instead of an auxiliary Git repository.
More details about this can be read in The road to fully native Mercurial in Heptapod. We have been running some projects with the :hg_without_git feature flag for a while, including Heptapod's own developmenet repository, but HGitaly was lately still lacking a few specialized methods.
Changes in Heptapod 0.40
In this version, we closed the latest gap, namely project license detection, which is of course an expected basic feature of GitLab, and to be fair, to most forges around. This analysis was still being performed by Gitaly, which of course expects a Git repository. We finally added the same functionality to HGitaly, our internal Mercurial content server, using the same detector as Gitaly does, in order to get the same results.
Once that was done, we changed the default value of the :hg_without_git feature flag to true, effectively disabling the conversion to Git by default.
An immediate benefit for native Mercurial projects is a much, much faster push to Heptapod, as the conversion to Git was well known to be the main bottleneck, to the point that we were running all repositories above 100k changesets without it already.
Exceptions
Some projects still undergo conversion to Git, namely:
- "Legacy (hg-git based)" projects. Creation of these has been disabled by default a couple of years ago in favor of "native" Mercurial projects. Existing such projects still have to be explictely converted to native. See the native migration post for detailed explanations about this.
- Native Mercurial projects for which the "Legacy conversion to Git" setting has been selected. The main use case is helping pushing to external Git centric services.
It is also possible to switch back the hg_without_git flag to false, or to define exceptions for certain projects, by using the hg_with_git_fallback feature flag for them. In that case, users should first perform the necessary catch-up to bring the auxiliary Git repository up to date with latest changes in the Mercurial repository. Explanations about this are given in The road to fully native Mercurial in Heptapod.
Future moves
We plan to remove the hg_without_git feature flag completely in Heptapod 0.41. This is actually already the case in the main Heptapod development branch.
Once that is released, this will be the end of a long journey, but that is definitely not the end of the road.
Further HGitaly groundwork will now be focused on implementing write methods that are currently relying on external hg processes so that the Web application does not need to have direct access to the repositories any more. Once that is done, we can enjoy the benefits that Gitaly brought to GitLab: independent scalabilty of components and Cloud Native deployments.
This further goal is also known as the HGitaly3 milestone. We are currently working on the core requirements to let HGitaly perform repository mutating tasks, so keep posted!
At some point later in the future, probably after Heptapod 1.0, we will consider making the migration to native automatic, which should allow us eventually to finally shed the technical debt of the prototyping phase of Heptapod.
Conclusion
Because such an important breakthrough occurred in Heptapod 0.40, we are close to be able to underline it further.
We don't want to make hasty promises, but it looks more and more like there will not be a Heptapod 0.42 release. Whether that is coincidential or the mark of destiny or irrelevant is left to the reader appreciation.
Thank you for reading so far!