Heptapod 1.0 and a roadmap for 2024

Closing the gap with GitLab upstream

Long ago, we set ourselves two requirements for Heptapod 1.0:

• Stop depending on internal Git conversions of Mercurial: This was reached with Heptapod 0.40.
• Be based on a supported GitLab version. This is brought by today's release.

In the past year, we've been improving our semi-automated process to convert and merge upstream GitLab commits, and gradually catching up.

Early last autumn, the upstream merge started running unattended, requiring human intervention only when needed. At this point it was worth making a big effort to land on a supported upstream version as soon as possible. This is where we stand now, with Heptapod 1.0 being based on GitLab 16.6.

Really closing the gap will require more work, and will still be the absolute priority of Heptapod development in the forthcoming months, with the goal of releasing Heptapod in the few days following the corresponding GitLab release.

At this point, our users should expect a monthly release pace. Meanwhile, we will still have to release faster than that.

We hope to release Heptapod 1.1 in the third week of February, at about the same time of upstream dropping support for GitLab 16.6. It will be based on GitLab 16.7, which is also a landmark version for data migrations.

In Heptapod 1.2, we will probably jump again, as far as GitLab 16.10 if possible.

New versioning policy

Starting with GitLab 17.0 (May 16th), we are considering aligning Heptapod minor version numbers (x.y) with GitLab's. This would make sense now that we are leaving the 0.something realm.

We hope this would bring us more clarity, both in development tools and to our users.

So, instead of Heptapod 1.3 or 1.4, we could have 2.0 and keep thereafter a policy that Heptapod x.y is based on GitLab (x+15).y, or even simply Heptapod 17.0 and then Heptapod x.y will always be based on GitLab x.y.

Don't hesitate to tell us what you think about it!

The third number (patch) for bug fixes and security releases will obviously have to stay independent.

Towards Cloud Native Heptapod

The other major work in progress is the transfer to HGitaly of all operations on Mercurial repositories. This is also known as the HGitaly3 milestone.

For historical perspective, the Heptapod project started a few months before GitLab introduced the Gitaly component to encapsulate all interactions with the Git repositories.

The goal of the Gitaly project was to stop requiring the repositories to be present on the file system of the web application (Rails), thus allowing independent horizontal scaling, sharding, easier upgrades without downtime, etc. A good analogy would be to say that Gitaly allows to treat Git repositories as a specialized database.

This brought Cloud Native GitLab (CNG): a distribution of GitLab as a set of containers, easy to deploy and manage, typically in a Kubernetes cluster.

In the development of HGitaly, our counterpart of Gitaly for Mercurial, we've been focusing first on the read-only side of the question. It was indeed what we needed to get rid of the internal conversion to auxiliary Git repositories. This was explained in The road to fully native Mercurial in Heptapod.

The auxiliary Git conversion has always been a reading helper, so we didn't have to change the existing write operations, which continued to work pretty much as back in the days of Heptapod 0.12, by spawning local hg processes. At some point we simply decided that any new operation should be done by HGitaly, to stop the increase of technical debt.

Having HGitaly take charge of all write operations originating from the web application is thus an important gap to close in order to make a cloud-native Heptapod. The other one would be to have HGitaly receive user pushes.

The Heptapod 0.40 release left us free to turn ourselves to this question. We hope to reach the point where HGitaly can be deployed in a separate system also by mid-2024.

For instance, in Heptapod 0.41, the fast-forward Merge Request merge method is handled by HGitaly and Heptapod 1.0 generalizes this to all merge methods. There are feature flags for fallback to the previous way of doing if needed. We expect these to be generally more performant and robust, and also easier on the development side. On the other hand, some tuning might be needed, since they increase the workload of HGitaly.

This leaves repository imports, pull-mirrors and a couple of simple housekeeping methods to be done. It is reasonable to schedule that for Heptapod 1.2.

Using HGitaly as the receptacle for end-user pushes will then be a serious endeavor, and we expect some work to be done at the transport/network level as well.

Mercurial deeper integrations

There is a lot of workflow related things and more generally of deeper integration of Mercurial concepts in the application to be done.

For many of them, it does not really make sense to start development until the major HGitaly work is fully finished, so that it can be done on a clean ground. For instance, advanced behaviours of Merge Requests for stacked topics will need first all the MR logic to be the sole responsibility of HGitaly.

We will thus really turn to them in the second semester of 2024.

Some simple features can be done earlier, though. An example would be to display the phase of a changeset in the commit view. We will not have much bandwidth to spare until we have finished catching up on upstream, but we will consider contributions.

Thanks for reading, we wish you a happy year!