Geo: Improve repository acceleration performance for monorepos with high activity (#455870) · Issues · GitLab.org / GitLab · GitLab (original) (raw)
Geo: Improve repository acceleration performance for monorepos with high activity
Release notes
Problem to solve
When Geo is used to accelerate remote teams and CI runners, fetching large mono repos with high levels of activity results from a secondary site, result in a higher-than-normal number of fetch requests being proxied to the primary Geo site. This notably reduces the advantage of deploying a Geo secondary site to deliver a faster experience git fetch experience for remote teams. When used to offload CI traffic from the primary site, it means the offload is less effective for these types of repositories.
The current proxying implementation checks whether the secondary has the latest data by comparing the verification checksums. The verification is performed as an async task which can take an indeterminate amount of time based on the system load. For repositories with high activity the chances of the checksums m
Proposal
Increase the threshold for forwarding repository fetch requests to the primary site. This may involve delaying a request to allow time for the secondary site to catch up with changes.
We should strive to be transparent with the end user (Sasha) about what is happening to their request to avoid the end-user believing their requested operation has hung or failed.
For example, if delays are introduced, we must indicate to the end user that the secondary waiting for changes from the primary. If the request is subsequently forwarded to the primary, this transition should also be communicated to the end user.
Today we indicate to the end user that their request is being forwarded to the primary.
Given customer deployment scenarios and infrastructure setups can differ widely, the threshold should be configurable.
Intended users
Feature Usage Metrics
TBD
Does this feature require an audit event?
No
Edited Apr 12, 2024 by Sampath Ranasinghe