LFS files are uploaded repeatedly during push (#31871) · Issues · GitLab.org / GitLab FOSS · GitLab (original) (raw)
LFS files are uploaded repeatedly during push
Summary
When uploading some large files (around 1 GB) to our Gitlab CE installation, the upload is repeated in an endless loop.
Similar issues where reported as referenced below, but are either closed, reported on other projects or may be different problems, especially the gitlab.com issue looks very similar.
- http://stackoverflow.com/questions/40136346/how-can-i-increase-gitlab-ce-lfs-file-size-limitation-as-to-not-get-500-server-e
- https://gitlab.com/gitlab-com/support-forum/issues/1621
- #21312 (closed)
- #22428 (closed)
- #29466 (closed)
Steps to reproduce
Add 1 GB LFS file to git repo. Push.
-> The upload is repeated several times until some timeouts are reached. On our network, one iteration takes less than 30 seconds.
What is the current bug behavior?
The upload of the LFS file is repeated in an endless loop. Eventually the push fails after uploading the file several times.
When setting client_body_in_file_only on
in nginx, I can verify that the file is uploaded completely to nginx. While supervising the tmp/upload dir, I can see that copying is aborted at around 300-400 MB.
What is the expected correct behavior?
The file is uploaded once and the push completed successfull.
Relevant logs and/or screenshots
2017-05-04_18:14:21.96002 2017/05/04 20:14:21 error: PUT "/X/Y.git/gitlab-lfs/objects/43362bb96598298e3967b8166ef742e48db68acecc0e8f2b5453c3e260028b0d/1073532906": handleStoreLfsObject: copy body to tempfile: unexpected EOF
2017-05-04_18:14:22.32222 git.X.eu @ - - [2017-05-04 20:14:12.070299762 +0200 CEST] "PUT /X/Y.git/gitlab-lfs/objects/43362bb96598298e3967b8166ef742e48db68acecc0e8f2b5453c3e260028b0d/1073532906 HTTP/1.1" 500 22 "" "git-lfs/2.1.0 (GitHub; linux amd64; go 1.8.1)" 10.251602
Output of checks
Results of GitLab environment info
System information System: Debian 8.7 Current User: git Using RVM: no Ruby Version: 2.3.3p222 Gem Version: 2.6.6 Bundler Version:1.13.7 Rake Version: 10.5.0 Redis Version: 3.2.5 Git Version: 2.11.1 Sidekiq Version:4.2.7
GitLab information Version: 9.1.2 Revision: df1403f Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: https://git.X.eu HTTP Clone URL: https://git.X.eu/some-group/some-project.git SSH Clone URL: git@git.X.eu:some-group/some-project.git Using LDAP: no Using Omniauth: no
GitLab Shell Version: 5.0.2 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
sudo gitlab-rake gitlab:check SANITIZE=true
Checking GitLab Shell ...
GitLab Shell version >= 5.0.2 ? ... OK (5.0.2) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:git? default... yes Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 5/5 ... ok [...] 4/925 ... ok Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Access to /var/opt/gitlab/.ssh/authorized_keys: OK Send ping to redis server: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Reply by email ...
Reply by email is disabled in config/gitlab.yml
Checking Reply by email ... Finished
Checking LDAP ...
LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab ...
Git configured with autocrlf=input? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config outdated? ... no Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory setup correctly? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) projects have namespace: ... 5/5 ... yes [...] 4/925 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.1.0 ? ... yes (2.3.3) Your git bin path is "/opt/gitlab/embedded/bin/git" Git version >= 2.7.3 ? ... yes (2.11.1) Active users: 11
Checking GitLab ... Finished
Edited Jan 05, 2023 by 🤖 GitLab Bot 🤖