Hi,
I’ve migrated RhodeCode from a rccontrol instance to a rcstack one, following the migration documentation at Migration to Docker & Rcstack - RhodeCode rcstack 5.13.0 documentation .
Everything seems to be working, but I am not able to clone large hg repositories.
I tried from the 4.28.0 image to the beta one and the issue remains.
It usually takes more than 30 seconds to clone those large repositories. The 30 seconds seems to be a hard limit in my current configuration.
The command fails with:
% hg clone --verbose https://rhodecode.example.com/a/bc/d "S:\Test\Config"
requesting all changes
adding changesets
adding manifests
adding file changes
transaction abort!
rollback completed
(sent 4 HTTP requests and 2242 bytes; received 336271052 bytes in responses)
abort: stream ended unexpectedly (got 9788 bytes, expected 32768)
[command returned code 255 Tue Jul 9 15:58:46 2024]
From the vccserver logs we can see a worker timeout:
{"timestamp": "2024-07-09T13:58:14.379815+00:00", "levelname": "INFO", "name": "vcsserver.tweens.request_wrapper", "message": "Req[ 8] IP: 127.0.0.1 POST
Request to a/bc/d@/a/bc/d/localrepository time: 0.0325s [mercurial/proto-1.0 (Mercurial 6.5.1)], VCSServer 5.1.0
", "req_id": null, "time": 0.032531023025512695, "ver": "5.1.0", "code": 200, "path": "/a/bc/d", "view_name": "stream_hg", "user_agent
": "mercurial/proto-1.0 (Mercurial 6.5.1)", "vcs_method": "localrepository", "repo_name": "a/bc/d"}
2024-07-09 13:58:44 [11] [CRITICAL] WORKER TIMEOUT (pid:13)
2024-07-09 13:58:44 [13] [INFO] [13 ] worker received SIGABRT signal
2024-07-09 13:58:44 [13] [ERROR] Error handling request
Traceback (most recent call last):
File "/home/rhodecode/venv/lib/python3.11/site-packages/gunicorn/workers/sync.py", line 183, in handle_request
for item in respiter:
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/hgweb/hgweb_mod.py", line 363, in run_wsgi
for r in self._runwsgi(req, res, repo):
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/hgweb/request.py", line 581, in sendresponse
for chunk in self._bodygen:
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/wireprotoserver.py", line 291, in genversion2
for chunk in gen:
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/utils/compression.py", line 688, in compressstream
for chunk in it:
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/bundle2.py", line 740, in getchunks
for chunk in self._compengine.compressstream(
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/bundle2.py", line 763, in _getcorechunk
for chunk in part.getchunks(ui=self.ui):
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/bundle2.py", line 1222, in getchunks
pycompat.raisewithtb(exc, tb)
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/pycompat.py", line 348, in raisewithtb
raise exc.with_traceback(tb)
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/bundle2.py", line 1194, in getchunks
for chunk in self._payloadchunks():
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/bundle2.py", line 1239, in _payloadchunks
chunk = buff.read(preferedchunksize)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/util.py", line 2722, in read
for chunk in self.iter:
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/util.py", line 2693, in splitbig
for chunk in chunks:
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/changegroup.py", line 1643, in generate
for delta in deltas:
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/changegroup.py", line 1197, in deltagroup
for i, revision in enumerate(revisions):
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/utils/storageutil.py", line 536, in emitrevisions
revision = store.rawdata(node)
^^^^^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/revlog.py", line 2784, in rawdata
return self._revisiondata(nodeorrev, raw=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/revlog.py", line 2746, in _revisiondata
rev, rawtext, validated = self._rawtext(node, rev)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/revlog.py", line 2729, in _rawtext
return self._inner.raw_text(node, rev)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/revlog.py", line 1031, in raw_text
bins = self._chunks(chain, targetsize=targetsize)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/revlog.py", line 960, in _chunks
offset, data = self.get_segment_for_revs(firstrev, lastrev)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/revlog.py", line 872, in get_segment_for_revs
return start, self._segmentfile.read_chunk(start, length)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/revlogutils/randomaccessfile.py", line 194, in read_chunk
return self._read_and_update_cache(offset, length)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/revlogutils/randomaccessfile.py", line 205, in _read_and_update_cache
with self._read_handle() as file_obj:
File "/usr/lib/python3.11/contextlib.py", line 137, in __enter__
return next(self.gen)
^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/revlogutils/randomaccessfile.py", line 151, in _read_handle
with self._open() as fp:
^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/revlogutils/randomaccessfile.py", line 134, in _open
return self.opener(self.filename, mode=mode)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/store.py", line 1104, in __call__
return self.vfs(encoded, mode, *args, **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/vfs.py", line 495, in __call__
f = self.join(path)
^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/vfs.py", line 592, in join
return self._join(*parts)
^^^^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/mercurial/vfs.py", line 165, in _join
for idx, p in enumerate(paths):
^^^^^^^^^^^^^^^^
File "/home/rhodecode/venv/lib/python3.11/site-packages/gunicorn/workers/base.py", line 203, in handle_abort
sys.exit(1)
SystemExit: 1
The vcsserver.ini is the one provided by the ./rcstack init
and rhodecode.ini is also the one provided by this command, minus the backported values pointed by the migration documentation.
Did this happen before, or did I miss anything in my migration?
Also, I’m not sure how to tweak this gunicorn timeout from the rhodecode.ini or the vcsserver.ini file to see if this can solve the clone issue.
Thanks for any help.