We have recently migrated to rcstack, first to rhodecode 4.28 and then to rhodecode 5.5.1. We are now having issues with pushing large files or commits with a large number of files to mercurial repos that are public.
The error we get is “abort: error: An existing connection was forcibly closed by the remote host”
We have the large files extension enabled and this worked prior to the upgrade. This problem does not happen if we set the repository to private. Additionally this does not seem to be an issue with git repositories
Hi! Can you provide logs from vcsserver and rhodecode container when you reproduce this issue? Are there any exceptions? Thanks!
the pattern is that only public repos have this issue? for private repos all is good, correct? Can you test and confirm this?
hg push --verbose --debug
can you run push with those flags?
other things to try - make sure instance have enough ram, large files sometimes require significant amount of RAM to process
check you local hgrc config
if this is pushed via ssh - make sure you switch to v2 version of ssh wrapper in .ini config
ssh.wrapper_cmd.v2 = /usr/local/bin/rhodecode_bin/bin/rc-ssh-wrapper-v2
try to run same push via http and see what happens
We have still been unable to come up with a true solution to this issue and have been having to work around it
- Currently running Rhodecode 5.5.1 via rcstack, will look to upgrade to the latest though nothing in the changelog indicates a fix to this issue
- All repos are using https (we have never used ssh in our environment prior to the upgrade from 4.20, In fact attempting ssh even with the v2 wrapper has not been successful at all
- The issue is only with out mercurial repos and not our git repos
- It seems to not only be large files but large commits
- It more looks like an authentication issue where on larger files / commits rhodecode is asking mercurial for authentication well after it is expecting it
- See the following example with --verbose and --debug. The first push works, asking for credentials for the push but the second one fails before credentials are asked for
~/projects$ hg clone https://rhodecode/sammercurialtest3
destination directory: sammercurialtest3
no changes found
updating to branch default
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
~/projects$ cd sammercurialtest3/
~/projects/sammercurialtest3$ cp ../testfile2M .
~/projects/sammercurialtest3$ hg add testfile2M
~/projects/sammercurialtest3$ hg commit -m "Added 2M File"
~/projects/sammercurialtest3$ hg push --verbose --debug
pushing to https://rhodecode/sammercurialtest3
using https://rhodecode/sammercurialtest3
sending capabilities command
query 1; heads
sending batch command
searching for changes
1 total queries in 0.1306s
preparing listkeys for "phases"
sending listkeys command
received listkey for "phases": 15 bytes
checking for updated bookmarks
preparing listkeys for "bookmarks"
sending listkeys command
received listkey for "bookmarks": 0 bytes
1 changesets found
list of changesets:
2f57374c5eb6e24404f8e7e25651a21159a41c76
bundle2-output-bundle: "HG20", 4 parts total
bundle2-output-part: "replycaps" 210 bytes payload
bundle2-output-part: "check:heads" streamed payload
bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
bundle2-output-part: "phase-heads" 24 bytes payload
sending unbundle command
sending 2098371 bytes
http authorization required for https://rhodecode/sammercurialtest3
realm: TGAP RhodeCode
user: sam.ashton
password:
http auth: user sam, password *************
bundle2-input-bundle: no-transaction
bundle2-input-part: "reply:changegroup" (advisory) (params: 0 advisory) supported
bundle2-input-part: "output" (advisory) supported
bundle2-input-part: total payload size 211
remote: RhodeCode: `sammercurialtest3` size summary .hg:2.0 MiB repo:0 B total:2.0 MiB
remote: RhodeCode: open pull request link: https://rhodecode/sammercurialtest3/pull-request/new?branch=default
remote: RhodeCode: push completed
bundle2-input-bundle: 2 parts total
preparing listkeys for "phases"
sending listkeys command
received listkey for "phases": 15 bytes
(sent 7 HTTP requests and 4199024 bytes; received 911 bytes in responses)
~/projects/sammercurialtest3$ cp ../SamTestFiles/testfile10M .
~/projects/sammercurialtest3$ hg add testfile10M
~/projects/sammercurialtest3$ hg commit -m "Added 10M File"
~/projects/sammercurialtest3$ hg push --verbose --debug
pushing to https://rhodecode/sammercurialtest3
using https://rhodecode/sammercurialtest3
sending capabilities command
query 1; heads
sending batch command
searching for changes
all remote heads known locally
preparing listkeys for "phases"
sending listkeys command
received listkey for "phases": 15 bytes
checking for updated bookmarks
preparing listkeys for "bookmarks"
sending listkeys command
received listkey for "bookmarks": 0 bytes
sending branchmap command
preparing listkeys for "bookmarks"
sending listkeys command
received listkey for "bookmarks": 0 bytes
1 changesets found
list of changesets:
738e1b8305e9b4e24cdfb1638deb64761db575cd
bundle2-output-bundle: "HG20", 5 parts total
bundle2-output-part: "replycaps" 210 bytes payload
bundle2-output-part: "check:phases" 24 bytes payload
bundle2-output-part: "check:updated-heads" streamed payload
bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
bundle2-output-part: "phase-heads" 24 bytes payload
sending unbundle command
sending 10488070 bytes
(sent 6 HTTP requests and 1703 bytes; received 641 bytes in responses)
abort: error: EOF occurred in violation of protocol (_ssl.c:2406)
~/projects/sammercurialtest3$