Error with locale when pushing git over ssh

Recenlty I switched to RhodeCode. I installed it on a new Debian Buster virtual machine. RhodeCode is running fine for Mercurial. But when I try to push a git repository over SSH an error occured. Pushing git over https is working well. Also pulling and cloning git over SSH are working.

Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), 222 bytes | 111.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: Traceback (most recent call last): remote: File “hooks/pre-receive”, line 10, in
remote: from vcsserver import hooks
remote: File “/opt/rhodecode/store/gw4159sv035nv616qkpc7m5vhhf9asn8-python2.7-rhodecode-vcsserver-4.17.3/lib/python2.7/site-packages/vcsserver/hooks.py”, line 36, in
remote: from vcsserver.hgcompat import get_ctx
remote: File “/opt/rhodecode/store/gw4159sv035nv616qkpc7m5vhhf9asn8-python2.7-rhodecode-vcsserver-4.17.3/lib/python2.7/site-packages/vcsserver/hgcompat.py”, line 31, in
remote: from mercurial import extensions
remote: File “/opt/rhodecode/store/s4nnxsslk0x3frd51nrnadk5jfh4fh6k-python2.7-mercurial-4.9.1/lib/python2.7/site-packages/mercurial/extensions.py”, line 22, in
remote: from . import (
remote: File “/opt/rhodecode/store/s4nnxsslk0x3frd51nrnadk5jfh4fh6k-python2.7-mercurial-4.9.1/lib/python2.7/site-packages/mercurial/cmdutil.py”, line 23, in
remote: from . import (
remote: File “/opt/rhodecode/store/s4nnxsslk0x3frd51nrnadk5jfh4fh6k-python2.7-mercurial-4.9.1/lib/python2.7/site-packages/mercurial/crecord.py”, line 33, in
remote: locale.setlocale(locale.LC_ALL, u’’)
remote: File “/opt/rhodecode/store/6rjyfgq8yrnzwsc4x9b6mlwi6h9qhay6-python-2.7.15/lib/python2.7/locale.py”, line 581, in setlocale
remote: return _setlocale(category, locale)
remote: locale.Error: unsupported locale setting To ssh://HOSTNAME/GROUP/REPO
! [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs to ‘ssh://USER@HOSTNAME/GROUP/REPO’

Things I already tried:
Download locale archive (https://issues.rhodecode.com/issues/5297#note-12) and add LOCALE_ARCHIVE environment to the .bashrc of the user running RhodeCode, also tried supervisor.ini
Also tried setting LC_ALL=C in supervisor.ini.

Hi,

This is a know issue with never operating systems shipping with later glibc. Please check this:

https://docs.rhodecode.com/RhodeCode-Enterprise/known-issues/known-issues.html#fedora-23-ubuntu-18-04

Hi marcin,

Thanks for your answer.
Made the changes again in supervisor.ini, also tried setting in the start up script. Rebooted the machine. Still getting the error. Is theire away to check if Rhodecode is receiving the new environment variables?

Not really, but it could be ssh issue, could you also post locale command output on your machine ?

We also think possibly in the begining of the file above we could add this as we do for HTTP protocol.

# due to Mercurial/glibc2.27 problems we need to detect if locale settings are
# causing problems and "fix" it in case they do and fallback to LC_ALL = C
try:
    locale.setlocale(locale.LC_ALL, '')
except locale.Error as e:
    log.error(
        'LOCALE ERROR: failed to set LC_ALL, fallback to LC_ALL=C, org error: %s', e)
    os.environ['LC_ALL'] = 'C'

The git client machine is Windows.

chcp = 850

In /opt/rhodecode/store/gw4159sv035nv616qkpc7m5vhhf9asn8-python2.7-rhodecode-vcsserver-4.17.3/lib/python2.7/site-packages/vcsserver/hooks.py I added

import locale
# due to Mercurial/glibc2.27 problems we need to detect if locale settings are
# causing problems and "fix" it in case they do and fallback to LC_ALL = C
try:
    locale.setlocale(locale.LC_ALL, '')
except locale.Error as e:
    log.error('LOCALE ERROR: failed to set LC_ALL, fallback to LC_ALL=C, org error: %s', e)
    os.environ['LC_ALL'] = 'C'

I needed the extra import locale because otherwise it was failing to start. After restarting rhodecode issue still exists.

Tried a lot of environment options but nothing did work. Finaly start changing parts in the codebase.

In /opt/rhodecode/store/s4nnxsslk0x3frd51nrnadk5jfh4fh6k-python2.7-mercurial-4.9.1/lib/python2.7/site-packages/mercurial/crecord.py at line 33 I hardcoded the locale to ‘C’

# This is required for ncurses to display non-ASCII characters in default user
# locale encoding correctly.  --immerrr
locale.setlocale(locale.LC_ALL, u'C')

This seems to be working. So fixed now…

How can I help RhodeCode creating a real fix for this issue?

In git case this is a problem, since Mercurial does this on import level.
Please find a diff that moves the imports to be lazy evaluated and only loaded when Mercurial code runs.


diff --git a/vcsserver/hooks.py b/vcsserver/hooks.py
--- a/vcsserver/hooks.py
+++ b/vcsserver/hooks.py
@@ -28,17 +28,16 @@ import base64
 from httplib import HTTPConnection
 
 
 import mercurial.scmutil
 import mercurial.node
 import simplejson as json
 
 from vcsserver import exceptions, subprocessio, settings
-from vcsserver.hgcompat import get_ctx
 
 log = logging.getLogger(__name__)
 
 
 class HooksHttpClient(object):
     connection = None
 
     def __init__(self, hooks_uri):
@@ -179,16 +178,17 @@ def _extras_from_ui(ui):
 
     extras = {}
     if hook_data:
         extras = json.loads(hook_data)
     return extras
 
 
 def _rev_range_hash(repo, node, check_heads=False):
+    from vcsserver.hgcompat import get_ctx
 
     commits = []
     revs = []
     start = get_ctx(repo, node).rev()
     end = len(repo)
     for rev in range(start, end):
         revs.append(rev)
         ctx = get_ctx(repo, rev)
@@ -198,16 +198,17 @@ def _rev_range_hash(repo, node, check_he
 
     parent_heads = []
     if check_heads:
         parent_heads = _check_heads(repo, start, end, revs)
     return commits, parent_heads
 
 
 def _check_heads(repo, start, end, commits):
+    from vcsserver.hgcompat import get_ctx
     changelog = repo.changelog
     parents = set()
 
     for new_rev in commits:
         for p in changelog.parentrevs(new_rev):
             if p == mercurial.node.nullrev:
                 continue
             if p < start:
@@ -388,16 +389,17 @@ def post_push_ssh(ui, repo, node, **kwar
     Mercurial post_push hook for SSH
     """
     if _extras_from_ui(ui).get('SSH'):
         return post_push(ui, repo, node, **kwargs)
     return 0
 
 
 def key_push(ui, repo, **kwargs):
+    from vcsserver.hgcompat import get_ctx
     if kwargs['new'] != '0' and kwargs['namespace'] == 'bookmarks':
         # store new bookmarks in our UI object propagated later to post_push
         ui._rc_pushkey_branches = get_ctx(repo, kwargs['key']).bookmarks()
     return
 
 
 # backward compat
 log_pull_action = post_pull