MySQL Upgrade Issue

myjm · February 19, 2024, 6:40am

Dear, Support Person.

We are using RhodeCode 4.27.1 Community Edition and utilizing MySQL 5.7.44 on AWS RDS as our backend database. Due to the support expiration of RDS, we upgraded MySQL to 8.0.36, after which RhodeCode could no longer connect to the database.

community.log

Traceback (most recent call last):
File “/opt/rhodecode/store/f3cphs7x07gnqgv9iiz6y2chglc63bxk-python2.7-gunicorn-19.9.0/lib/python2.7/site-packages/gunicorn/arbiter.py”, line 583, in spawn_worker
worker.init_process()
File “/opt/rhodecode/store/f3cphs7x07gnqgv9iiz6y2chglc63bxk-python2.7-gunicorn-19.9.0/lib/python2.7/site-packages/gunicorn/workers/base.py”, line 129, in init_process
self.load_wsgi()
File “/opt/rhodecode/store/f3cphs7x07gnqgv9iiz6y2chglc63bxk-python2.7-gunicorn-19.9.0/lib/python2.7/site-packages/gunicorn/workers/base.py”, line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File “/opt/rhodecode/store/f3cphs7x07gnqgv9iiz6y2chglc63bxk-python2.7-gunicorn-19.9.0/lib/python2.7/site-packages/gunicorn/app/base.py”, line 67, in wsgi
self.callable = self.load()
File “/opt/rhodecode/store/f3cphs7x07gnqgv9iiz6y2chglc63bxk-python2.7-gunicorn-19.9.0/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py”, line 50, in load
return self.load_pasteapp()
File “/opt/rhodecode/store/f3cphs7x07gnqgv9iiz6y2chglc63bxk-python2.7-gunicorn-19.9.0/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py”, line 46, in load_pasteapp
return load_pasteapp(self.cfgurl, self.relpath, global_conf=self.cfg.paste_global_conf)
File “/opt/rhodecode/store/f3cphs7x07gnqgv9iiz6y2chglc63bxk-python2.7-gunicorn-19.9.0/lib/python2.7/site-packages/gunicorn/app/pasterapp.py”, line 71, in load_pasteapp
global_conf=global_conf)
File “/opt/rhodecode/store/l0sh0hkgzpqlpd3w9j4zs68gnj3nzhzf-python2.7-pastedeploy-2.1.0/lib/python2.7/site-packages/paste/deploy/loadwsgi.py”, line 253, in loadapp
return loadobj(APP, uri, name=name, kw)
File “/opt/rhodecode/store/l0sh0hkgzpqlpd3w9j4zs68gnj3nzhzf-python2.7-pastedeploy-2.1.0/lib/python2.7/site-packages/paste/deploy/loadwsgi.py”, line 278, in loadobj
return context.create()
File “/opt/rhodecode/store/l0sh0hkgzpqlpd3w9j4zs68gnj3nzhzf-python2.7-pastedeploy-2.1.0/lib/python2.7/site-packages/paste/deploy/loadwsgi.py”, line 715, in create
return self.object_type.invoke(self)
File “/opt/rhodecode/store/l0sh0hkgzpqlpd3w9j4zs68gnj3nzhzf-python2.7-pastedeploy-2.1.0/lib/python2.7/site-packages/paste/deploy/loadwsgi.py”, line 235, in invoke
filtered = context.next_context.create()
File “/opt/rhodecode/store/l0sh0hkgzpqlpd3w9j4zs68gnj3nzhzf-python2.7-pastedeploy-2.1.0/lib/python2.7/site-packages/paste/deploy/loadwsgi.py”, line 715, in create
return self.object_type.invoke(self)
File “/opt/rhodecode/store/l0sh0hkgzpqlpd3w9j4zs68gnj3nzhzf-python2.7-pastedeploy-2.1.0/lib/python2.7/site-packages/paste/deploy/loadwsgi.py”, line 152, in invoke
return fix_call(context.object, context.global_conf, context.local_conf)
File “/opt/rhodecode/store/l0sh0hkgzpqlpd3w9j4zs68gnj3nzhzf-python2.7-pastedeploy-2.1.0/lib/python2.7/site-packages/paste/deploy/util.py”, line 55, in fix_call
val = callable(*args, kw)
File “/opt/rhodecode/store/8mliv4kj7db3qz4jmy3nyca7v7r6kav0-python2.7-rhodecode-enterprise-ce-4.27.1/lib/python2.7/site-packages/rhodecode/config/middleware.py”, line 112, in make_pyramid_app
load_pyramid_environment(global_config, settings)
File “/opt/rhodecode/store/8mliv4kj7db3qz4jmy3nyca7v7r6kav0-python2.7-rhodecode-enterprise-ce-4.27.1/lib/python2.7/site-packages/rhodecode/config/environment.py”, line 84, in load_pyramid_environment
rhodecode.CONFIG[‘default_user_id’] = utils.get_default_user_id()
File “/opt/rhodecode/store/8mliv4kj7db3qz4jmy3nyca7v7r6kav0-python2.7-rhodecode-enterprise-ce-4.27.1/lib/python2.7/site-packages/rhodecode/config/utils.py”, line 99, in get_default_user_id
.filter(User.username == User.DEFAULT_USER)
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/query.py”, line 3391, in scalar
ret = self.one()
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/query.py”, line 3358, in one
ret = self.one_or_none()
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/query.py”, line 3327, in one_or_none
ret = list(self)
File “/opt/rhodecode/store/8mliv4kj7db3qz4jmy3nyca7v7r6kav0-python2.7-rhodecode-enterprise-ce-4.27.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py”, line 93, in iter
return super_.iter()
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/query.py”, line 3403, in iter
return self._execute_and_instances(context)
File “/opt/rhodecode/store/8mliv4kj7db3qz4jmy3nyca7v7r6kav0-python2.7-rhodecode-enterprise-ce-4.27.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py”, line 118, in execute_and_instances
return super._execute_and_instances(context)
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/query.py”, line 3425, in _execute_and_instances
querycontext, self._connection_from_session, close_with_result=True
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/query.py”, line 3440, in _get_bind_args
mapper=self._bind_mapper(), clause=querycontext.statement, kw
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/query.py”, line 3418, in _connection_from_session
conn = self.session.connection(kw)
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py”, line 1133, in connection
execution_options=execution_options,
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py”, line 1139, in _connection_for_bind
engine, execution_options
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py”, line 432, in _connection_for_bind
conn = bind._contextual_connect()
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 2251, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 2289, in _wrap_pool_connect
e, dialect, self
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 1555, in handle_dbapi_exception_noconnection
sqlalchemy_exception, with_traceback=exc_info[2], from=e
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 2285, in _wrap_pool_connect
return fn()
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/pool/base.py”, line 363, in connect
return _ConnectionFairy._checkout(self)
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/pool/base.py”, line 773, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/pool/base.py”, line 492, in checkout
rec = pool._do_get()
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/pool/impl.py”, line 139, in _do_get
self._dec_overflow()
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py”, line 69, in exit
exc_value, with_traceback=exc_tb,
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/pool/impl.py”, line 136, in _do_get
return self._create_connection()
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/pool/base.py”, line 308, in _create_connection
return _ConnectionRecord(self)
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/pool/base.py”, line 437, in init
self.__connect(first_connect_check=True)
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/pool/base.py”, line 657, in __connect
pool.logger.debug(“Error on connect(): %s”, e)
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py”, line 69, in exit**
exc_value, with_traceback=exc_tb,
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/pool/base.py”, line 652, in __connect
connection = pool._invoke_creator(self)
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py”, line 114, in connect
return dialect.connect(*cargs, **cparams)
File “/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/default.py”, line 488, in connect
return self.dbapi.connect(*cargs, cparams)
File “/opt/rhodecode/store/qy6y6npssdfx02kg1cg66lgazr98lx1l-python2.7-mysql-python-1.2.5/lib/python2.7/site-packages/MySQLdb/init**.py”, line 81, in Connect
return Connection(*args, kwargs)
File “/opt/rhodecode/store/qy6y6npssdfx02kg1cg66lgazr98lx1l-python2.7-mysql-python-1.2.5/lib/python2.7/site-packages/MySQLdb/connections.py”, line 193, in init**
super(Connection, self).init(*args, **kwargs2)
OperationalError: (_mysql_exceptions.OperationalError) (2019, “Can’t initialize character set unknown (path: compiled_in)”)
(Background on this error at: Error Messages — SQLAlchemy 2.0 Documentation)

Changing the character set configuration of MySQL to either utf8mb4 or utf8mb3 does not resolve the connection issue. Is RhodeCode 4.27.1 Community Edition not compatible with MySQL 8.0?

rhodecode-support · February 19, 2024, 8:08am

Hi !

Have you tried to set this into the connection string in rhodecode.ini ?

sqlalchemy.db1.url = mysql://secret:pass@localhost/rhodecode?charset=utf8

myjm · February 28, 2024, 1:39pm

Thank you for your response. By adding “charset=utf8”, I was successfully able to connect to MySQL 8.0.36.

After the update, the following warnings appeared in the MySQL log:

[Warning] [MY-011825] [InnoDB] Cannot add field last_merge_metadata in table rhodecode.pull_requests because after adding it, the row size is 8368 which is greater than maximum allowed size (8126) for a record on index leaf page.

[Warning] [MY-011825] [InnoDB] Cannot add field last_merge_metadata in table rhodecode.pull_request_versions because after adding it, the row size is 8372 which is greater than maximum allowed size (8126) for a record on index leaf page.

Therefore, I executed the following commands:
ALTER TABLE pull_requests ROW_FORMAT=DYNAMIC;
ALTER TABLE pull_request_versions ROW_FORMAT=DYNAMIC;

it seems there have been no warnings since then. Thank you very much.