Encoding Problem after Upgrade from 2.x


#1

Hi,

after Upgrading from V 2.x to current 4.3.1 I have an encoding Problem in my Repository Descriptions:
All my Descriptions that contain german umlaut are displayed as if it were Non-UTF8 (like “für” instead of “für”). My Commit-Descriptions (on the same page) are correctly encoded.

Regards,
Kyrodan


#2

Hi,

It looks like database encoding problem. Is your database using UTF8 ?

Best,


#3

I’ve checked this:
It’s all UTF-8. Any other ideas?

mysql> SELECT table_name, column_name, character_set_name FROM information_schema.`COLUMNS`  WHERE table_schema = "rhodecode";
+--------------------------------+-----------------------------------------+--------------------+
| table_name                     | column_name                             | character_set_name |
+--------------------------------+-----------------------------------------+--------------------+
| cache_invalidation             | cache_id                                | NULL               |
| cache_invalidation             | cache_key                               | utf8               |
| cache_invalidation             | cache_args                              | utf8               |
| cache_invalidation             | cache_active                            | NULL               |
| changeset_comments             | comment_id                              | NULL               |
| changeset_comments             | repo_id                                 | NULL               |
| changeset_comments             | revision                                | utf8               |
| changeset_comments             | pull_request_id                         | NULL               |
| changeset_comments             | line_no                                 | utf8               |
| changeset_comments             | hl_lines                                | utf8               |
| changeset_comments             | f_path                                  | utf8               |
| changeset_comments             | user_id                                 | NULL               |
| changeset_comments             | text                                    | utf8               |
| changeset_comments             | created_on                              | NULL               |
| changeset_comments             | modified_at                             | NULL               |
| changeset_comments             | renderer                                | utf8               |
| changeset_comments             | pull_request_version_id                 | NULL               |
| changeset_comments             | display_state                           | utf8               |
| changeset_statuses             | changeset_status_id                     | NULL               |
| changeset_statuses             | repo_id                                 | NULL               |
| changeset_statuses             | user_id                                 | NULL               |
| changeset_statuses             | revision                                | utf8               |
| changeset_statuses             | status                                  | utf8               |
| changeset_statuses             | changeset_comment_id                    | NULL               |
| changeset_statuses             | modified_at                             | NULL               |
| changeset_statuses             | version                                 | NULL               |
| changeset_statuses             | pull_request_id                         | NULL               |
| db_migrate_version             | repository_id                           | utf8               |
| db_migrate_version             | repository_path                         | utf8               |
| db_migrate_version             | version                                 | NULL               |
| external_identities            | provider_name                           | utf8               |
| external_identities            | local_user_id                           | NULL               |
| external_identities            | external_id                             | utf8               |
| external_identities            | external_username                       | utf8               |
| external_identities            | access_token                            | utf8               |
| external_identities            | alt_token                               | utf8               |
| external_identities            | token_secret                            | utf8               |
| gists                          | gist_id                                 | NULL               |
| gists                          | gist_access_id                          | utf8               |
| gists                          | gist_description                        | utf8               |
| gists                          | user_id                                 | NULL               |
| gists                          | gist_expires                            | NULL               |
| gists                          | gist_type                               | utf8               |
| gists                          | created_on                              | NULL               |
| gists                          | modified_at                             | NULL               |
| gists                          | acl_level                               | utf8               |
| groups                         | group_id                                | NULL               |
| groups                         | group_name                              | utf8               |
| groups                         | group_parent_id                         | NULL               |
| groups                         | group_description                       | utf8               |
| groups                         | enable_locking                          | NULL               |
| groups                         | user_id                                 | NULL               |
| groups                         | created_on                              | NULL               |
| integrations                   | integration_id                          | NULL               |
| integrations                   | integration_type                        | utf8               |
| integrations                   | enabled                                 | NULL               |
| integrations                   | name                                    | utf8               |
| integrations                   | settings_json                           | utf8               |
| integrations                   | repo_id                                 | NULL               |
| notifications                  | notification_id                         | NULL               |
| notifications                  | subject                                 | utf8               |
| notifications                  | body                                    | utf8               |
| notifications                  | created_by                              | NULL               |
| notifications                  | created_on                              | NULL               |
| notifications                  | type                                    | utf8               |
| permissions                    | permission_id                           | NULL               |
| permissions                    | permission_name                         | utf8               |
| permissions                    | permission_longname                     | utf8               |
| pull_request_reviewers         | pull_requests_reviewers_id              | NULL               |
| pull_request_reviewers         | pull_request_id                         | NULL               |
| pull_request_reviewers         | user_id                                 | NULL               |
| pull_request_versions          | title                                   | utf8               |
| pull_request_versions          | description                             | utf8               |
| pull_request_versions          | status                                  | utf8               |
| pull_request_versions          | created_on                              | NULL               |
| pull_request_versions          | updated_on                              | NULL               |
| pull_request_versions          | revisions                               | utf8               |
| pull_request_versions          | org_ref                                 | utf8               |
| pull_request_versions          | other_ref                               | utf8               |
| pull_request_versions          | last_merge_org_rev                      | utf8               |
| pull_request_versions          | last_merge_other_rev                    | utf8               |
| pull_request_versions          | merge_status                            | NULL               |
| pull_request_versions          | merge_rev                               | utf8               |
| pull_request_versions          | pull_request_version_id                 | NULL               |
| pull_request_versions          | pull_request_id                         | NULL               |
| pull_request_versions          | org_repo_id                             | NULL               |
| pull_request_versions          | user_id                                 | NULL               |
| pull_request_versions          | other_repo_id                           | NULL               |
| pull_requests                  | pull_request_id                         | NULL               |
| pull_requests                  | title                                   | utf8               |
| pull_requests                  | description                             | utf8               |
| pull_requests                  | status                                  | utf8               |
| pull_requests                  | created_on                              | NULL               |
| pull_requests                  | updated_on                              | NULL               |
| pull_requests                  | user_id                                 | NULL               |
| pull_requests                  | revisions                               | utf8               |
| pull_requests                  | org_repo_id                             | NULL               |
| pull_requests                  | org_ref                                 | utf8               |
| pull_requests                  | other_repo_id                           | NULL               |
| pull_requests                  | other_ref                               | utf8               |
| pull_requests                  | last_merge_org_rev                      | utf8               |
| pull_requests                  | last_merge_other_rev                    | utf8               |
| pull_requests                  | merge_status                            | NULL               |
| pull_requests                  | merge_rev                               | utf8               |
| repo_rhodecode_settings        | repository_id                           | NULL               |
| repo_rhodecode_settings        | app_settings_id                         | NULL               |
| repo_rhodecode_settings        | app_settings_name                       | utf8               |
| repo_rhodecode_settings        | app_settings_value                      | utf8               |
| repo_rhodecode_settings        | app_settings_type                       | utf8               |
| repo_rhodecode_ui              | repository_id                           | NULL               |
| repo_rhodecode_ui              | ui_id                                   | NULL               |
| repo_rhodecode_ui              | ui_section                              | utf8               |
| repo_rhodecode_ui              | ui_key                                  | utf8               |
| repo_rhodecode_ui              | ui_value                                | utf8               |
| repo_rhodecode_ui              | ui_active                               | NULL               |
| repo_to_perm                   | repo_to_perm_id                         | NULL               |
| repo_to_perm                   | user_id                                 | NULL               |
| repo_to_perm                   | permission_id                           | NULL               |
| repo_to_perm                   | repository_id                           | NULL               |
| repositories                   | repo_id                                 | NULL               |
| repositories                   | repo_name                               | utf8               |
| repositories                   | clone_uri                               | utf8               |
| repositories                   | repo_type                               | utf8               |
| repositories                   | user_id                                 | NULL               |
| repositories                   | private                                 | NULL               |
| repositories                   | statistics                              | NULL               |
| repositories                   | downloads                               | NULL               |
| repositories                   | description                             | utf8               |
| repositories                   | created_on                              | NULL               |
| repositories                   | updated_on                              | NULL               |
| repositories                   | landing_revision                        | utf8               |
| repositories                   | enable_locking                          | NULL               |
| repositories                   | locked                                  | utf8               |
| repositories                   | fork_id                                 | NULL               |
| repositories                   | group_id                                | NULL               |
| repositories                   | changeset_cache                         | NULL               |
| repositories                   | repo_state                              | utf8               |
| repositories                   | repo_name_hash                          | utf8               |
| repositories_fields            | repo_field_id                           | NULL               |
| repositories_fields            | repository_id                           | NULL               |
| repositories_fields            | field_key                               | utf8               |
| repositories_fields            | field_label                             | utf8               |
| repositories_fields            | field_value                             | utf8               |
| repositories_fields            | field_desc                              | utf8               |
| repositories_fields            | field_type                              | utf8               |
| repositories_fields            | created_on                              | NULL               |
| rhodecode_settings             | app_settings_id                         | NULL               |
| rhodecode_settings             | app_settings_name                       | utf8               |
| rhodecode_settings             | app_settings_value                      | utf8               |
| rhodecode_settings             | app_settings_type                       | utf8               |
| rhodecode_ui                   | ui_id                                   | NULL               |
| rhodecode_ui                   | ui_section                              | utf8               |
| rhodecode_ui                   | ui_key                                  | utf8               |
| rhodecode_ui                   | ui_value                                | utf8               |
| rhodecode_ui                   | ui_active                               | NULL               |
| statistics                     | stat_id                                 | NULL               |
| statistics                     | repository_id                           | NULL               |
| statistics                     | stat_on_revision                        | NULL               |
| statistics                     | commit_activity                         | NULL               |
| statistics                     | commit_activity_combined                | NULL               |
| statistics                     | languages                               | NULL               |
| user_api_keys                  | user_api_key_id                         | NULL               |
| user_api_keys                  | user_id                                 | NULL               |
| user_api_keys                  | api_key                                 | utf8               |
| user_api_keys                  | description                             | utf8               |
| user_api_keys                  | expires                                 | NULL               |
| user_api_keys                  | created_on                              | NULL               |
| user_api_keys                  | role                                    | utf8               |
| user_email_map                 | email_id                                | NULL               |
| user_email_map                 | user_id                                 | NULL               |
| user_email_map                 | email                                   | utf8               |
| user_followings                | user_following_id                       | NULL               |
| user_followings                | user_id                                 | NULL               |
| user_followings                | follows_repository_id                   | NULL               |
| user_followings                | follows_user_id                         | NULL               |
| user_followings                | follows_from                            | NULL               |
| user_group_user_group_to_perm  | user_group_user_group_to_perm_id        | NULL               |
| user_group_user_group_to_perm  | target_user_group_id                    | NULL               |
| user_group_user_group_to_perm  | permission_id                           | NULL               |
| user_group_user_group_to_perm  | user_group_id                           | NULL               |
| user_ip_map                    | ip_id                                   | NULL               |
| user_ip_map                    | user_id                                 | NULL               |
| user_ip_map                    | ip_addr                                 | utf8               |
| user_ip_map                    | active                                  | NULL               |
| user_ip_map                    | description                             | utf8               |
| user_logs                      | user_log_id                             | NULL               |
| user_logs                      | user_id                                 | NULL               |
| user_logs                      | repository_id                           | NULL               |
| user_logs                      | repository_name                         | utf8               |
| user_logs                      | user_ip                                 | utf8               |
| user_logs                      | action                                  | utf8               |
| user_logs                      | action_date                             | NULL               |
| user_logs                      | username                                | utf8               |
| user_repo_group_to_perm        | group_to_perm_id                        | NULL               |
| user_repo_group_to_perm        | user_id                                 | NULL               |
| user_repo_group_to_perm        | group_id                                | NULL               |
| user_repo_group_to_perm        | permission_id                           | NULL               |
| user_to_notification           | user_id                                 | NULL               |
| user_to_notification           | notification_id                         | NULL               |
| user_to_notification           | read                                    | NULL               |
| user_to_notification           | sent_on                                 | NULL               |
| user_to_perm                   | user_to_perm_id                         | NULL               |
| user_to_perm                   | user_id                                 | NULL               |
| user_to_perm                   | permission_id                           | NULL               |
| user_user_group_to_perm        | user_user_group_to_perm_id              | NULL               |
| user_user_group_to_perm        | user_id                                 | NULL               |
| user_user_group_to_perm        | permission_id                           | NULL               |
| user_user_group_to_perm        | user_group_id                           | NULL               |
| users                          | user_id                                 | NULL               |
| users                          | username                                | utf8               |
| users                          | password                                | utf8               |
| users                          | active                                  | NULL               |
| users                          | admin                                   | NULL               |
| users                          | firstname                               | utf8               |
| users                          | lastname                                | utf8               |
| users                          | email                                   | utf8               |
| users                          | last_login                              | NULL               |
| users                          | ldap_dn                                 | utf8               |
| users                          | api_key                                 | utf8               |
| users                          | inherit_default_permissions             | NULL               |
| users                          | extern_type                             | utf8               |
| users                          | extern_name                             | utf8               |
| users                          | created_on                              | NULL               |
| users                          | user_data                               | NULL               |
| users_group_repo_group_to_perm | users_group_repo_group_to_perm_id       | NULL               |
| users_group_repo_group_to_perm | users_group_id                          | NULL               |
| users_group_repo_group_to_perm | group_id                                | NULL               |
| users_group_repo_group_to_perm | permission_id                           | NULL               |
| users_group_repo_to_perm       | users_group_to_perm_id                  | NULL               |
| users_group_repo_to_perm       | users_group_id                          | NULL               |
| users_group_repo_to_perm       | permission_id                           | NULL               |
| users_group_repo_to_perm       | repository_id                           | NULL               |
| users_group_to_perm            | users_group_to_perm_id                  | NULL               |
| users_group_to_perm            | users_group_id                          | NULL               |
| users_group_to_perm            | permission_id                           | NULL               |
| users_groups                   | users_group_id                          | NULL               |
| users_groups                   | users_group_name                        | utf8               |
| users_groups                   | users_group_active                      | NULL               |
| users_groups                   | users_group_inherit_default_permissions | NULL               |
| users_groups                   | user_id                                 | NULL               |
| users_groups                   | user_group_description                  | utf8               |
| users_groups                   | created_on                              | NULL               |
| users_groups                   | group_data                              | NULL               |
| users_groups_members           | users_group_member_id                   | NULL               |
| users_groups_members           | users_group_id                          | NULL               |
| users_groups_members           | user_id                                 | NULL               |
+--------------------------------+-----------------------------------------+--------------------+
246 rows in set (0,06 sec)

#4

This can be closed. I don’t know why this happened, but I fixed it “by hand”.