Import SVN - never completes

1

Hi

I am trying to import one of my smaller SVN repos - 9GB

After several hours progress is stuck on 7GB :frowning: - how can I troubleshoot this, and is there a way to speed up the process in general? - I am using the import feature via svn URL when creating the rhodecode repo

Thanks

LOGS:

2017-03-28 03:42:33.241 INFO [rhodecode.model.repo] creating repo game3 in /mnt/repos/game3 from url: svn://192.168.183.105/game3
[2017-03-28 04:39:36 +0000] [1988] [CRITICAL] WORKER TIMEOUT (pid:2066)
[2017-03-28 04:39:36 +0000] [2066] [INFO] [<2066 >] worker received SIGABRT signal
2017-03-28 04:39:36.936 INFO [rhodecode.lib.middleware.request_wrapper] IP: 192.168.183.1 Request to /_admin/repos time: 21600.936s
[2017-03-28 04:39:36 +0000] [2066] [INFO] Worker exiting (pid: 2066)
[2017-03-28 04:39:37 +0000] [2972] [INFO] Booting worker with pid: 2972
[2017-03-28 04:39:37 +0000] [2972] [INFO] [<2972 >] WORKER spawned
2017-03-28 04:39:38.189 INFO [rhodecode.model] initializing db for sqlite:////root/.rccontrol/community-1/rhodecode.db
2017-03-28 04:39:38.189 INFO [rhodecode.config.environment] Enabled VCS backends: [‘git’, ‘hg’, ‘svn’]
2017-03-28 04:39:38.985 INFO [rhodecode.lib.auth] getting information about all available permissions
2017-03-28 05:33:20.045 INFO [rhodecode.lib.base] IP: 192.168.183.113 User: <AuthUser(‘id:2[admin] ip:192.168.183.113 auth:True’)> accessed / [home.index]

<<< Started new transaction, based on original revision 2769
* editing path : trunk/Proto … done.
[2017-03-28 04:39:36 +0000] [1987] [CRITICAL] WORKER TIMEOUT (pid:2061)
[2017-03-28 04:39:36 +0000] [2061] [INFO] [<2061 >] worker received SIGABRT signal
2017-03-28 04:39:36.916 INFO [vcsserver.tweens] IP: 127.0.0.1 Request to /svn time: 21600.751s
[2017-03-28 04:39:36 +0000] [2061] [INFO] Worker exiting (pid: 2061)

------- Committed revision 2769 >>>

[2017-03-28 04:39:36 +0000] [2971] [INFO] Booting worker with pid: 2971
[2017-03-28 04:39:36 +0000] [2971] [INFO] [<2971 >] WORKER spawned
2017-03-28 04:39:37.485 INFO [vcsserver.http_main] Settings locale: LC_ALL to en_US.UTF-8
2017-03-28 04:39:37.485 INFO [vcsserver.http_main] Locale set to language code “en_US” with encoding “UTF-8”.
2017-03-28 04:39:37.485 INFO [vcsserver.http_main] Initializing beaker cache: {‘enabled’: True, ‘expire’: None, ‘data_dir’: None, ‘cache_regions’: {‘repo_object’: {‘key_length’: 250, ‘lock_dir’: None, ‘expire’: 300, ‘type’: ‘memorylru’, ‘enabled’: True, ‘data_dir’: None, ‘max_items’: ‘100’}}, ‘type’: ‘memory’, ‘log_file’: None}

2

Hi,

The remote import is not the most efficient one unfortunetly. However RhodeCode can pick any repository if it’s copied into it’s storage.

So best way would be tu use svnadmin and produce a repository dump and move it over to the repository storage, and run remap&rescan from admin > settings

Here are some ways to move a svn repo over: http://stackoverflow.com/questions/360758/moving-a-subversion-repository-to-another-server

Hope that helps.

3

Thanks I used the svnadmin dump to migrate the repository. But I guess there is a bug with the builtin import function - it never completes.

Felix

4

I think it might be a timeout, or maybe some other limit you have hit when reaching 7gb…

we test this which much smaller data-sets to make it faster, afair we did upto ~2gb import and it went ok.

we’ll double check maybe at some point if there’s no other problems, or maybe increase the timeout

5

Ok I also tested it with the Enterprise version , same result.

But know you know :slight_smile:

Thanks

6

Thanks for reporting :wink:

feedback is gold, please send always if you have any (especially for SVN)

Cheers