[svk-devel] Problem syncing mirror

[Jason Dillon]

FYI, there is also a more general problem when using svk on many  
Apache svn repos whre the repos contain large revisions.  I believe  
the svn.apache.org httpd might be configured to reject some large  
transfers to avoid DoS attacks, though I'm not 100% sure this is the  
problem.

<snip>
RA layer request failed: REPORT request failed on '/repos/asf/!svn/ 
vcc/default': REPORT of '/repos/asf/!svn/vcc/default': Could not read  
chunk delimiter: Secure connection truncated (https://svn.apache.org)
</snip>

I do know however that I *can not* use SVK with any of these repos  
because of this:

    https://svn.apache.org/repos/asf/geronimo/
    https://svn.apache.org/repos/asf/geronimo/server/
    https://svn.apache.org/repos/asf/geronimo/server/trunk

NOTE: http fails exactly the same as https in a RA layer request.

Its been a pain for quite some time, and consequently I can't use SVK  
with these Apache projects :-(  Which sucks, cause I really need SVK  
smerge to help manage all of the dang branches we have.

It is possible to use SVK with some svn.apache.org repo paths, but  
many which have large revisions simply kill the SVK sync process, re- 
sync does nothing to help.

I've tried requesting a `svnadmin dump` but since the ASF uses one  
huge SVN repo for everything (well *almost* everything), its non- 
trivial to get a dump which contains just the projects I'm interested  
in.

When asking for status about the dump again from ASF infra William A.  
Rowe, Jr. <wrowe at rowe-clan.net>, mentioned something about new SVN  
sync features in 1.4 which might help solve this problem:

On Feb 10, 2007, at 8:21 AM, William A. Rowe, Jr. wrote:
> Jason Dillon wrote:
>> Is there a new API or is it just the new svnsync binary?
>
> From 1.4 new features, svnsync entry notes...
>
> """
> A new tool — svnsync — is now installed as part of the standard  
> distribution.
> This tool provides the ability to replicate history from one  
> repository to
> another. The replication can happen all at once, or can be done  
> incrementally
> through repeated 'sync' operations. Because the tool uses the  
> abstract network
> (RA) API, the source and destination repositories can be either  
> local, remote,
> or any combination thereof.
>
> Compatibility note: in order to "push" information into a destination
> repository, any version of the server will suffice. The pushing is  
> done through
> ordinary network commits. To "pull" history from the source  
> repository, however,
> requires a 1.4 (or later) server.
> """
>
> So it's both, the RA API, and the ability do do smaller syncs which  
> would solve
> the issue you have today with svk.  I'm certain there would be  
> porting work for
> the svk authors to pick up the RA API.

It would really be helpful if SVK could handle sync'ing repos with  
large revisions better.  Perhaps using the new 1.4 sync support in  
SVN might be more efficient/friendly on the httpd and it won't kill  
the connection too?

Can anyone confirm there are new sync API's in SVN 1.4 which can be  
used by SVK to get more efficient mirror syncing?

--jason