[svk-devel] [PATCH] better progress information for sync

[Michael Brouwer]

In trunk I believe that retriving logs and revisions are pipelined (or will
be soon).  That something you might want to take into consideration, since
you'd have to update 2 status lines at once.  Pehaps the status for logs and
commits can be folded into one line somehow. Something like this might work
I suppose (though the wording could be changed):

Syncing from file://localhost/home/jon/test_parrot
Metadata 50% (5032/10456) data 25% (2541/10436)

Michael

On 12/18/06, Jonathan Rockway <jon at jrock.us> wrote:
>
> I tried git for the first time this weekend, and while I won't be
> switching from svk to git any time soon, I did like the progress
> indication that's output during "git clone".  So, I've added it to
> SVN::Mirror (patch on RT), and SVK::Mirror in trunk of svk.
>
Instead of:
> svk sync //test_parrot/mirror
> Syncing file://localhost/home/jon/test_parrot
> Retrieving log information from 106 to 10561
>   <really long wait with no feedback>
> Committed revision 107 from 106
> Committed revision 108 from 107
>   <snip 10000 lines of useless information>
>
> You get:
> Syncing file://localhost/home/jon/test_parrot
> Retrieving log information from 106 to 10561
>   100% (10456/10456)
> Downloading revision information...
>    29% (3070/10456)
>
> My patch isn't without problems, though.  The tests that expect
> "Committed ## from ##" all fail, obviously, since that's not printed
> anymore.  I'd like to somehow add a log level called "progress" and then
> let the user pick whether he wants the verbose information, or if he
> wants a nice progress bar instead.  (people running pullmi [or is that
> pushyu; i'm confused] will not want the progress bar, for example).  I'm
> not sure how to best do this, though.
>
> Also, the callback that prints the progress meter after "Downloading
> revision number" can't currently get at $torev if it's "HEAD", so I
> added a nasty hack to change that.  However, it can easily be made less
> nasty if this is the best way to do it (add a torev accessor, and then
> pass it to the callback as my patch does, sort of).
>
> Finally, the "Retrieving log information from 1 to 10561
> 100% (10561/10561)
> Downloading revision information..." part is specific to the RA backend,
> which seems non-optimal to me (but I can live with it for now, because
> it's the only backend).
>
> Anyway, suggestions welcome.  If people like this, I'm going to add the
> same thing to checkout (for the initial checkout), because nobody ever
> reads the list of files from svk checkout anyway (but they do want to
> know how long it's going to take).  After that's done, I plan to add a
> "clone" operation, so you can write:
>
> $ svk clone http://svn.somewhere.com/repos/foo
>
> and have svk do:
> svk mirror //mirror/foo http://svn.somewhere.com/repos/foo
> svk sync //mirror/foo
> svk copy //mirror/foo //local/foo
> svk co //local/foo
>
> The output will look like:
>
> Syncing http://svn.somewhere.com/repos/foo
> Mirroring http://svn.somewhere.com/repos/foo to //mirror/foo
> Retrieving log information from 1 to 10561
> 100% (10561/10561)
> Downloading revision information...
> 100% (10561/10561)
> Creating local branch...
> Checking out a working copy...
> 100% (1234/1234)
> $ cd foo
> $ # start your work
>
> Yes, this is completely stolen from git, but it's a good idea :)
> Comments welcome (especially about the //mirror and //local naming
> conventions) before I start hacking on this.
>
> Regards,
> Jonathan Rockway
>
> --
> package JAPH;use Catalyst qw/-Debug/;($;=JAPH)->config(name => do {
> $,.=reverse qw[Jonathan tsu rehton lre rekca Rockway][$_].[split //,
> ";$;"]->[$_].q; ;for 1..4;$,=~s;^.;;;$,});$;->setup;
>
>
> ==== Patch <svk_sync_progress> level 2
> Source: d7608cd0-831c-0410-93c0-e5b306c3c028:/local/svk:13011
> Target: 4ccb8a5b-8ae0-0310-9b02-86531366991d:/:2240
>         (http://code.bestpractical.com/svk)
> Log:
> r12556 at foo:  jon | 2006-12-16 19:20:44 -0600
> local branch of svk
> r13010 at foo:  jon | 2006-12-18 14:49:19 -0600
> added progress bar for svn sync; both log info and revision commits
> r13011 at foo:  jon | 2006-12-18 16:04:35 -0600
> sync meter off-by-one, and also
> off-by-looking-at-the-wrong-counter.  fixed.
>
> === trunk/lib/SVK/Mirror.pm
> ==================================================================
> --- trunk/lib/SVK/Mirror.pm     (revision 2240)
> +++ trunk/lib/SVK/Mirror.pm     (patch svk_sync_progress level 2)
> @@ -363,15 +363,27 @@
>
>      $logger->info(loc("Syncing %1",
> $self->url).($self->_backend->_relayed ? loc(" via %1", $self->server_url) :
> ""));
>
> +    my $fromrev = $self->fromrev;
> +    $self->_show_progress(0, $torev-$fromrev+1) if $torev;
>      $self->mirror_changesets($torev,
>          sub {
> -            my ( $changeset, $rev ) = @_;
> -            $logger->info("Committed revision $rev from revision
> $changeset.");
> +           my ( $changeset, $rev, $torev ) = @_;
> +           #$logger->info("Committed revision $rev from revision
> $changeset.");
> +           $self->_show_progress($changeset-$fromrev, $torev-$fromrev);
>          }
>      );
> +    print "\n";
>      die $@ if $@;
> }
>
> +sub _show_progress {
> +    my $self = shift;
> +    my ($current, $to) = @_;
> +    local $| = 1;
> +    $to ||= 1;
> +    printf " %3d%% (%d/%d)\r", int(100*$current/$to), $current, $to;
> +}
> +
> sub sync_snapshot {
>      my ($self, $snapshot) = @_;
>      $logger->warn(loc("
> === trunk/lib/SVK/Mirror/Backend/SVNRa.pm
> ==================================================================
> --- trunk/lib/SVK/Mirror/Backend/SVNRa.pm       (revision 2240)
> +++ trunk/lib/SVK/Mirror/Backend/SVNRa.pm       (patch svk_sync_progress
> level 2)
> @@ -287,7 +287,7 @@
>
>      my $err;
>      {
> -        local $SVN::Error::handler = sub { $err = $_[0]; die \'error
> handled' };
> +        local $SVN::Error::handler = sub { $err = $_[0]; die \'error
> handled' }; #'}; # pacify emacs
>          if ( eval { $ra->replay( 0, 0, 0, SVK::Editor->new ); 1 } ) {
>              $self->_ra_finished($ra);
>              return $self->_has_replay(1);
> @@ -408,15 +408,19 @@
>      my $ra = $self->_new_ra;
>      $to = $ra->get_latest_revnum() if $to == -1;
>      return if $from > $to;
> +    $self->{torev} = $to; # XXX hack
>      print "Retrieving log information from $from to $to\n";
> +    $self->mirror->_show_progress(0, $to-$from+1);
>      eval {
>      $ra->get_log([''], $from, $to, 0,
>                   0, 1,
>                   sub {
>                       my ($paths, $rev, $author, $date, $msg, $pool) = @_;
>                       $code->($rev, { author => $author, date => $date,
> message => $msg });
> +
> $self->mirror->_show_progress($rev-$from+1,$to-$from+1);
>                   });
>      };
> +    print "\n";
>      $self->_ra_finished($ra);
>      die $@ if $@;
> }
> @@ -436,7 +440,9 @@
>              $t->repos->fs->change_rev_prop( $_[0], 'svn:date',
>                  $metadata->{date} );
>              $self->fromrev( $_[0] );
> -            $callback->( $changeset, $_[0] ) if $callback;
> +
> +           # XXX: torev is a hack
> +            $callback->( $changeset, $_[0], $self->{torev} ) if
> $callback;
>          }
>      );
>
> @@ -536,6 +542,7 @@
>      $self->refresh;
>      my @revs;
>      $self->traverse_new_changesets( sub { push @revs, [@_] }, $torev );
> +    $torev = $self->{torev}; # XXX do this more cleanly
>      return unless @revs;
>
>      # prepare generator for pipelined ra
> @@ -550,6 +557,7 @@
>          }
>          $ra = SVK::Mirror::Backend::SVNRaPipe->new( $ra, sub { shift @gen
> } );
>      }
> +    print "Downloading revision information...\n";
>      my $pool = SVN::Pool->new_default;
>      for (@revs) {
>          $pool->clear;
>
> ==== BEGIN SVK PATCH BLOCK ====
> Version: svk 1.99_05 (linux)
>
>
> eJyNVt2O20QUzgUXECGeAKRh68AujZMZ/9tRo4ifvQEqVEpVtUXR2B4npo4nGju7jeqVdtUKCW64
>
> 6QVPwQ2v0Ct4BFRueAkkbjgzsb3ZJdvFipSZM+d858fnO55DcXc0IdV4jCuN4Orre58HwVe0jOY9
>
> YlWaU7E4LbnQ7CpjRyzTzCrjM82qcrpgcFrwlYjkoqRixkq5SKPHrByPCcB5G7jPFEQDq1BDWvK8
>
> 0HwFPy0FYxqpvIlfTUz5m2rErwoGJwp2KthRWqQ8hzCIiQkBFdAnYM+XLJ8Kzkt5ZNi2PTGkNa6i
>
> jBdsKuFBMrGlvqFBSsogTgWLIKY1WJVilT9WaA2C8m/XCOeqoNJCmTug/A3UMEtD5exaOKOFs3bA
>
> EfMcbwh1VF6vxTRlARWm3VQnSTOmGZfQhl+mQnAxWC6U+wZXWTo7ojHwTvsLxlcGZW0SVad0uczW
>
> 05I9KWOWlVQFahqVgYllGqETkiiMQubZkUXMyDfsGDMXE0+zXRO66Xan88f7r6q3n/c6v73XOYW/
>
> n4LfZ4s10hLBF9Am6BbSCpYl+rgWjLoInlo2Leb8eLoUfCZYUezjPtIgRHakN+Y3yQFKk1p6wXSh
>
> 8p1Gc5rPGLRmsb9R6isl+RSrED3tviWXENA+0lpdcCM1a1h0AEFOpqON6g0NCDVjQh+necL39z7h
>
> i0ValixGTdMrYyQD3BK12IO9gxppd47nmm2S/8n6YNQmcaJWtWAp0rxEe4/yvc0+ThnSJqpAk1H3
>
> pNuVKV/0BxWoC6DCgUyLeZqUo0YK8ayEYHmpgmgqIQ8zHtEMaRWISF35kqOqarcqmgTtoZ4Z93po
>
> vxcPe/HBI7HXR3CwTzD+qAEfSmzwsO1rYm0PBkWKpvcYwRZ2rBjT0LRs3yVxZIWGH0Vhkjih2dLU
>
> 3UUMbycxhh9TmIN5rOh0LUGc1oV3gbWm+1pwEN2+Qzccdq/34r6Whp4sBTUTajBqho7tex4NXTtM
>
> HBobie16cWi5muX7Gxqeff/F2Q8f9J/5nZeHnRedn998eXh69k7n7N6rN04fdJ7TP6PTR50X5l/5
>
> r51nD/9+95fOjw/+EejGhycjdAMtaZQma8QWNCrqtn2qWvJE8rfkUuf+/ftoDnlud+IdVooUOJDP
>
> oF1mSDJGLGgpGaH4oRoaQdsARtu1Fwh8xQzYUEGy/1oDrWXOTdLfNgUaAg9Pau6cXKaQ4mjNeZlc
>
> gDbTIC0QPU+0eTQgQxaCVB9fmiTTh/jbPrpYtXqw3LokbsoYc1TOwc8CpCjKGM2zdR3Yp/w4zziN
>
> ZUnb6bJV18FgIAu5kzyqY1hsY5+Enstc308odgmJbMPGxCeObarvuqGZ2KmQkO3pTBLOA4S+AzcV
>
> MjB2dGLoxEHEDwwcWBbSsYNxtx4HoaB5NEc8QcURFEjIrz/eDeEhYgWWHxC/gaBxDGO0nUwhFQjy
>
> AqQcFes8GqGQl/O2kRDNt2ZupMZwUbskV7p0AmwFpt24lLhowUomIOZED9c6z1lfQdOs4I0w4/wx
>
> VFynpV7OmX4sOGwivsrBcIBQkj5h8aCrEbeCtKcStO2/8dioNMPY3KzuqutREHyTp0dMFDTrORVc
>
> zpa0nMNNDZKBzWqVxvLVDVVFhwDY3KI006li18FeFGPdM0mkY4tg3TcjrDM7hJcWwdLwxgdGdSVw
> NYSlYVhYolkwMj1qh7pHGdaxKdFCbOgetAIxHcf3SazZ5v/yGpwHHKhw/wXQ/1d9
> ==== END SVK PATCH BLOCK ====
>
>
> _______________________________________________
> svk-devel mailing list
> svk-devel at bestpractical.com
> http://lists.bestpractical.com/cgi-bin/mailman/listinfo/svk-devel
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.bestpractical.com/pipermail/svk-devel/attachments/20061218/9773943f/attachment-0001.htm