[Bps-public-commit] brackup annotated tag, 1.09-01-6, created. 1.09-01-6
Alex Vandiver
alexmv at bestpractical.com
Wed May 26 23:39:15 EDT 2010
The annotated tag, 1.09-01-6 has been created
at 8c1eb81d12272b9d10508e4e8e6d5a221a5588a3 (tag)
tagging 1151c6844534b2c4f056dd3bbd2d6b060fcd9e02 (commit)
tagged by Alex Vandiver
on Thu Apr 8 22:56:33 2010 -0400
- Log -----------------------------------------------------------------
libbrackup-perl (1.09-01-6)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEABECAAYFAku+l2EACgkQMflWJZZAbqAiuwCeI2Nytj9qB/3DWiAEaNb++6cK
HcsAoLLVDAPodsqxNN1ylfslgjHuuyTn
=cmA8
-----END PGP SIGNATURE-----
Alex Vandiver (10):
Ignore .git directories
Add build dependencies
Allow for multiple space-separated key IDs for encryption
Preliminary debian-ization
Ignore debian files
Additional release
Create a GNUPGHOME for ourselves before testing, rather than use the user's
Chek in the version bump I've been using locally
Merge branch 'master' into debian
Release -6, now passing buil on launchpad
brad at danga.com (20):
actually respect the --just flag on restore
don't complain to stderfrif parent process has gone away
changes entry from previous commit
- new on-disk layout for the Filesystem target. fully backwards
don't stat old-style dirs if only operating in new-style world
Checking in changes prior to tagging of version 1.06. Changelog diff is:
start of content-aware file chunkers.
bradfitz: optional smart mp3 chunking. (start of file-aware chunking
smarts to filesystem target, noticed when using sshfs (slow filesystem).
actually strip non-ASCII characters (gary.richardson at gmail.com)
fix utime warnings on restore
remove some done stuff
Checking in changes prior to tagging of version 1.07. Changelog diff is:
Checking in changes prior to tagging of version 1.08. Changelog diff is:
move all decryption to one place; support decrypting metafiles.
Checking in changes prior to tagging of version 1.09. Changelog diff is:
Start of AppEngine blobstore target support.
do retries to AppEngine on transient failures.
also ignore brackup-digest.db files
More Google AppEngine docs.
bradfitz (137):
initial commit
get gnupg kinda working
Changes regarding this:
some docs
include original link
tech notes
more docs, and s/cache/digdb/ in File.pm
work for the night. works, but no metafile written yet.
change data structures a bit. make the chunk keys based on sha1 of
can write the .brackup rfc822 metafile now.
let you specify output file
functional now, but no restore.
cleanup
make Amazon target kinda work. Net::Amazon::S3 is still buggy, as is
don't backup the digest database
a --help mode, and docs
make a starter template for them
packaging
more packaging
add no warranty statement
make amazon target more robust, now requiring changes to Net::Amazon::S3
0.71
remove debug
add gpg-rcpt to cache key
whitespace fixups, and don't put Digest into the outfile for 0 byte files
remove noise
let digestdb file be configurable
let targets add fields to metafile. and whitespace cleanup, whoops
start of restore support, and start of a good test suite
a lot of work on restoring files, and tests for everything. not quite perfect yet
improving tests
verify restored files' digests, fix test suite. restore now works perfectly, with testing.
move shared testing code into Brackup::Test, in prep for doing GPG
clean up output
-- bitch when source config has unrecognized keys
refactoring for readability
TODO / Changes
gpg restoring is mostly working now
bug fixes with gpg backups/restores. chunkref has to be memoized so it's not encrypted differently multiple times within processing of a file. this also means backups are 2x as fast now! :)
passes both regular/gpg tests backing up and restoring.
notes, docs
prep for 0.80
more todos
updated changes
restore front-end tool.
amazon s3 restore works now
make a Brackup.pm, just to hold version number and stuff
trust-model=always ... thanks Frank!
forgot the restore and changes
0.91 was never committed
not yaml
start of stats and --dry-run options
minor stuff
rearrange
+ - deal w/ encryption better: tell chunks when the backup target
+ - new --du-stats to command to act like the du(1) command, but
actually, we do need to use wanted callback for right ordering
- support putting .meta files besides .chunk files on the Target
require gpg-agent, and help out with it... also add --batch and info messages
clean up tree
- merge in the refactoring branch, in which a lot of long-standing
sort this
todo
fix Amazon target
- cleanups, strictness
clean up quicker
not done yet
halve number of stats when walking backup root
show progress with -v, and include new doc
- clean up old, dead code in Amazon target (the old inventory db which
say something's happening on start
- retry PUTs to Amazon on failure, a few times, pausing in-between,
- make PositionedChunk use the digest cache (which I guess was
+ - fix memory leak in case where chunk exists on target, but local
add a chunk iterator object, to be used by future refactor of backup
noise
switch backup.pm to use a stream of events from an iterator object.... in prep for future release with child processes doing gpg ahead of time for main process, working off multi-plexed
multiplexed slave iterator stuff.... you can start to see now how
change where we encrypt, to make next transition more clear
fix tests, previously broken.
misc
if restore failed, don't proceed to tree diff
intro GPG process management stuff. for now, only manages one process at a time, but infrastructure is all in place now to have lot of GPG processes runnign at once (use multiple CPUs), and keep data pipe loaded, for network uploads to continue saturating the network.
more
manifest cleanup
add to comment
parallel gpg stuff
start of some code to show size actually needed to upload, ahead of time. usually undesirable, so guard with env variable for now... maybe an option later?
show better percentage bars, if we precalc'd the data
add Brackup::DigestCache, subclass of Dict::SQLite, but will have more
fix:
bogus dep
memoize full digest
less/smarter verbose output:
suck in the whole sqlite file, so it's not so damn slow.
skip files ending in whitespace, and don't cache stat info is file is being skipped
legacy migration thing.
note for future
defer loading everything until first access
more debug
noatime support
cleanup
lot was already done
- track in meta header the default (most often occuring) modes for
document verbose
Checking in changes prior to tagging of version 1.00. Changelog diff is:
more docs
lot of docs
Checking in changes prior to tagging of version 1.01. Changelog diff is:
todo
so search.cpan.org picks it up
support for merging little files together into big chunks
don't tail-pack... only merge chunks into composite chunks that represent entire files
Checking in changes prior to tagging of version 1.02. Changelog diff is:
this was unused.
doc fixes
use default
start of target management tool
more todo
fixes
doc updates
implement list_targets command
commands to get brackup files from server
don't leak tempfiles
more brackup-target work, and todo/changes updates
start of tempfile leak auditing, moving all tempfile creation into Brackup::Util wrapper, which currently does nothing.
stop leaking tempfiles
brackup-restore's verbose flag is more verbose now, showing files
update
Checking in changes prior to tagging of version 1.03. Changelog diff is:
make tests pass on OS X (Jesse Vincent)
+ - Amazon list_backups and delete backups (and delete for filesystem
Checking in changes prior to tagging of version 1.04. Changelog diff is:
'prune' and 'gc' commands commands for both Amazon
Checking in changes prior to tagging of version 1.05. Changelog diff is:
GoogleAppEngine store_backup_meta implementation.
App Engine restore support.
gavin at openfusion.com.au (86):
Add initial Brackup::Target::Sftp target (using Net::SFTP::Foreign) and unit test.
Precompile Brackup::Root::ignore regexes for performance.
Rename t/01-backup.t to t/01-backup-filesystem.t (more explicit, and ordering).
Tweak explanatory comments in t/01-backup-ftp.t, and add to MANIFEST.
Add --just=DIR and (failing) --just=FILE restore tests to fs/ftp/sftp unit tests.
Fix --just=file case in Brackup::Restore by stripping Path eq prefix case more carefully.
Fix a couple of buglets in Brackup::Target::Sftp.
Add headings, date formatting, and sorting to brackup-target list_backups.
Add Brackup::Util::noclobber_filename and use for brackup backup files.
Trival patch to chomp brackup_restore errors before reporting.
Make brackup-restore report errors if nothing is restored.
Fix typo in Brackup::Metafile error message.
Fix inversion bug in brackup-target prune --keep-backups option handling.
Tweak 04_gc.t test to prevent orphan chunk count = zero.
Fix typo in Brackup::Target::Sftp::chunks.
Rename 04-gc.t to 04-gc-filesystem.t and tweak.
Add colon-unescaping in Brackup::Target::X chunks() methods.
Add new 04-gc tests to MANIFEST.
Add missing t/04-gc-{,s}ftp.t tests.
Add t/04-gc-{,s}ftp.t back to MANIFEST.
Make brackup-target --verbose prune more verbose about what is being removed.
Make brackup-target --verbose gc more verbose to give more indication of progress.
Make brackup-target gc more verbose still, reporting chunks with -v -v.
Accept hyphenated versions of commands in brackup-target.
Add --interactive mode to brackup-target gc.
Add a --source option to brackup-target to restrict prune to a particular source.
Update Amazon target to handle new ISO8601 modify timestamps (Ben Schumacher).
Add recent brackup-target changes to changelog.
Add count, each, and delete methods to Brackup::{Dict::SQLite,InventoryDatabase}.
Add size() method to Brackup::Target::Filesystem.
Convert Brackup::Target::{Ftp,Sftp} _size() methods to public size().
Add size and chunkpath methods to Target::Amazon.
Add brackup-verify-inventory utility.
Add FindBin stanza to brackup-verify-inventory.
Trivial perldoc tweaks.
Add metafile path name escaping using String::Escape.
Add String::Escape to Makefile.PL.
Add missing trailing semicolon in Brackup::Backup.
Add brackup-mount and brackup-verify-inventory to Makefile.PL.
Add name attribute and accessor to Target, like for Root.
Change default metafile name from source-date to source-target-date.
Replace some instances of open() with sysopen() to fix trailing whitespace handling.
Limit Brackup::Root trailing whitespace skipping to encrypted backups.
Comment out Brackup::Root trailing whitespace skipping (works with gpg 1.4.5, at least?).
Add t/05-filename-escaping.t unit test and data.
Stats output, BackupStats implementation, --save-stats arg.
Write metafile incrementally, instead of big bang at end.
Create brackup-restore restore directory if it doesn't exist.
Perldoc tweaks to brackup-restore.
Add CompositeChunk ftp/sftp tests; tweak ftp/sftp tests to use skip_all.
Sort restore files to avoid reloading composite chunks from target (Kostas Chatzikokolakis).
Allow brackup-restore to use config file sections/passwords, if available.
Update Target::Amazon::new_from_backup_header to use config file settings, if available.
Update Target::CloudFiles::new_from_backup_header to use config file settings, if available.
Add Amazon::backup_header; fix Restore::restore $confsec to be ConfigSection, not Target.
Turn on recommended retry flag in Target::Amazon (Kostas Chatzikokolakis).
Remove location and backup_prefix from Amazon::backup_header etc. (not required for restores).
Tweak Brackup::Restore::restore to automatically relativise absolute --just paths.
Fix incorrect path reporting with --just in Brackup::Restore::restore.
Tweak brackup-restore to not require brackup.conf, or initialise template one.
Add brackup-restore config file entry to Changes.
Update metafile storage to pass filehandle instead of full contents (Gavin Carr, Kostas Chatzikokolakis).
Remove one tempfile from gpg processing, use filehandles instead of content (Kostas Chatzikokolakis).
Convert encryption and chunk storage to filehandle-based versions (Kostas Chatzikokolakis).
Add Util::tempfile_obj, and use for GPGProcess tempfile auto-unlinking (Kostas Chatzikokolakis).
Convert DigestCache to has-a dict; allow specifiable dict type; add Dict::SQLite2.
Update InventoryDatabase to allow dict type to be specified; perldoc updates.
Add seekable File::Temp require to Makefile.PL.
Remove bogus encryption warning from DigestCache docs.
Add simple config section inheritance ('inherit' directive).
Add config inheritance unit test.
Add config inheritance unit test (take 2).
Add missing Dict::{SQLite2,DBM,Null} modules.
Add explicit close to StoredChunk::_forget_chunkref.
Throttle GPGProcManager if too many uncollected chunks.
Fix InventoryDatabase->new to return $self, not dict.
Add check_inventory_db test to Test::do_backup (Kostas Chatzikokolakis).
Add ability to specify inventorydb_type to 04-gc-*.t unit tests.
Fix minor bugs in Dict::{SQLite,SQLite2,DBM} preventing some repeated methods calls.
Add a prune + garbage collection step to gc unit tests.
Ignore .svn directories in gc unit tests.
Remove orphaned chunks from inventory as part of garbage collection.
Prefetch chunks in brackup-verify-inventory to minimise target round-trips
Report inventory_db backing file in Test::check_inventory_db test.
If IO::Compress is available, write/read compressed metafiles in unencrypted mode.
Add missing t/data-2 test files.
imsnah (12):
Fix several typos
Don't print code line for “Backup complete†message
Split generic code from Filesystem.pm into Filebased.pm
Add FTP target
Remember whether colons have been used for a backup
Backup fifo files
Fix backups listing when using FTP target
Remove trailing whitespace
Create default configuration file with more secure permissions
Automatically reconnect after FTP connection loss
FTP garbage collection bugfixes
Unit tests for FTP target
imsnah at gmail.com (5):
Use a single directory for all temporary files
Fix valid_params function by importing Carp
Don't confuse vim's syntax highlighting
Remove unused foreach_chunk function from Brackup::File
Hide code location for completed restores by adding \n to string
mart (8):
Allow user to specify config file location on the command line.
Only try to expand ~/ if $ENV{HOME} is set.
S_ISLNK doesn't work under ActiveState Perl on Windows, so fall back on the -l operator if the call fails.
The trick of calling tempfile() and then holding the filehandle open while GPG uses the file doesn't work on Windows because Brackup ends up holding a write lock on the file, preventing gpg from writing to it.
Must read chunk files in binary mode.
The naming scheme for chunk files is rather unfortunate since colons aren't allowed in filenames on Windows.
Preparation for enumerating backups on a particular target.
Implementation of the "backups" method for the Filesystem target.
mart at degeneration.co.uk (8):
Add better error message for the "restored filesize is wrong" error.
added the aws_prefix option to configure multiple backup targets on a single Amazon account.
fixed tempfile creation in gc (orphaned files were left in the working directory)
added the aws_location option to set the datacenter location for S3 targets.
An initial attempt at a FUSE-based filesystem for mounting a backup read-only.
Fix Restore.pm to use binmode for operating systems that care about such things.
Add a new option to the Amazon S3 target to force a prefix to be added to the names of any saved backups.
New option --zenity-progress that produces output suitable for piping into zenity --progress to get a pretty GUI progress bar while running a backup.
throughnothing (2):
added Rackspcae/Mosso CloudFiles Target
Got rid of typo in CloudFiles.pm
-----------------------------------------------------------------------
More information about the Bps-public-commit
mailing list