duplicity.diffdir module

Functions for producing signatures and deltas of directories

Note that the main processes of this module have two parts. In the first, the signature or delta is constructed of a ROPath iterator. In the second, the ROPath iterator is put into tar block form.

class duplicity.diffdir.DeltaTarBlockIter(input_iter)[source]

Bases: TarBlockIter

TarBlockIter that yields parts of a deltatar file

Unlike SigTarBlockIter, the argument to __init__ is a delta_path_iter, so the delta information has already been calculated.

get_data_block(fp)[source]

Return pair (next data block, boolean last data block)

process(delta_ropath)[source]

Get a tarblock from delta_ropath

process_continued()[source]

Return next volume in multivol diff or snapshot

exception duplicity.diffdir.DiffDirException[source]

Bases: Exception

duplicity.diffdir.DirDelta(path_iter, dirsig_fileobj_list)[source]

Produce tarblock diff given dirsig_fileobj_list and pathiter

dirsig_fileobj_list should either be a tar fileobj or a list of those, sorted so the most recent is last.

duplicity.diffdir.DirDelta_WriteSig(path_iter, sig_infp_list, newsig_outfp)[source]

Like DirDelta but also write signature into sig_fileobj

Like DirDelta, sig_infp_list can be a tar fileobj or a sorted list of those. A signature will only be written to newsig_outfp if it is different from (the combined) sig_infp_list.

duplicity.diffdir.DirFull(path_iter)[source]

Return a tarblock full backup of items in path_iter

A full backup is just a diff starting from nothing (it may be less elegant than using a standard tar file, but we can be sure that it will be easy to split up the tar and make the volumes the same sizes).

duplicity.diffdir.DirFull_WriteSig(path_iter, sig_outfp)[source]

Return full backup like above, but also write signature to sig_outfp

duplicity.diffdir.DirSig(path_iter)[source]

Alias for SigTarBlockIter below

class duplicity.diffdir.DummyBlockIter(input_iter)[source]

Bases: TarBlockIter

TarBlockIter that does no file reading

process(delta_ropath)[source]

Get a fake tarblock from delta_ropath

class duplicity.diffdir.FileWithReadCounter(infile)[source]

Bases: object

File-like object which also computes amount read as it is read

__init__(infile)[source]

FileWithReadCounter initializer

close()[source]
read(length=-1)[source]
class duplicity.diffdir.FileWithSignature(infile, callback, filelen, *extra_args)[source]

Bases: object

File-like object which also computes signature as it is read

__init__(infile, callback, filelen, *extra_args)[source]

FileTee initializer

The object will act like infile, but whenever it is read it add infile’s data to a SigGenerator object. When the file has been read to the end the callback will be called with the calculated signature, and any extra_args if given.

filelen is used to calculate the block size of the signature.

blocksize = 32768
close()[source]
read(length=-1)[source]
class duplicity.diffdir.SigTarBlockIter(input_iter)[source]

Bases: TarBlockIter

TarBlockIter that yields blocks of a signature tar from path_iter

process(path)[source]

Return associated signature TarBlock from path

class duplicity.diffdir.TarBlock(index, data)[source]

Bases: object

Contain information to add next file to tar

__init__(index, data)[source]

TarBlock initializer - just store data

class duplicity.diffdir.TarBlockIter(input_iter)[source]

Bases: object

A bit like an iterator, yield tar blocks given input iterator

Unlike an iterator, however, control over the maximum size of a tarblock is available by passing an argument to next(). Also the get_footer() is available.

__init__(input_iter)[source]

TarBlockIter initializer

Return closing string for tarfile, reset offset

get_previous_index()[source]

Return index of last tarblock, or None if no previous index

get_read_size()[source]
process(val)[source]

Turn next value of input_iter into a TarBlock

process_continued()[source]

Get more tarblocks

If processing val above would produce more than one TarBlock, get the rest of them by calling process_continue.

queue_index_data(data)[source]

Next time next() is called, we will return data instead of processing

recall_index()[source]

Retrieve index remembered with remember_next_index

remember_next_index()[source]

When called, remember the index of the next block iterated

tarinfo2tarblock(index, tarinfo, file_data=b'')[source]

Make tarblock out of tarinfo and file data

duplicity.diffdir.collate2iters(riter1, riter2)[source]

Collate two iterators.

The elements yielded by each iterator must be have an index variable, and this function returns pairs (elem1, elem2), (elem1, None), or (None, elem2) two elements in a pair will have the same index, and earlier indicies are yielded later than later indicies.

duplicity.diffdir.combine_path_iters(path_iter_list)[source]

Produce new iterator by combining the iterators in path_iter_list

This new iter will iterate every path that is in path_iter_list in order of increasing index. If multiple iterators in path_iter_list yield paths with the same index, combine_path_iters will discard all paths but the one yielded by the last path_iter.

This is used to combine signature iters, as the output will be a full up-to-date signature iter.

duplicity.diffdir.delta_iter_error_handler(exc, new_path, sig_path, sig_tar=None)[source]

Called by get_delta_iter, report error in getting delta

duplicity.diffdir.get_block_size(file_len)[source]

Return a reasonable block size to use on files of length file_len

If the block size is too big, deltas will be bigger than is necessary. If the block size is too small, making deltas and patching can take a really long time.

duplicity.diffdir.get_combined_path_iter(sig_infp_list)[source]

Return path iter combining signatures in list of open sig files

duplicity.diffdir.get_delta_iter(new_iter, sig_iter, sig_fileobj=None)[source]

Generate delta iter from new Path iter and sig Path iter.

For each delta path of regular file type, path.difftype with be set to “snapshot”, “diff”. sig_iter will probably iterate ROPaths instead of Paths.

If sig_fileobj is not None, will also write signatures to sig_fileobj.

duplicity.diffdir.get_delta_path(new_path, sig_path, sigTarFile=None)[source]

Return new delta_path which, when read, writes sig to sig_fileobj, if sigTarFile is not None

duplicity.diffdir.log_delta_path(delta_path, new_path=None, stats=None)[source]

Look at delta path and log delta. Add stats if new_path is set

duplicity.diffdir.sigtar2path_iter(sigtarobj)[source]

Convert signature tar file object open for reading into path iter

duplicity.diffdir.write_block_iter(block_iter, out_obj)[source]

Write block_iter to filename, path, or file object