Treearrange: a compliment to rsync

[brad]

My quick hack of the evening is treearrange, which rearranges a directory tree based on a description of a directory tree, which the tool also generates

Comments

[gaal]

Oh, excellent, I've been wanting something like this for music. I wonder though if this can't be made better by knowing more about tags. The problem is syncing files several ways, when sometimes the updates are to metadata. Unfortunately the filenames can sometimes change too, so there's no key for this!

[evan]

I have exactly this problem! I guess you could fingerprint the files minus the tags -- the one part that doesn't change is the music data itself.

[gaal]

But then how can resume work?

I start syncing by pulling a new file from remotehost to my localhost. Then the download is interrupted, and I resume it. What identifies the partial file on localhost?

[gaal]

Hm, maybe the syncer should pre-tag all files with their own fingerprint and make sure that gets transmitted early?

[goldfischegirl]

I just have to say, that is sure one homosexual icon.
Lovin' it.

[jwz]

You know the file names are unique, so you don't need to hash: I do this kind of thing on the fly with a keyboard macro that generates "mv" commands...

[iamo]

However, by using a fingerprint it's somewhat more flexible. It'll work even when the original structure of the two trees were not the same.

[ciphergoth]

I would say that makes it less flexible, as well as somewhat slower.

I'd rather something that was the same "shape" as rsync (ie runs on both ends at once) that tries to move files around to make "rsync" work, based on a number of heuristics (file name, size, last modified date, first bytes, last bytes...) applied in order.

[brad]

Yeah, that's what I wanted too, but I realized it was only a few minute problem if I did the minimal work first.

Later I can add an rsync-ish interface, maybe just ssh'ing to the remote host, running Perl, and piping the original script into it, so the other side doesn't even need treearrange.

unison

[yuval_kogman]

unison can do this, albeit not intelligently.

It doesn't have smart tracking of mv's for 3-way merging, but it does have the xferbycopying feature which will check if a remote file with the same checksum exists.

Re: unison

[edm]

Excellent. I was going to suggest that this feature should really be part of something like rsync (which already walks the directories, takes checksums, compares them, etc). Having a separate script is useful, but it'd be even nicer if it "just worked" in the face of running something like rsync (ie, copied/moved files that have moved, and uploaded any new ones).

Ewen

Re: unison

[awwaiid]

Yes, I adore unison... use it to sync my home directory between about six machines, many of them using ssh rsa keys to do it unattended. Work on a project on my machine here tonight, tomorrow I go in to work and it's there waiting.

[lakeguy]

you could always just sort at work :P