windows - How to copy files and create hardlinks instead of copying when files are identical

Are there any tools for Windows that can

• copy one dir to another


• read copied content


• generate MD5


• if the current file is identical to a previously copied one, create a hardlink in destination dir instead of writing the content?

Answer

If you're copying the content just to hardlink it immediately afterwards, why not just generate the hardlinks straight away? Link Shell Extension makes this particular job easy.

If there's a reason you need to go through that particular sequence of actions, LSE's author also wrote a command line tool called dupemerge to do almost exactly what you're asking.

One thing to keep in mind is that NTFS does not do "copy-on-write" semantics for hardlinks. If something modifies the contents of a file, all hardlinked versions are immediately "updated", since they're all essentially directory entries to the same data extent on disk. What's more, many programs do a "save to temp file, delete original, rename temp to old name" procedure rather than overwriting a file, which will effectively break other hardlinks to the data, since they're pointing at the old data extent.