Sunday, 26 January 2020

linux - dd writing on USB flash drive causes massive reads


I am currently restoring a usb stick with a dd image, I have previously created. However is noticed very poor write speed around ~7 MB/s although the stick is usually much faster.


After looking around a little, I noticed that a lot of data is being read from the USB drive which explains the poor write performance. Nothing but dd is accessing the device at that time.


Edited atop line:


sdd: busy 101% | read 18020 | write 613 | MBr/s 7.04 | MBw/s 7.1

I noticed a similar behavior when writing directly to my md array using dd. However when writing to a mounted filesystem with dd, this does not happen.


So my question is why is read from the usb stick when writing directly to the block device and whether this can be prevented to improve write performance?



Answer



So, not 100% clear what might be happening, but my suspicion is that you're writing small blocks (512-byte) to the USB, and then dd checks to make sure the byte is written properly - leading to LOTS of reading and writing, and reading and writing.


Tuning your block size to the "right" size isn't trivial, but there are a handfuls of rules of thumb on how to do it. In general, I've found a 64kB block size to be a good compromise on most fronts - but your file systems can drive changes to that, if you're running anything exotic (like zfs, for example).


Try something like this: lbzcat image-dd.bz2 | dd of=/deb/sdX bs=64k


For a more lengthy discussion, check out this post.


No comments:

Post a Comment

How can I VLOOKUP in multiple Excel documents?

I am trying to VLOOKUP reference data with around 400 seperate Excel files. Is it possible to do this in a quick way rather than doing it m...