linux - Deleting files doesn't free space

I've got a disk formatted as ext3, which was filled completely. I'm attempting to free up some space on it by deleting files, but it's not working. I can rm the files, and they don't show up in the directory listing, but I don't have free space available.

Below is a copy of attempting to delete some files. As you can see from the ls -la, the files aren't hardlinked to another location, and the delete appears to succeed. In the df output, the number of used blocks decreased by 182556, which is the space taken by the files, but the available count remained at zero. fsck detected no problems with the filesystem, and didn't change the free space at all.

I'm using CentOS 6 right now to attempt to delete the files, but most of the files were written using a Debian distribution (I'm not entirely sure which version, whatever Clonezilla uses).

# ls -la
total 182564
drwxr-xr-x. 2 root root      4096 Mar  4  2011 .
drwxr-xr-x. 5 root root      4096 Aug 13 13:18 ..
-rw-------. 1 root root   4030114 Mar  4  2011 sda1.vfat-ptcl-img.gz.aa
-rw-------. 1 root root 182667379 Mar  4  2011 sda2.ntfs-ptcl-img.gz.aa
-rw-r--r--. 1 root root        37 Mar  4  2011 sda-chs.sf
-rw-r--r--. 1 root root     31744 Mar  4  2011 sda-hidden-data-after-mbr
-rw-r--r--. 1 root root       512 Mar  4  2011 sda-mbr
-rw-r--r--. 1 root root       375 Mar  4  2011 sda-pt.parted
-rw-r--r--. 1 root root       259 Mar  4  2011 sda-pt.sf

# df .
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sdb2            484009516 473901232         0 100% /media/Images

# rm -f *

# ls -la
total 8
drwxr-xr-x. 2 root root 4096 Aug 13 15:05 .
drwxr-xr-x. 5 root root 4096 Aug 13 13:18 ..

# df .
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sdb2            484009516 473718676         0 100% /media/Images

# cd /media ; umount Images

# e2fsck -fv /dev/sdb2
e2fsck 1.41.12 (17-May-2010)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

     589 inodes used (0.00%)
      37 non-contiguous files (6.3%)
       0 non-contiguous directories (0.0%)
         # of inodes with ind/dind/tind blocks: 278/256/0
119390761 blocks used (97.89%)
       0 bad blocks
       5 large files

     551 regular files
      29 directories
       0 character device files
       0 block device files
       0 fifos
       0 links
       0 symbolic links (0 fast symbolic links)
       0 sockets
--------
     580 files

# mount /dev/sdb2 Images

# df Images
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sdb2            484009516 473718676         0 100% /media/Images

Answer

The ext3 and ext4 family of filesystems reserve a portion of the disk to keep it from becoming truly "100% full", for stability reasons, and so that the FS can store metadata in some of that space.

You can directly tweak how much of the space is reserved:

tune2fs -m 1 /dev/sdXX

replacing "1" with the percentage (0 to 100) of the disk to reserve, and "XX" with the device node and partition number of the device node, so in your case "XX" would be "b2".

Analogy: you have a full bus, and the only remaining seat is next to a very large person who occupies a seat and a half by himself. A person who is very insistent on getting a seat walks up and demands to sit there. Although most people would consider the second seat taken, this person is insistent. So the large person goes, "Whoa, OK!" and squishes in to allow them to sit. But as soon as 1 person gets off the bus and opens up another seat, even if the passenger next to the large person moves into a seat, most people still consider the bus to be 100% full, because nobody wants to sit next to the large person.

Source

You can also check this to see Reserved Blocks count…

dumpe2fs -h /dev/sdb2

To quote the very intelligent user who nailed the issue in the source above:

You will see "Available" go positive when "Used" is reduced to below 0.95*136236548 blocks = 129424720 blocks roughly.

(we have to adjust the "0.95" to the reserved % in your specific case, and the 136236548 blocks to the total block size of your device).