|
Clearing SpaceThis qref gives a number of tips, tricks, and hints for clearing up space in your home directory on Turing. Finding things to deleteThe first step to clearing up space is to figure out what is eating all that space to begin with. Three useful tools for doing this are ls, du, and ded. For more information on ded, go to the ded qref; ls and du are covered in this one. If you've gotten far enough to get worried about filling up space, you probably know how to use the ls command to list the contents of your home directory. However, you may not realize that ls can take several useful arguments which can help you keep your disk usage under control. First of all, when you run ls, there is a whole set of files which gets completely ignored. Any file whose name begins with a period will not show up in a normal ls listing, even though it is there, taking up space and filling your quota. To escape this, use the -a option to ls. While you're at it, use the -l option as well, which gives you an expanded listing, including how much space each file takes up. Here's a simplified example: turing:~> ls -al total 845 drwx-----x 19 jdoe students 1536 Jun 30 16:13 ./ dr-xr-xr-x 29 root root 29 Jun 30 16:20 ../ -rw------- 1 jdoe students 880 Jun 30 12:24 .TTauthority -rw------- 1 jdoe students 8582 Jun 30 14:08 .Xauthority -rw------- 1 jdoe students 516 Sep 30 1999 .Xdefaults -rw-r--r-- 1 jdoe students 1908 Jun 23 10:23 .acrorc -rw-r--r-- 1 jdoe students 237 Jun 23 10:23 .acrosrch -rw-r--r-- 1 jdoe students 0 Sep 8 1999 .addressbook -rw------- 1 jdoe students 2285 Sep 8 1999 .addressbook.lu -rw-r--r-- 1 jdoe students 3079 May 31 17:16 .cshrc drwxr-xr-x 9 jdoe students 512 Jun 30 12:24 .dt/ -rwxr-xr-x 1 jdoe students 5111 Jun 2 17:29 .dtprofile* -rw-r--r-- 1 jdoe students 757 Apr 2 19:12 .emacs -r--r--r-- 1 jdoe students 4374 Sep 1 1999 .emacs_old -r--r--r-- 1 jdoe students 12334 Sep 1 1999 .fvwmrc drwxr-xr-x 11 jdoe students 512 May 25 15:54 .gimp/ drwxr-xr-x 2 jdoe students 512 Oct 11 1999 .hotjava/ -r--r--r-- 1 jdoe students 2158 Sep 1 1999 .login -rw------- 1 jdoe students 96406 Jun 23 10:49 .lsof_turing -r--r--r-- 1 jdoe students 7901 Jun 5 14:15 .mailcap -rw-r--r-- 1 jdoe students 727 Jun 5 14:15 .mime.types drwxr-xr-x 2 jdoe students 512 Jun 23 10:34 .ncftp/ drwx------ 5 jdoe students 512 Jun 30 14:04 .netscape/ -rw------- 1 jdoe students 9395 Jul 1 16:59 .pine-debug1 -rw------- 1 jdoe students 9761 Jul 1 16:37 .pine-debug2 -rw------- 1 jdoe students 9432 Jul 1 16:07 .pine-debug3 -rw------- 1 jdoe students 9894 Jul 1 16:06 .pine-debug4 -rw-r--r-- 1 jdoe students 13993 Jul 1 16:04 .pinerc -rw-r--r-- 1 jdoe students 0 Sep 8 1999 .rexlog -rw-r--r-- 1 jdoe students 66 Mar 3 14:27 .saves-17079-turing.cs.hmc.edu~ -rw-r--r-- 1 jdoe students 78 Jul 1 16:54 .saves-18965-turing.cs.hmc.edu~ -rw-r--r-- 1 jdoe students 128 Nov 12 1999 .saves-4293-turing.cs.hmc.edu~ -rw-r--r-- 1 jdoe students 148 Apr 10 22:03 .saves-4534-turing.cs.hmc.edu~ drwxr-xr-x 2 jdoe students 512 Oct 12 1999 .ssh/ drwxr-xr-x 3 jdoe students 512 Feb 18 14:16 .ssh2/ -r--r--r-- 1 jdoe students 578 Sep 1 1999 .xinitrc -rw------- 1 jdoe students 18446 Jul 1 16:38 .xsession-errors -rw-rw-rw- 1 jdoe students 22016 Feb 25 15:01 AdobeFnt.lst drwx------ 2 jdoe students 512 Jun 28 17:57 Mail/ drwxrwxr-x 4 jdoe cs60 1536 Feb 11 14:15 cs60/ drwxr-xr-x 11 jdoe students 512 May 31 18:17 cs70/ -rw------- 1 jdoe students 1859 Jun 28 14:41 dead.letter -rw------- 1 jdoe students 119636 Jul 1 16:59 mailbox drwxr-xr-x 2 jdoe students 512 Oct 13 1999 public_html/ drwxr-xr-x 2 jdoe students 512 Apr 14 15:53 temp/Note: if the output of ls overfills your screen, just send its output to the less program, which allows you to scan through it with the arrow keys, like so: ls -al | less. The more program is (ironically) less full-featured, but works better with some terminals and telnet programs. This listing can get quite overwhelming, but the thing to keep in mind is that the column after the group name indicates the size of the marked file, in bytes. Since you are concerned here with clearing up space, concentrate your attention on the files with the largest sizes, and the groups of similar files. Usually, no dot-file should be over 100 KB at a maximum, and most are much smaller. If you find a large dotfile, ask a consultant if it is safe to delete. Here are a few points to keep in mind:
One thing you may have noticed in the above listing is that all of the directories listed have a size of 512 bytes. This is because ls shows you the size of the directory itself, not the size of its contents. If you have an idea that one directory might be taking up a lot of space, you can simply cd into it and run ls -al. However, if you have no idea what directory is eating all of your space, the du command can be very helpful. du prints out a listing of every directory and subdirectory of the directory you're in, and how much space that directory (and its subdirectories) takes up, including its contents. To get the most out of this, it often helps to run it as du -k | sort -n. This runs du (the -k option gives output in kilobytes rather than disk blocks), and runs its output through the sort program, which sorts the output. As a result, you get the largest directories at the bottom of the list where they are easy to see. Here's an example: 1 ./.dt/appmanager 1 ./.dt/help 1 ./.dt/icons 1 ./.dt/tmp 1 ./.dt/types 1 ./.gimp/brushes 1 ./.gimp/gfig 1 ./.gimp/gflares 1 ./.gimp/gradients 1 ./.gimp/patterns 1 ./.gimp/plug-ins 1 ./.gimp/scripts 1 ./.gimp/tmp 1 ./.netscape/archive 1 ./cs60/a12 1 ./cs60/grader 1 ./nsmail 2 ./.dt/sessions 2 ./.hotjava 2 ./.netscape/xover-cache/host-news 2 ./Mail 3 ./.netscape/xover-cache 3 ./.ssh 7 ./.ddd/sessions/.ddd 8 ./.ddd/sessions 13 ./.dt/sessionlogs 16 ./cs70/a6 18 ./.ncftp 26 ./.ssh2/hostkeys 28 ./.dt 28 ./.ssh2 48 ./.netscape/cache/0D 50 ./public_html 68 ./.netscape/cache/15 70 ./.netscape/cache/01 85 ./.netscape/cache/0F 91 ./cs70/a8 94 ./.netscape/cache/0C 99 ./.netscape/cache/08 101 ./cs70/a1 112 ./.netscape/cache/06 114 ./.netscape/cache/1F 121 ./.netscape/cache/04 122 ./.netscape/cache/12 124 ./.netscape/cache/1C 124 ./.netscape/cache/1E 131 ./.netscape/cache/0E 132 ./.gimp/palettes 136 ./.netscape/cache/19 147 ./.netscape/cache/11 148 ./.netscape/cache/00 152 ./.netscape/cache/0B 158 ./.netscape/cache/1D 164 ./.netscape/cache/0A 171 ./.netscape/cache/05 178 ./.netscape/cache/03 186 ./cs60 187 ./.netscape/cache/09 191 ./.netscape/cache/1B 192 ./.netscape/cache/07 194 ./.ddd 204 ./.netscape/cache/10 206 ./.netscape/cache/16 223 ./.netscape/cache/14 227 ./.netscape/cache/18 235 ./.netscape/cache/13 254 ./cs70/a7 281 ./.netscape/cache/1A 288 ./.netscape/cache/02 335 ./.gimp 339 ./cs70/a10 345 ./cs70/a2 671 ./cs70/a3 846 ./cs70/a9 977 ./temp 5150 ./.netscape/cache 5521 ./.netscape 10180 ./cs70/a4-5 12847 ./cs70 20876 .Note: As with ls, you can use less or more to help you handle all this output, but the most useful information is at the bottom, so this is often unnecessary. As you can see, the largest subdirectory here is cs70/, followed closely by .netscape/. This user could clear up 5 megabytes just by clearing her Netscape cache. In cleaning up your directories, focus your attention on the directories at the bottom of the sorted du listing, because those are the ones that take up the most space. Deleting thingsOnce you have identified files and directories that are taking up too much space, there are several things you can do to deal with them. The simplest thing to do is simply to delete them with the rm command. If it's a single file, simply type rm filename. If it's a directory, type rm -rf dirname, which recursively deletes the directory and everything in it, without asking you if you want to delete it. Remember, though, that once you delete something, it cannot be gotten back except by contacting the CS staff to try to recover your files from a backup. Bear in mind that if your file is too new, or there is a problem in the backup system, it may not be possible for us to recover your file, and even if we can, we will be grumpy about it. Do not rely on the availability of backups- only delete things you know you don't want. If you aren't sure whether you will want something again, but want to get it out of the way for the moment, there are several tools for doing this. If you have a single large file that you don't need at the moment, you can compress it with gzip: gzip filename. The file will be compressed and placed in a new file, filename.gz. If you ever need it again, simply type gunzip filename.gz, and the file will be uncompressed and restored. Exactly how much space you gain by compression varies with the file, but most text files can be compressed to less than a quarter of their original size. Binary files such as programs and graphics files often cannot be compressed very much, so using gzip on them is far less effective. If you are trying to clean up an entire directory, or a large group of files, and you're not sure if you will need them again, there is a very useful tool called gtar which can help with that. gtar takes groups of files and packs them into one single file. This alone makes it very useful as an organizational tool. However, it can also compress the files as it packs them, producing an output file that can be much smaller than the input. To use gtar to archive a directory, the command is gtar -czvf dirname.tar.gz dirname/, where dirname is the name of the directory being archived. This will produce a file called dirname.tar.gz, which can later be unpacked to reproduce the original directory. gtar does not delete the original directory, so you will have to do this yourself. If you are short on space, you may need to create the tar file in the /tmp directory like so: gtar -czvf /tmp/dirname.tar.gz dirname/. Then, delete the original directory and move the tar file into your home directory (be sure to do this, as /tmp gets automatically cleared out periodically. To recover the original directory, the command is gtar -xzvf dirname.tar.gz. Before you do this, make sure you have enough space, because, as with creating a tar file, the source file is not deleted. Special topic: CS Course DirectoriesCS course directories are often among the worst offenders when it comes to hogging space. Fortunately, they are usually quite easy to tame. Much of the space in a typical course directory is taken up by redundant files, such as editor temporaries (such as those that end with the ~ character if you use emacs), object files, and compiled programs (including .class files). Often, all you really need is the program source and associated READMEs and other documentation- you can always recreate the program by recompiling. Deleting compiled programs and object files is especially useful because such files don't compress well Once you have removed the deadwood and redundancy from your course directory, it is a simple matter to use the procedure described above to create a tar archive of it. This is often very effective, since program source code and documentation, being text files, compress very well. Command Summary
Copyright (c) HMC Computer Science Department.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.1
or any later version published by the Free Software Foundation;
with the no Invariant Sections, with no
Front-Cover Texts, and with no Back-Cover Texts.
A copy of the license is included in the section entitled ``GNU Free Documentation License.''
HMC Computer Science Department Contact Information |