Misc Tasks Scripts

Please sort this list of FQN host names by the second key, then by IP

sort -t . -k 2

sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4

Just a few one line-ers actually that I can never remember how to do without looking up.

Where is all my space at?

du -h --max-depth=1 -x
The -h is for Human readable and the -x is to skip different file systems, like things in the mnt folder, though you may want other disk partitions.
du -k /usr |sort -n |tail -30

A slightly more detailed look than the above...and a longer to execute one

Who's pounding my web server

tail -20000 temp.log | cut -d " " -f7 | sort | uniq -c | sort -bn

Take the last 20K hits in the file, cut out everything but the IPs, sort it, count the unique entries, then sort those.

What was that IP asking for?

tail -200000 temp.log | grep some-ip | cut -d " " -f 4 | less


General rename

find . -name default.htm -exec mv {} index.html \; to find and execute a rename

Rename and Edit

changes just part of a file name (aug-pcounter.log becomes aug-pcounterauth.log)

for FILE in `ls data/hosts/*pcounter*`; do mv $FILE $(echo $FILE | sed s/pcounter/pcounterauth/); done

Edit (stream editing with sed )

Windows (dos) Text Files
We have had to append onto the end of files, normall a sed s/$/NEW/, however windows log files require something different

cat temp | sed 's/\r/ - /' | more >access_log

Unix text files
If you need to separate a delimited file and extract certain fields, use cut. For example, to extract users' login names and shells from the system password file, /etc/password, use…

$ cut -d : -f 1,7 /etc/passwd

Remove Extra Spaces
To remove extraneous spaces in a file, use this command. It finds every occurrence of more than one space and replace it with one space. Add a /t for tabs.

sed 's/[ ][ ]//g'

Remove Blank Lines

sed '/^$/d'

Some good general pages


Grep in general

To search for inverse and combine it with an or, as in search for everything not a 200 or 302

cat some_file | cut -d " " -f 9 | grep -v -E "200|302"