System Notes Org

Here are a few notes to add to the previous article on ping. This time, we look at some bash one-liner tips and tricks. Combine multiple commands Return Values Ping Multiple Hosts Using Bash Nmap and Fping Ping, and Command Line Variables Start from the beginning. We want to see what happens when attempting to ping a host that resolves an IP address from DNS, but is not reachable from our network at the moment. This is to show how we might build a simple monitoring tool from scratch, and also to see what kind of fun we can have with the command line. To ping a host only one time use the count option, which in Windows is -n, and in Linux is -c. Windows C:\>set host=google.com C:\>ping -n 1 %host% Pinging google.com [216.58.219.46] with 32 bytes of data: Request timed out. Ping statistics for 216.58.219.46: Packets: Sent = 1, Received = 0, Lost = 1 (100% loss), Bash: Linux, Mac, Unix Note that in bash, we don't use the set command to assign a varia...

Here is an example of how to use awk to extract a column from a text file. The default field separator is a space, so this will work with some text files:    cat somefile.txt | awk '{print $2}' But, if the fields are separated by a delimiter, such as tab, or comma, just specify the field separator by using "-F" before the print statement    cat somefile.txt | awk -F"\t" '{print $2}'   Unix and linux distributions generally come with sed and awk, among other things. The problem. You want to get a list of companies to research before investing in them, and paste only the stock symbols into a Yahoo finance portfolio.   You run a stock screen on a site such as  magicformulainvesting.com , or investors.com but if you try to paste the text into a spreadsheet it may show up as a single row, with no way to extract the column. Here is the process 1.)  Run a Stock Screen Get output similar to this: Company Name (in alphabetical...

At SystemNotes, we love free software. Especially when it is Open Source (source code is readable and may be freely modified by the end users), and available on different platforms (Linux, Mac, Windows, iPad, Android, etc.). When preparing tax returns, many people turn to paid software, or tax professionals. There's nothing wrong with paying for a useful service or software, but linux geeks often turn to shell scripts, or whatever open source software they can find before shelling out any cash -- no pun intended. Here are some useful tools for preparing the US 1040 tax return and most common schedules: Open Tax Solver If your needs are simple, Open tax solver is quick and easy. There are even options for some state taxes. Open Tax Solver http://opentaxsolver.sourceforge.net There is a simple interface that takes inputs, and when finished, just click "Compute Tax." for the results. Microsoft Excel Spreadsheet Income Tax Calculator If you prefer to ha...

Simple ways to loop in bash shell scripts, or one-liners. Note that the variable in bash is declared without a "$", and called with it, but in perl the variable is declared with and called with a "$" at the beginning of the name. The Bash "for" Loop Loop a number of times for i in {1..10}; do echo -n "$i ";done;echo Output: 1 2 3 4 5 6 7 8 9 10 For short lists, the items can be entered manually. for num in 1 2 3;do echo -n $num;done;echo Output: 1 2 3 Loop Through Items on a List for host in `cat all.txt `;do ping -c 1 $host; done See also: How to Run a Bash Command on All Items in a List and: Ping Multiple Hosts Using Bash Nmap and Fping Loop using Perl This syntax should be familiar to C programmers. In Perl. the scalar variable does not have to be declared as it would in C (int $counter). Note that the variable name starts with "$" to indicate a scalar. #!/usr/bin/perl for($counter = 1; $counter <= 10; $counter++){ print ...

I wrote a script to unlock Firefox on a linux machine. The symptom is when firefox is not running, but it won't let you start because it thinks it's still running. If you have already run "pkill -9 –f firefox", then all you have to do is delete the lock files -- .parentlock, and lock. This is user specific, so you'll have to find your own home directory under ~/.mozilla/firefox/. Mine is dz4bq7je.scottm on this host. $ cat ~/bin/unlock_firefox.sh #!/bin/sh rm ~/.mozilla/firefox/dz4bq7je.scottm/.parentlock rm ~/.mozilla/firefox/dz4bq7je.scottm/lock $ ll ~/.mozilla/firefox/ total 12 drwx------ 9 scottm users 4096 Sep 9 11:25 dz4bq7je.scottm -rw------- 1 scottm users 1264 Jun 23 2008 pluginreg.dat -rw-r--r-- 1 scottm users 162 Nov 15 2005 profiles.ini $ The advantage of this simple script is that you don't have to go looking for directories and files, or remember where firefox puts them. Just remember when it's l...

The idea to write a shell script is often the result of having to do repeated tasks. Imagine having to check hundreds of domain names by typing "whois domain.com" each time. It would take much longer than writing a simple script. Using whois to Check for Available Domain Names Suppose you want to find a good domain name, but you want to see if all of the domain suffixes are available? For example, you want a good .com domain, but you also want the .net, and .org. Not only that, but you might want to use blogspot.com, or wordpress.com for your blog. You can always check a domain availability by using the whois command, e.g. $ whois google.com Obviously, google.com is not available, so whois will display information about the company that registered the domain name. If the domain is not taken, then whois will usually return "No match", or "NOT FOUND". You have to keep trying a domain name until you find one that is available. This can be time-consumi...

Here is a very simple way to backup and compress subdirectories. Normally you would want to copy them to a remote location in case something happens to the original drive, but in this case the concern is that disk space is filling up. Suppose you have a program that is generating log files every day, let's say they are key performance indicators (kpi) for some system that you are testing. The log files are pure text, but after a while, they take up a lot of space. One way to compress text is to use gzip, but since tar has an option (-z) to tar and gzip all at once, it may be one of the easiest solutions. #!/bin/sh # compress logs in /opt/multicast_kpi/record # by ScottM, 20080613 # Set variables TODAY = " `date +%m%d%Y` " ARCHIVE_DIR =/opt/multicast_kpi/record # move to working directory cd $ARCHIVE_DIR # For each subdirectory, make a .tgz file of it's contents, and then delete the originals. for dir in `ls -1 $ARCHIVE_DIR | grep -v " .tgz " | gre...

perl -MCPAN -e shell I always seem to forget this simple syntax, so I thought it would be good to post it where I can find it. Where else, my blog -- by the way, this is another excellent use for blogs. Not only do we publish information for others to find, but occasionally a blog is the best place to put frequently accessed info that the author may need. At least I know the source is reliable. :-) How do I install a module from CPAN? The easiest way is to have a module also named CPAN do it for you. This module comes with perl version 5.004 and later. To get a cpan shell, try either cpan: Linux:   $ cpan Windows:  C:\>cpan or the more complete command: $perl -MCPAN -e shell Terminal does not support AddHistory. cpan shell -- CPAN exploration and modules installation (v1.7601) ReadLine support available (try 'install Bundle::CPAN') cpan> The first time the command runs, it will prompt for a series of configuration questions. Now, within the...

This little shell script is a good example of how to simplify a task with a wrapper script. Rather than trying to remember which settings to use for cdrecord, I just run my cdroast.sh script. ----- begin script ----- #!/bin/sh # cdroast.sh # scottm, 2005-07-20 # wrapper for cdrecord INPUT = $1 if [ $INPUT ] then cdrecord -v -sao dev = 0 , 0 , 0 $INPUT else printf " \n Enter name of file \n " printf " e.g. $0 filename.iso \n\n " fi ----- end script ----- Now, with the script in my path, all I have to type is: $ cdroast.sh myfile.iso and the burn process will start. This type of script is often called a wrapper, since it wraps a simple script around a more complicated procedure.

I explained how to get a list of hosts using nmap -- using-nmap-to-generate-host-lists.html , but here is another look at the subject. The question is, how do I ping multiple hosts to find out which ones are down? Sure this could be considered a topic of system monitoring, but maybe you just want to reboot a bunch of machines, and make sure they all come back online. This quick check will tell you whether there is a problem or not. Here are three methods for pinging a list of hosts: 1.) for host in `cat all.txt `;do ping -c 1 $host; done 2.) nmap -sP -R -iL all.txt 3.) sudo fping -u < xyz/all.txt First, we assume that you have a text file named all.txt that contains a list of hostnames, one per line. Obviously, the examples here contain fake hostnames, domain names and IPs, as described in another article about vi: vim-tips-search-and-replace.html . --- all.txt --- xyz-1 xyz-2 xyz-3 xyz-4 xyz-5 xyz-6 xyz-7 xyz-8 --- end all.txt --- 1.) Use a for loop in ...

For Linux / Unix Follow as instructed For Windows Download cygwin from http://cygwin.com For instructions, see this article: http://systemnotesorg.blogspot.com/2007/04/use-cygwin-to-run-linux-on-windows.html Two Easy Steps for One-Liners e.g., you have a list of servers, and would like to do something, such as ping each one, or check their IP address. Here is a quick two-step process, with a one-line shell script that can be run from the command line. 1) Create a Text file, with one hostname per line 2) Run a for loop on the file $ cat servers.txt server01 server02 server03 $ for host in `cat servers.txt`;do host $host;done server01.example.com has address 10.10.10.10 server02.example.com has address 10.10.10.11 server03.example.com has address 10.10.10.12 $ If you want different output, you can use awk, but that's another topic. Notice a few things about this one line script. 1) Each command is separated by a semicolon ";" 2) The ba...

This shell script demonstrates how to write to a text file, and then modify the contents. #!/bin/sh # modfile.sh # by ScottM, 04/12/2007 # demonstrates writing text to a file, and then using sed to modify it. TESTFILE=test.txt FRUIT=banana # add some content to the file (note: file will be overwritten) echo "apple" > $TESTFILE # modify the content sed -e "s/apple/& $FRUIT/g" -i $TESTFILE # sed uses the "s" option, which uses regular expressions to search and replace text # "s/apple/" means search for any lines that contain the characters "apple" # "& " means use the results of whatever was found # "/& $FRUIT/g" -- replace "apple" with "apple banana", # the g is for global, or all lines containing the pattern # output: # $ cat test.txt # apple banana # $ Sed One Liner This is really only a one line script, commonly referred to as a on-liner, so we don't really need a...

An easy way to get a list of hosts from a single domain that you are a part of, is to query DNS host -l mydomain.com But that is not always practical. Sometimes you have machines that are in different domains, but they all are part of a network you manage. Rather than trying write a script that pings hosts and reports the output, just use nmap for a very fast scan. To scan all hosts in a list of subnets 1) Create a subnets.dat file with one subnet on each line: $ cat subnets.dat 192.168.0.* 192.168.1.* 2) Run nmap with the subnets.dat file as input $ nmap -sP -R -iL subnets.dat Reading target specifications from FILE: subnets.dat Starting nmap V. 3.00 ( www.insecure.org/nmap/ ) Host (192.168.0.0) appears to be down. Host box1.mydomain.com (192.168.0.1) appears to be up. Host box2.mydomain.com (192.168.0.2) appears to be down. Host box3.mydomain.com (192.168.0.3) appears to be up. Host (192.168.0.4) appears to be up. ... Notice how names are resolved ...

How do you get the current date to use in a Linux / Unix shell script? Take a look at this simple backup script: ---------------------------------------------------- #!/bin/sh export   FILE_DATE = `date +%Y%m%d` export   backup_file =backup_ ${ FILE_DATE } .tgz [   !   -d  /opt/backups ]   &&  mkdir -p  /opt/backups tar -zcvf  /opt/backups/ ${ backup_file }  /var/named ---------------------------------------------------- What does that FILE_DATE variable do for me? Well, let's try it on the command line. bash-2.05b$ export FILE_DATE=`date +%Y%m%d` bash-2.05b$ echo $FILE_DATE 20070323 Ah, so that's how we get the current date in a usable format. This method makes the files sortable later. Of course, if you need more than one backup per day, you can always add more date options. Notice a few things about this method of scripting. 1) export, is optional, but it makes the environmet variable available to other s...