Tuesday, October 2, 2018

Create a Symlink as a Workaround for Partition Full Error

Sometimes, a mounted partition does not have enough space, and it may take time to expand the LVM volume, if there is not currently enough physical space available, and a separate team handles storage.  Of course making a separate partition is a more proper fix, and LVM is flexible enough to accommodate the changes, but sometimes space is filling up, and something has to be done right away.  Maybe /var/log, or /opt is filling up, and you did not think ahead to make it a separate partition.

As a quick workaround, it is fairly common to use a symbolic link, or as sysadmins affectionately call it, a symlimk.

Let's say an application wants to install under /opt, and you are out of space in /opt, but have plenty of space in /app.  You can use a symlink to trick the system into thinking that it is using /opt, when in fact it is using /app/opt.

I like to perform a directory move and symlink creation at the same time.  Note that both /opt and /app already exist, but we are moving the entire /opt directory to be under /app, as /app/opt.

-->
mv /opt /app/ && ln -s /app/opt /opt

Note the use of double ampersand "&&", which tells the shell to run the next command, only if the result of the first command was successful.

Results of ls -l, should give something like this:

     lrwxrwxrwx   1 root   root       8 Oct  1 18:01 opt -> /app/opt


Sometimes this will fail, if there is not enough space at the target location, or if files are in use.  You can use fuser -c /opt to see which files are open, and if necessary, close them with fuser -ck /opt.  It may also be necessary to temporarily close any apps that are using the old partition.

Note that there can also be occasional problems with symlinks, for example mounting across different volume types or NAS shares, but a symlink can be a very convenient solution in many cases.

Thursday, June 28, 2018

Career Resources

I thought it might be helpful to point out some career resources for those who may be interested, or actively involved in job hunting.  I realize we all  need to update our resume from time to time, and some of you may not have a LinkedIn profile.

Here are some learning resources, and some useful links for career research.  Some of you know me as someone who is always promoting self-learning.

Learning Resources

Paid Learning Sites are often made available by your employer, so check to see if you have a subscription, or if you can expense it (get reimbursed for the expense).  It doesn't hurt to ask.



  • http://www.safaribooksonline.com -- Huge selection of technical books, videos, seminars, etc.
  • http://www.pluralsight.com -- High Quality, from overview to advanced topics for programmers
  • http://www.lynda.com -- High Quality, some topics are not very advanced, but courses are very well done
  • https://skillport.com -- Books, courses, videos



  • You can find career related courses by searching for “job search”, or “resume”,  "interview", or “career.”  For example, here are some courses I recommend in 2018:




    You may find it useful to look at job postings when updating your resume to see what kind of requirements are currently out there.  Then do some research:

    Also see this article for additional learning resources:  
    http://systemnotesorg.blogspot.com/2015/11/online-learning-resources.html


    Career Resources

    Connect and Organize
    http://linkedin.com – connect, research, apply
    http://www.jibberjobber.com - Organize and Manage your Career, job search, interviews, etc. - Free site at the time I started using it.

    Job Search
    http://linkup.com – job postings directly from companies, without 3rd party recruiters
    http://indeed.com -- Popular job search site
    https://www.usajobs.gov – US federal government jobs
    https://www.zoominfo.com -- for startups

    Company and Pay Info, also good for company research
    http://glassdoor.com – Jobs, Company Reviews, Salaries

    Paid Service
    https://www.righteverywhere.com – Excellent career management resources, including company research

    Some of these notes are also available in the public forum for job seekers.  This is something I quickly threw together in 2015 to help my coworkers who were impacted by a massive layoff:

    http://www.lifeisnojoke.com/upward



    Friday, January 5, 2018

    Use ssh instead of ping to check server status

    Server Availability cannot always be checked by using ping, since often a secured subnet will not allow traffic back.  That common practice of dropping packets is an attempt to hide servers and provide additional security, but it can make troubleshooting more difficult.

    Since ssh is typcially running and responding on any linux servers that need to be managed, check_ssh seems to be more practical than the default check_ping as the check_command for each host definition in Nagios.

    This simple shell script uses a Nagios plugin /usr/local/nagios/libexec/check_ssh to quickly check the status of a server.  One thing to note is that a Nagios installation is not required, just the plugins have to be compiled so the binary can be available to the wrapper script.

    #!/bin/sh
    
    # checkssh.sh
    #
    # Author: Scott McClelland
    # 2017-03-29
    #
    # checkssh, uses Nagios plugin check_ssh
    # /usr/local/nagios/libexec/check_ssh
    #
    # nagios-plugins are downloadable from:
    # https://www.nagios.org/downloads/nagios-plugins/
    
    if [ "$1" = "" ]
    then
    echo enter host name
     exit 1
    fi
    host=$1
    
    checkssh()
    {
    chkssh=/usr/local/nagios/libexec/check_ssh
    
    $chkssh $host  2>&1 >/dev/null
    RESULT=$?
    
    if [ "$RESULT" -eq "0" ]
    
    then
            echo $host : OK
    else
            echo $host : DOWN
    fi
    
            return $RESULT
    }
    
    
    Run the command with:
    
    
       checkssh $host

    To run the command on a group of servers, use a for loop.

    $ for s in web0{1..3} db0{1,2} xyz01; do ./chkssh.sh $s;done
    web01 : OK
    web02 : OK
    web03 : DOWN
    db01 : OK
    db02 : OK
    xyz01 : DOWN
    
    $
    

    Related article:  http://systemnotesorg.blogspot.com/2007/12/ping-multiple-hosts-using-bash-nmap-and.html

    Useful nmap Tips


    OS Fingerprint Scanning

    Sometimes there is a question about what OS is running on a server. There are advanced options from –help, but usually running nmap with -A is enough to tell you exactly what OS is running on the server in question.

    Simple nmap scan, which includes OS detection:

    nmap -v -A $s

    Find an Available IP

    Note that your company may have another system of record, so check there for the final authoritative answer.
    A quick way to find an IP that is not in DNS, and not responding to ping, or ssh, is to run an nmap scan of the subnet.

    Use nmap to find available IPs

     net=10.1.1.0;p=24;nmap -v -R -sn ${net}/${p} -o /tmp/subnet-${net}-${p}.txt
    Note: This may scan a large number of IPs, so it may be more convenient to write to a file to analyze later, rather tthan running the scan multiple times.
    An available IP would be one that shows an IP address with no DNS name, and also “host down.”