Site Feed

Tuesday, October 2, 2018

Create a Symlink as a Workaround for Partition Full Error

Sometimes, a mounted partition does not have enough space, and it may take time to expand the LVM volume, if there is not currently enough physical space available, and a separate team handles storage.  Of course making a separate partition is a more proper fix, and LVM is flexible enough to accommodate the changes, but sometimes space is filling up, and something has to be done right away.  Maybe /var/log, or /opt is filling up, and you did not think ahead to make it a separate partition.

As a quick workaround, it is fairly common to use a symbolic link, or as sysadmins affectionately call it, a symlimk.

Let's say an application wants to install under /opt, and you are out of space in /opt, but have plenty of space in /app.  You can use a symlink to trick the system into thinking that it is using /opt, when in fact it is using /app/opt.

I like to perform a directory move and symlink creation at the same time.  Note that both /opt and /app already exist, but we are moving the entire /opt directory to be under /app, as /app/opt.

-->
mv /opt /app/ && ln -s /app/opt /opt

Note the use of double ampersand "&&", which tells the shell to run the next command, only if the result of the first command was successful.

Results of ls -l, should give something like this:

     lrwxrwxrwx   1 root   root       8 Oct  1 18:01 opt -> /app/opt


Sometimes this will fail, if there is not enough space at the target location, or if files are in use.  You can use fuser -c /opt to see which files are open, and if necessary, close them with fuser -ck /opt.  It may also be necessary to temporarily close any apps that are using the old partition.

Note that there can also be occasional problems with symlinks, for example mounting across different volume types or NAS shares, but a symlink can be a very convenient solution in many cases.

Thursday, June 28, 2018

Career Resources

I thought it might be helpful to point out some career resources for those who may be interested, or actively involved in job hunting.  I realize we all  need to update our resume from time to time, and some of you may not have a LinkedIn profile.

Here are some learning resources, and some useful links for career research.  Some of you know me as someone who is always promoting self-learning.

Learning Resources

Paid Learning Sites are often made available by your employer, so check to see if you have a subscription, or if you can expense it (get reimbursed for the expense).  It doesn't hurt to ask.



  • http://www.safaribooksonline.com -- Huge selection of technical books, videos, seminars, etc.
  • http://www.pluralsight.com -- High Quality, from overview to advanced topics for programmers
  • http://www.lynda.com -- High Quality, some topics are not very advanced, but courses are very well done
  • https://skillport.com -- Books, courses, videos



  • You can find career related courses by searching for “job search”, or “resume”,  "interview", or “career.”  For example, here are some courses I recommend in 2018:




    You may find it useful to look at job postings when updating your resume to see what kind of requirements are currently out there.  Then do some research:

    Also see this article for additional learning resources:  
    http://systemnotesorg.blogspot.com/2015/11/online-learning-resources.html


    Career Resources

    Connect and Organize
    http://linkedin.com – connect, research, apply
    http://www.jibberjobber.com - Organize and Manage your Career, job search, interviews, etc. - Free site at the time I started using it.

    Job Search
    http://linkup.com – job postings directly from companies, without 3rd party recruiters
    http://indeed.com -- Popular job search site
    https://www.usajobs.gov – US federal government jobs
    https://www.zoominfo.com -- for startups

    Company and Pay Info, also good for company research
    http://glassdoor.com – Jobs, Company Reviews, Salaries

    Paid Service
    https://www.righteverywhere.com – Excellent career management resources, including company research

    Some of these notes are also available in the public forum for job seekers.  This is something I quickly threw together in 2015 to help my coworkers who were impacted by a massive layoff:

    http://www.lifeisnojoke.com/upward



    Friday, January 5, 2018

    Use ssh instead of ping to check server status

    Server Availability cannot always be checked by using ping, since often a secured subnet will not allow traffic back.  That common practice of dropping packets is an attempt to hide servers and provide additional security, but it can make troubleshooting more difficult.

    Since ssh is typcially running and responding on any linux servers that need to be managed, check_ssh seems to be more practical than the default check_ping as the check_command for each host definition in Nagios.

    This simple shell script uses a Nagios plugin /usr/local/nagios/libexec/check_ssh to quickly check the status of a server.  One thing to note is that a Nagios installation is not required, just the plugins have to be compiled so the binary can be available to the wrapper script.

    #!/bin/sh
    
    # checkssh.sh
    #
    # Author: Scott McClelland
    # 2017-03-29
    #
    # checkssh, uses Nagios plugin check_ssh
    # /usr/local/nagios/libexec/check_ssh
    #
    # nagios-plugins are downloadable from:
    # https://www.nagios.org/downloads/nagios-plugins/
    
    if [ "$1" = "" ]
    then
    echo enter host name
     exit 1
    fi
    host=$1
    
    checkssh()
    {
    chkssh=/usr/local/nagios/libexec/check_ssh
    
    $chkssh $host  2>&1 >/dev/null
    RESULT=$?
    
    if [ "$RESULT" -eq "0" ]
    
    then
            echo $host : OK
    else
            echo $host : DOWN
    fi
    
            return $RESULT
    }
    
    
    Run the command with:
    
    
       checkssh $host

    To run the command on a group of servers, use a for loop.

    $ for s in web0{1..3} db0{1,2} xyz01; do ./chkssh.sh $s;done
    web01 : OK
    web02 : OK
    web03 : DOWN
    db01 : OK
    db02 : OK
    xyz01 : DOWN
    
    $
    

    Related article:  http://systemnotesorg.blogspot.com/2007/12/ping-multiple-hosts-using-bash-nmap-and.html

    Useful nmap Tips


    OS Fingerprint Scanning

    Sometimes there is a question about what OS is running on a server. There are advanced options from –help, but usually running nmap with -A is enough to tell you exactly what OS is running on the server in question.

    Simple nmap scan, which includes OS detection:

    nmap -v -A $s

    Find an Available IP

    Note that your company may have another system of record, so check there for the final authoritative answer.
    A quick way to find an IP that is not in DNS, and not responding to ping, or ssh, is to run an nmap scan of the subnet.

    Use nmap to find available IPs

     net=10.1.1.0;p=24;nmap -v -R -sn ${net}/${p} -o /tmp/subnet-${net}-${p}.txt
    Note: This may scan a large number of IPs, so it may be more convenient to write to a file to analyze later, rather tthan running the scan multiple times.
    An available IP would be one that shows an IP address with no DNS name, and also “host down.”

    Saturday, December 30, 2017

    How to Obtain Bitcoin and Other Cryptocurrencies

    How to Buy, Accept, or Mine Bitcoin and Other Cryptocurrencies - Quick Intro

    Buy Bitcoin on an Exchange

    One easy way to get started with owning bitcoin is to buy some on an exchange, such as Coinbase.

    I offer this link because I think it is helpful, not just because I could get $10.  Sometimes you can't tell why people recommend things.  I figure, if you plan to sign up anyway, why not save a little money?

    With this referral link, we each get $10 in bitcoin after the new account is opened, and a trade of $100 or more is completed:
    https://www.coinbase.com/join/592f371c2ae3540ae4a4eb70

    Note that it is not necessary to buy a full coin, but there are commissions for buying and selling, and  mining fees for transfers.   To minimize commissions, try entering different amounts to see how much makes sense, before completing the transaction.  For example, buying $100 of bitcoin may cost as much in commissions as buying $200 worth, or around $3.00.  In 2017, commissions have been around 1.5%.

    Other ways to get bitcoin and related assets

    Buy stocks related to bitcoin, blockchain, or cryptocurrency

    For a while, to buy a stock that invests directly in bitcoin, pretty much all that was available was GBTC.  Late in 2017, the list started to grow quickly, especially after bitcoin became available on the commodities markets.  We may expect to see an ETF at some time.  A search on some of those keywords will turn up some new companies, and there are also companies that create hardware for mining.

    Accept bitcoin as payments, tips, or gifts

    This requires a crypto currency wallet, or account on an exchange that manages the wallet.  Probably the quicket way to start receiving bitcoin is to install a local wallet such as Jaxx on a smartphone.  If using an exchange such as Coinbase, there is also an app that can be used to receive bitcoin.

    Speaking of tips, some websites may offer a link where they would gladly except tips.  Here is ours for systemnotes.org, of you are so inclined:

    BTC:  1Wqg1EeNuuURFA39cYAQnmgijBQANW7yB
    ETC:  0xbf5c22ef84eeb0c84bf0e70be5beb28379d0e278
    LTC:  LasBvJfk7ypwic1HjKaUtGqZ9MwisXMMG9
    BCH:  1HqnEB8xy7Nmfm6GuY7iXeA1eEvf69HthT

    Mine Bitcoin

    Cryptocurrency Mining continues to require faster and more specialized hardware, and the difficulty continues to increase, but as prices increase the investment could pay off sooner than expected.  This could be an entire series of articles, but for now, just understand that this is a way to obtain bitcoin.

    Cloud Mining

    A sub topic of mining is cloud mining, which is a good way to learn how the process works, but profitiability is questionable.

    Where to Store Bitcoin 

    Crypto currency is actually stored on the blockchain, and is publicly visible to all.  The way to control transactions is with the use of public and private keys.  Funds are received by sharing the public key with the sender.  The way to send funds, is with the public, and private keys, and possibly with a password, or PIN.  The way to recover a wallet, is ususally with a 12 word passphrase.  The public key, password, and passphrase should never be shared with anyone, other than someone who should have full access to all the funds, such as  a spouse, or the recepient of an inheritance.

    Caution must be taken to never lose the keys and passphrase.  If a wallet or device is lost or damaged,  the wallet can be recovered using the secrets, but if they are lost, the funds are gone forever.  That means you need to have multiple backups that you can use to recover, but not allow the secret information to be used by anyone else, since they could steal all your funds.

    Exchange - OK for small amounts, and short periods of time. Most of the big exchanges are relatively safe, but they control your keys, so there is always a risk they could be hacked.  Also, make sure to remember the username and password.  Again, this is probably the easiest way to get started but consider other wallets for larger amounts.

    Software Wallet, or Hot wallet - A software app, where keys are usually stored locally on the device, e.g. a computer, phone, or tablet. More control of the keys, but still a risk that the device could be hacked. Usually recommended for small amounts of spending or trading money.

    Hardware wallet, or cold storage wallet - A physical device which is safest as far as privacy and control of your keys, since the wallet is disconnected from the network. The biggest risk is losing the keys and passphrase.  If the device is lost, or damaged, the wallet can still be recovered from the pass phrase that was written down somewhere.

    Some popular choices are:
     Ledger Nano S
     Trezor Wallet

    Notes:  Other sites have recommendations, and reviews for each of the items mentioned.  This article is meant to be a quick introduction.  In the future this information will probably be well known, and obvious, such as how to open a bank account, or how to use a debit card, checkbook, or an ATM.  All those banking activities are now common knowledge, but in 2018, how to deal with cryptocurrencies is still not widely known.

    Wednesday, March 22, 2017

    Discover Switch Port Using tcpdump and wireshark

    Discover the Switch Port to which the Server is Connected

    A previous article, Advanced Linux Networking Tools, covered the basic usage of tcpdump and tshark to discover switch ports on a Cisco switch, but there are times a system may be connected to a different brand, or using a different protocol.  Also, wireshark is not always installed, so relying on tshark is not always the most convenient way to get the required info.

    By default, Cisco uses a proprietary method of communication between switches and routers called Cisco Discovery Protocol (CDP).  There is another protocol called Link Layer Discovery Protocol (LLDP), which used by other brands, so it is useful to list other options.

    Find switch information

    First, use ifconfig to find the interface names.  Then, use tcpdump to listen for packets.  Optional:  write the packet capture output to a .cap file, and use tshark to read the output.

    Cisco Discovery Protocol (CDP)

    YOUR_INTERFACE=eth0
     tcpdump -n -v -i eth0 -s 1500 -c 1 'ether[20:2] == 0x2000'

    Link Layer Discovery Protocol (LLDP)
    YOUR_INTERFACE=eth0
     tcpdump -v -s 1500 -c 1 -i $YOUR_INTERFACE '(ether[12:2]=0x88cc)'

    tcpdump -v -s 1500 -c 1 -i $YOUR_INTERFACE '(ether[12:2]=0x88cc)'  and ether dst 01:80:c2:00:00:0e

    With Wireshark

    sudo tcpdump -nv -c 1 -i eth0 -s 1500 '(ether[12:2]=0x88cc)' 

     sudo tcpdump -nv -c 1 -s 1500 -w /tmp/pkt1.cap -i bond1 '(ether[12:2]=0x88cc)'
      sudo tshark -V -r /tmp/pkt1.cap

    Check LLDP on eth0
    sudo tcpdump -nv -c 1 -i eth0 -s 1500 -w /tmp/pkt0.cap '(ether[12:2]=0x88cc)';sudo tshark -V -r /tmp/pkt0.cap

    Limit Results by using egrep
    sudo tcpdump -nv -c 1 -i eth0 -s 1500 -w /tmp/pkt0.cap '(ether[12:2]=0x88cc)';sudo tshark -V -r /tmp/pkt0.cap | egrep -i "Chassis Id: | Port Id: | System Name = |port Description ="


     Without Wireshark


    CDP
     sudo  tcpdump -n -v -i eth0 -s 1500 -c 1 'ether[20:2] == 0x2000'
     sudo  tcpdump -n -v -i bond1 -s 1500 -c 1 'ether[20:2] == 0x2000'

    LLDP
     sudo tcpdump -nv -c 1 -i eth0 -s 1500 '(ether[12:2]=0x88cc)'
     sudo  tcpdump -n -v -i bond1 -s 1500 -c 1  '(ether[12:2]=0x88cc)'

    Sometimes, a little patience is required.  After all, tcpdump is listening for packets, so it may take a minute, or so for a packet to show up on the interface of interest.  Also, if the interface is bonded, then bond0 may be the name of the interface to use instead of something like eth0.

    Bash One-Liners for Ping

    Here are a few notes to add to the previous article on ping. This time, we look at some bash one-liner tips and tricks. Combine multiple commands Return Values Ping Multiple Hosts Using Bash Nmap and Fping

    Ping, and Command Line Variables

    Start from the beginning. We want to see what happens when attempting to ping a host that resolves an IP address from DNS, but is not reachable from our network at the moment. This is to show how we might build a simple monitoring tool from scratch, and also to see what kind of fun we can have with the command line. To ping a host only one time use the count option, which in Windows is -n, and in Linux is -c.

    Windows

    C:\>set host=google.com
    
    C:\>ping -n 1 %host%
    
    Pinging google.com [216.58.219.46] with 32 bytes of data:
    Request timed out.
    
    Ping statistics for 216.58.219.46:
        Packets: Sent = 1, Received = 0, Lost = 1 (100% loss),
    

    Bash: Linux, Mac, Unix

    Note that in bash, we don't use the set command to assign a variable, but we may want to use export. So we could use:
    host=google.com
    ping -c 1 $host
    
    Or with export:
    export host=google.com
    ping -c 1 $host
    
    The only difference should be that the variable is still available in a sub shell, if we use export. With both lines combined, use a semicolon.
    host=google.com;ping -c 1 $host
    

    Return Codes

    Remember that commands will return a result that is placed in the $? variable, which can be used later in our script. Return codes for ping:
     0 = success (host is up)
     1 = ping failed (host is down)
     2 = unknown host (host is not in DNS)
    
    To see how this works with an unknown host, run these two commands. First, ping an non-existent host, and then echo the $? variable.
    $ ping -c 1 xyz-nothing
    ping: unknown host xyz-nothing
    $ echo $?
    2
    
    The same thing, combined on one line, with a host variable:
    $  host=xyz-nothing;ping -c 1 $host;echo $?
    ping: unknown host xyz-nothing
    2
    
    If a ping check failed, then give the hostname, the return code, and the date and time.
    $ host=google.com;ping -c 1 $host > /dev/null 2>&1; RESULT=$?;echo $host: $RESULT; if [ "$RESULT" != "0" ]; then date --rfc-3339=ns;fi
    google.com: 1
    2016-09-01 14:35:10.833175095-07:00
    
    Or a simplified version, using the logical "or" double pipe "||", though we need the parentheses to run multiple commands
    $ host=google.com;ping -c 1 $host > /dev/null 2>&1 || ( echo "$host $RESULT";date --rfc-3339=ns )                            
    google.com 1
    2016-09-01 14:35:31.012810392-07:00