Pages

Tuesday, January 17, 2017

Tons of blogger bug, difficulties encountered in the past, pilot-testing other blogger platforms, so far all anything seems better than blogger.com.

Last incidence: the text will exceed to left margin and couple of letters on the left will not be visible! Amateur quality!

Saturday, December 17, 2016

python thread management

Python thread management example. Here in this example, I have attempted to do something: two thread communicate to each other, but in terms of threads: create two threads 1 and 2 and tell thread1 tell thread2 something and as a result thread2 takes some action. This probably is quite rudimentary method compared to sophisticated IPC mechanism where threads or process communicate through semaphore or locks.

Specifically, the thread 2 in the example below will start a counter and if the counter reaches 5 it sets the global variable endThreadOne to 1 which thread 1  will also check it periodically. Once it founds out that the flag is set to 1, it will exit itself. Since it is python's thread library, thread is represented by function instead of object and exiting the function will effectively ends the thread.

Credits for giving fast and furious information to get me started in no time to following:

This gives very good basic intro-s on basics of python thread:
http://www.techbeamers.com/python-multithreading-concepts/

Good example which I extended as described above: 
https://www.tutorialspoint.com/python/python_multithreading.htm

#!/virtualenv/bin/python

import thread
import time
import os

global endThreadOne
print "setting end thread to No=0: "
endThreadOne = 0

TTHREAD_1_NAME = "Thread-1: "
TTHREAD_2_NAME = "Thread-2: "

# Define a function for the thread

def printStatThread( threadName, delay):
    count = 0
    global endThreadOne

    print "\nendThreadOne stat: ", endThreadOne

    while count < 20:
        time.sleep(delay)
        count += 1
        print "    -", threadName, count

        if threadName == TTHREAD_2_NAME and count > 5 and endThreadOne == 0:
            print threadName, "Telling thread 1 to exit."
            endThreadOne = 1
        if threadName == "Thread-1: " and endThreadOne == 1:
            print threadName, "I was told to exit. I am exiting."
            return 0

    print threadName, "Timeout, I am exiting"


# Create two threads as follows

try:
    thread.start_new_thread( printStatThread, (TTHREAD_1_NAME, 2, ) )
    thread.start_new_thread( printStatThread, (TTHREAD_2_NAME, 4, ) )
except:
    print "Error: unable to start thread"

#os.system("clear")
counter = 0


while 1:
    time.sleep(5)

    if counter > 10:
        print "Exiting."
        break

    counter += 1
    print "1. loop " + str(counter)
    continue

print "done"


Resulting output appears to do what it is intended to do. Both thread starts and started looping using count and prints its thread name along with counter. Once thread 2 detects counter is reached to 5, it sets the global flag which thread 1 uses it to end itself.

endThreadOne stat:  0
endThreadOne stat:
 0
    - Thread-1:  1
    - Thread-2:  1
    - Thread-1:  2
1. loop 1
    - Thread-1:  3
    - Thread-2:  2
    - Thread-1:  4
1. loop 2
    - Thread-1:  5
    - Thread-2:  3
    - Thread-1:  6
    - Thread-1:  7
1. loop 3
    - Thread-2:  4
    - Thread-1:  8
    - Thread-1:  9
1. loop 4
    - Thread-2:  5
    - Thread-1:  10
    - Thread-1:  11
    - Thread-2:  6
Thread-2:  Telling thread 1 to exit.
    - Thread-1:  12
Thread-1:  I was told to exit. I am exiting.
1. loop 5
    - Thread-2:  7
1. loop 6
    - Thread-2:  8
1. loop 7
    - Thread-2:  9
1. loop 8
    - Thread-2:  10
    - Thread-2:  11
1. loop 9
    - Thread-2:  12
1. loop 10
    - Thread-2:  13
1. loop 11
    - Thread-2:  14
Exiting.
done

Sunday, December 11, 2016

linux shell script sed with regexp expression examples

Simple regexp expression using sed. In this examples, grep output is piped to sed. Sed in turn looks for .*= expression and replaces with empty. /s signifies substitution operation. Resulting value is fed into MOUNT_USER[$i] variable.
This is most commonly used cases for me:

MOUNT_USERS[$i]=`cat $PASSWORD_FILE | grep ${MOUNT_USERS_LABELS[$i]} | sed -r 's/.*=//'`

Following examples shows the replacement of every occurrence of \. and replaces with - in var2 and result is fed into var1. Notice that the var2 is the variable holding the name of file which is output using cat command which is piped into sed expression. Last part of expression '/g' signifies that it wants to replace every occurrence.
 
var1=`cat $var2 | sed -r 's/\./-/g'`
 



linux shell syntaxes

Here in the examples, I published commonly used linux shell syntaxes that are commonly looked up.

  • Simple variable assignment and referring it later:
 LINPACK_NAME=l_lpk_p_11.1.2.005.tgz
echo $LINPACK_NAME

  • Declaring arrays:
MISC_SCRIPTS=(
        poll.curr.freq.sh \
        reconfig.multi.eth.py \
        init.satool.sh \
)


  • IF and FOR loop example, also test if file exists:

for i in "${PASSWORD_LOCS[@]}"
do
    echo looking for password file in $i

    if [ -s $i/passwd.log ]
    then
        echo found the password file in $i.
        PASSWORD_FILE=$i/passwd.log
        break
    fi
done

  • For loop over array structures, as well as checking if directory exists:
for i in "${DIRECTORIES[@]}"
do
        echo ---------------
        if [ ! -d $i ]
        then
                mkdir $i
        else
                echo $i already exist.
        fi
done

  • For loop over array structures and referencing another array elements using index:

for (( i=0; i < ${#MOUNT_SHARES[@]}; i++ ))
do
    echo ---------------------------
    echo setting user credentials for ${MOUNT_USERS_LABELS[$i]} ...

    # if user/pw is not defined at this point.

    if [ ${MOUNT_USERS[$i]} == 0 ] || [ ${MOUNT_PWS[$i]} == 0 ]
    then

...


    fi
done
 
  • Check if variable is empty 
if [ -z $PASSWORD_FILE ]
then
    echo unable to locate password file.
else
    echo password file: $PASSWORD_FILE
fi


The trick here is the if the variable is not empty, the above snippet will fail with syntax error, here is the better version, that works with both empty and non-empty cases by enclosing the variable within double-quote:
 

if [ -z "$PASSWORD_FILE" ]
then
    echo unable to locate password file.
else
    echo password file: $PASSWORD_FILE
fi

  • Case structure as well as default case.
case "$RHEL_VERSION" in
        "Red Hat Enterprise Linux Server release 6.7 (Santiago)")
        echo "RHEL 6.7 is detected"
        RHEL_ISO_PATH=$RHEL67_PATH_STRING
        ;;
        "Red Hat Enterprise Linux Server release 6.8 (Santiago)")
        echo "RHEL 6.8 is detected"
        RHEL_ISO_PATH=$RHEL68_PATH_STRING
        ;;
        "Red Hat Enterprise Linux Server release 7.2 (Maipo)")
    echo "RHEL 7.2 is detected"
        RHEL_ISO_PATH=$RHEL72_PATH_STRING
        ;;
        *)
    echo "Unknown linux version."
        RHEL_ISO_PATH=$RHEL_VERSION_UNSUPPORTED
        RHEL_VERSION=$RHEL_VERSION_UNSUPPORTED
esac



  • Test if two strings are equal:

if [ "$RHEL_ISO_PATH" == $RHEL72_PATH_STRING ]
then
        yum-config-manager --enable
else
        echo no need for yum-config-manager --enable
fi








linux shell intro

With increasing need to do a shell scripting, I decided to take some notes on shell scripting specially for sed using regexp expression. It turns out the regexp expression syntax used in shell script is much different than python regexp. In addition, in this part of the blog, I also decide to note the syntaxes for various linux shell control flows, variable declaration and assignment, operator syntaxes.

Here are some starting notes:

I am taking examples based on two variables VAR1 is the string on which regexp is operating on and VAR2 is the result of the regexp operation.

From the example below the cat $var2's value will be printed and piped to sed expression.
From the output of cat, <searching string> is searched and replaced by <replacement string>. s/ signifies it is a substitution operaion:


var2=`cat $var2 | sed -r 's/<search string>/<replacement string>/'`



Friday, November 11, 2016

python paramiko example

Here I post the simple working paramiko example. The web has full of junk on the usage of paramiko, conflicting information, incomplete, inaccurate documentation. Had to sift through a lot of garbage to get this working. I am not a fun of python programming language (it is not actually a real programming language, rather poorly documented scripting language that runs slow by default) at all just has to use it.

This is done RHEL6.5 with Python 2.7.

yum install python-paramiko.noarch -y


#!/usr/bin/python
import paramiko

try:
    client = paramiko.SSHClient()
    client.load_system_host_keys()
    client.set_missing_host_key_policy(paramiko.WarningPolicy())

    client.connect("10.0.0.26", port="22", username="root", password=<password>)

    stdin, stdout, stderr = client.exec_command("ls -l /")
    print stdout.read(),

finally:
    client.close()

print "done."