Python Quick Notes :: Part - 3

Hands-on With Python - III

Python provides excellent support for file IO. The following are some of the core file IO functions:

• fo = open(name, mode) - open a file with specified name in the specified mode. Returns a file object fo on success. The following are some of the commonly used modes:

• r - read mode with file pointer at the beginning of the file

• r+ - read and write mode with file pointer at the beginning of the file

• w - write mode with file pointer at the beginning of the file. Existing file will be overwritten

• w+ - read and write mode with file pointer at the beginning of the file. Existing file will be overwritten

• a - append write mode with file pointer at the end of the file

• a+ - append write and read mode with file pointer at the end of the file

• fo.close() - close the file using the file object fo

• fo.read() - read and return all the contents of the file using the file object fo

• fo.read(size) - read and return the specified size of bytes from the file using the file object fo

• fo.readline() - read and return a single line (with the newline character) from the file using the file object fo

• fo.readlines() - read and return a list of all the lines (with the newline character) from the file using the file object fo

• fo.write(s) - write the contents of the specified string s to the file using the file object fo

• fo.writelines(l) - write the specified list of lines l to the file using the file object fo

The following are some of the commonly used file object fo attributes:

• fo.name - represents the name of the file

• fo.mode - represents the mode with which the file was opened

• fo.closed - returns True if the file is closed

The following is the python program named sample-13.py:

#
# Name: sample-13.py
#

# File IO

import os

fname = "/tmp/test_file"

print("---> Opening file /tmp/test_file in write mode")

fo = open(fname, "w")

print("fo.name =", fo.name)
print("fo.mode =", fo.mode)
print("fo.closed =", fo.closed)

print("---> Writing to file /tmp/test_file")

fo.write("Hello, Welcome to the exciting world of Python\n")

print("---> Closing file /tmp/test_file")

fo.close()

print("---> Opening file /tmp/test_file in read mode")

fo = open(fname, "r")

print("---> Reading from file /tmp/test_file")

while True:
    ln = fo.readline()
    if not ln:
        break
    print("---> Read line: ", ln)

print("---> Closing file /tmp/test_file")

fo.close()

print("---> Removing file /tmp/test_file")

os.remove(fname)

Execute the following command:

python sample-13.py

The following is the output:

Output (sample-13.py)

---> Opening file /tmp/test_file in write mode
fo.name = /tmp/test_file
fo.mode = w
fo.closed = False
---> Writing to file /tmp/test_file
---> Closing file /tmp/test_file
---> Opening file /tmp/test_file in read mode
---> Reading from file /tmp/test_file
---> Read line:  Hello, Welcome to the exciting world of Python

---> Closing file /tmp/test_file
---> Removing file /tmp/test_file

What happens when we try to open a file for read that does not exist ? This is an error condition and Python runtime raises an exception. In other words, an exception is an unexpected error that occurs during program execution. To handle an exception we use the try-except-finally statement which has the following syntax:

try:     try-code-block except type-a:     type-a-code-block except type-b, info-arg:     type-b-code-block else:     else-code-block finally:     finally-code-block

where:

• The try-code-block after the try: statement is one or more Python operation(s) that needs to be executed which can potentially raise exception(s)

• type-a (or type-b) is a particular type of exception raised

• type-a-code-block (or type-b-code-block) is the block of code we want to execute when the particular type of exception is raised

• info-arg is the optional argument that can give additional information about the exception

• The except: statement without any exception type is for handling any exception

• The else-code-block after the else: statement is executed if no exception

• The Python statement finally: and the corresponding finally-code-block is an optional and if present will be executed irrespective of an exception being raised

To raise an exception, use the syntax: raise exception, info-arg.

The following is the python program named sample-14.py:

#
# Name: sample-14.py
#

# File IO Exception

fname = "/tmp/test_file"

print("---> 1. Opening file /tmp/test_file in read mode")

try:
    fo = open(fname, "r")
except IOError:
    print("*** 1. File " + fname + " not found !!!")
else:
    print("---> 1. File " + fname + " successfully opened for read")

print("---> 2. Opening file /tmp/test_file in read mode")

try:
    fo = open(fname, "r")
except IOError as e:
    print("*** 2. File " + fname + " not found !!!\n", e)
else:
    print("---> 2. File " + fname + " successfully opened for read")

print("---> 3. Opening file /tmp/test_file in read mode")

try:
    fo = open(fname, "r")
except:
    print("*** 3. File " + fname + " not found !!!")
finally:
    print("---> 3. In the finally block")

print("---> Raise an exception")

n = -1

try:
    if n < 0:
        raise RuntimeError("n cannot be negative")
except RuntimeError as e:
    print(e)

Execute the following command:

python sample-14.py

The following is the output:

Output (sample-14.py)

---> 1. Opening file /tmp/test_file in read mode
*** 1. File /tmp/test_file not found !!!
---> 2. Opening file /tmp/test_file in read mode
*** 2. File /tmp/test_file not found !!!
[Errno 2] No such file or directory: '/tmp/test_file'
---> 3. Opening file /tmp/test_file in read mode
*** 3. File /tmp/test_file not found !!!
---> 3. In the finally block
---> Raise an exception
n cannot be negative

Python's standard library module pickle provides excellent support for object pickling (serialization) and unpickling (deserialization). The following are some of the commonly used methods from the pickle class:

• pickle.dump(obj, fo) - Serialize the specified object obj to the specified file object fo

• pickle.load(fo) - Deserialie previously serialized object from the specified file object fo

The following is the python program named sample-15.py:

#
# Name: sample-15.py
#

# ----- Python Pickling and Unpickling -----

import pickle
import time
import os

# ----- Start Definition the class StickyNote -----

class StickyNote:

    """ This is a test class to demostrate pickle/unpickle"""

    def __init__(self):
        self.tag = ""
        self.time = ""
        self.lines = []

    def getTag(self):
        return self.tag

    def getTime(self):
        return self.time

    def setTag(self, tag):
        self.tag = tag

    def setTime(self, time):
        self.time = time

    def addLine(self, line):
        self.lines.append(line)

    def __str__(self):
        return """*** StickyNote ***
    Tag: %s
    Time: %s
    %s""" % (self.tag, self.time, self.lines)

# ----- End Definition the class StickyNote -----

fname = "/tmp/test_file"

# Create a StickyNote instance

snote1 = StickyNote()
snote1.setTag("Test")
snote1.setTime(time.ctime())
snote1.addLine("1. Check Pickle")
snote1.addLine("2. Check UnPickle")

# Open file for saving the sticky note - pickle

print("Ready to pickle .....")

print(snote1)

try:
    fo = open(fname, "wb")
    pickle.dump(snote1, fo)
    fo.close()
except RuntimeError as e:
    print(e)

# Open file for retrieving the sticky note - unpickle

print("Ready to unpickle .....")

try:
    fo = open(fname, "rb")
    snote2 = pickle.load(fo)
    fo.close()
    print(snote2)
except RuntimeError as e:
    print(e)

# Remove file

os.remove(fname)

Execute the following command:

python sample-15.py

The following is the output:

Output (sample-15.py)

Ready to pickle .....
*** StickyNote ***
    Tag: Test
    Time: Sat Mar  2 21:42:28 2013
    ['1. Check Pickle', '2. Check UnPickle']
Ready to unpickle .....
*** StickyNote ***
    Tag: Test
    Time: Sat Mar  2 21:42:28 2013
    ['1. Check Pickle', '2. Check UnPickle']

Python's standard library module threading provides excellent support for multi-threaded programming. The following are some facts about threads in Python:

• There are two ways of creating a new thread in Python:

• Method-1 :: Creating an instance of a thread using the default threading.Thread class and specifying the target method to be executed in the thread of control

• Method-2 :: Creating an instance of a thread by subclassing the threading.Thread class and overriding the run() method

• The following are some of the commonly used methods from the Thread class:

• threading.currentThread() - returns the current thread object in the caller's thread of control

• threading.enumerate() - returns a list of thread objects that are currently active

• start() - starts a thread

• getName() - get the name of the thread

• join() - wait for the thread to terminate

The following is the python program named sample-16.py demonstrates multi-threading using Method-1:

#
# Name: sample-16.py
#

# ----- Python Multi-Threading -----

import threading
import time

def hello():
    """Test method to be invoked in a thread"""
    name = threading.currentThread().getName()
    print("[" + time.ctime() + "] - <" + name + "> :: Hello Python !!!!!")
    return

def message(txt):
    """Test method to be invoked in a thread"""
    name = threading.currentThread().getName()
    print("[" + time.ctime() + "] - <" + name + "> :: " + txt)
    return

# Create a thread using the Thread class

thr1 = threading.Thread(target=hello)
thr1.start()

thr2 = threading.Thread(name="HelloThread", target=hello)
thr2.start()

thr3 = threading.Thread(target=message, args=('Aloha Python',))
thr3.start()

thr1.join()
thr2.join()
thr3.join()

Execute the following command:

python sample-16.py

The following is the output:

Output (sample-16.py)

[Sat Mar  2 22:11:42 2013] - <Thread-1> :: Hello Python !!!!!
 [Sat Mar  2 22:11:42 2013] - <HelloThread> :: Hello Python !!!!!
[Sat Mar  2 22:11:42 2013] - <Thread-2> :: Aloha Python

The following is the python program named sample-17.py that demonstrates multi-threading using Method-2:

#
# Name: sample-17.py
#

# ----- Python Multi-Threading Using Subclassing -----

import threading
import time

# Subclass threading.Thread

class MyThread(threading.Thread):
    def __init__(self, name, txt, cnt):
        super(MyThread, self).__init__()
        self.setName(name)
        self.txt = txt
        self.cnt = cnt

    def run(self):
        """Test method to be invoked in a thread"""
        name = threading.currentThread().getName()
        print("[" + time.ctime() + "] - <" + name + "> :: " + self.txt)
        for i in range(1, self.cnt+1):
            print("<" + name + "> :: count = " + str(i))
            time.sleep(5)
        return


# Create a thread using the Thread class

thr1 = MyThread('MyThread-1', 'Hello Python', 5)
thr2 = MyThread('MyThread-2', 'Aloha Python', 5)
thr3 = MyThread('MyThread-3', 'Ola Python', 5)

# Start all the threads

thr1.start()
thr2.start()
thr3.start()

# Enumerate threads

mthr = threading.currentThread()

for th in threading.enumerate():
    if th is mthr:
        continue
    th.join()
    print("---> Joined thread " + th.getName())

Execute the following command:

python sample-17.py

The following is the output:

Output (sample-17.py)

[Sat Mar  2 22:41:11 2013] - <MyThread-1> :: Hello Python
<MyThread-1> :: count = 1
[Sat Mar  2 22:41:11 2013] - <MyThread-2> :: Aloha Python
<MyThread-2> :: count = 1
 [Sat Mar  2 22:41:11 2013] - <MyThread-3> :: Ola Python
<MyThread-3> :: count = 1
<MyThread-1> :: count = 2
 <MyThread-2> :: count = 2
<MyThread-3> :: count = 2
<MyThread-1> :: count = 3
<MyThread-3> :: count = 3
 <MyThread-2> :: count = 3
<MyThread-2> :: count = 4<MyThread-1> :: count = 4
<MyThread-3> :: count = 4
<MyThread-2> :: count = 5 <MyThread-3> :: count = 5
<MyThread-1> :: count = 5
---> Joined thread MyThread-1
---> Joined thread MyThread-2
---> Joined thread MyThread-3

To control access to a shared resource among multiple threads, Python's standard library module threading provides support for synchronization using the Lock class. The following are the two commonly used methods in the Lock class:

• acquire() - Acquire the synchronization lock

• release() - Release the synchronization lock

The following is the python program named sample-18.py that demonstrates locking mechanism among multiple threads using the Lock class:

#
# Name: sample-18.py
#

# ----- Python Multi-Thread Synchronization -----

import threading
import time

# ----- Start Definition the class SeqNo -----

class SeqNo:

    """ This is a test class to demonstrate locking to access
    shared resources by threads"""

    LOCK = threading.Lock()

    def __init__(self):
        self.seqno = 0

    def getNextSeqno(self):
        name = threading.currentThread().getName()
        SeqNo.LOCK.acquire()
        print("[" + time.ctime() + "] - <" + name + "> : Acquired lock")
        self.seqno += 1
        seq = self.seqno
        SeqNo.LOCK.release()
        print("[" + time.ctime() + "] - <" + name + "> : Released lock")
        return seq

# ----- End Definition the class SeqNo -----

# Create an instance of SeqNo

seqno = SeqNo()

# Define thread worker to fetch the next seqno

def worker():
    """Test method to be invoked in a thread"""
    name = threading.currentThread().getName()
    for i in range(1, 6):
        sno = seqno.getNextSeqno()
        print("<" + name + "> :: seqno = " + str(sno))
        time.sleep(5)
    return

# Create and start threads

thr1 = threading.Thread(target=worker)
thr2 = threading.Thread(target=worker)

thr1.start()
thr2.start()

thr1.join()
thr2.join()

Execute the following command:

python sample-18.py

The following is the output:

Output (sample-18.py)

[Sun Mar  3 13:10:04 2013] - <Thread-1> : Acquired lock
[Sun Mar  3 13:10:04 2013] - <Thread-1> : Released lock
 [Sun Mar  3 13:10:04 2013] - <Thread-2> : Acquired lock
<Thread-1> :: seqno = 1
[Sun Mar  3 13:10:04 2013] - <Thread-2> : Released lock
<Thread-2> :: seqno = 2
[Sun Mar  3 13:10:09 2013] - <Thread-2> : Acquired lock
[Sun Mar  3 13:10:09 2013] - <Thread-2> : Released lock
<Thread-2> :: seqno = 3
[Sun Mar  3 13:10:09 2013] - <Thread-1> : Acquired lock
[Sun Mar  3 13:10:09 2013] - <Thread-1> : Released lock
<Thread-1> :: seqno = 4
[Sun Mar  3 13:10:14 2013] - <Thread-2> : Acquired lock
[Sun Mar  3 13:10:14 2013] - <Thread-2> : Released lock
 [Sun Mar  3 13:10:14 2013] - <Thread-1> : Acquired lock
<Thread-2> :: seqno = 5
[Sun Mar  3 13:10:14 2013] - <Thread-1> : Released lock
<Thread-1> :: seqno = 6
[Sun Mar  3 13:10:19 2013] - <Thread-2> : Acquired lock
[Sun Mar  3 13:10:19 2013] - <Thread-2> : Released lock
 [Sun Mar  3 13:10:19 2013] - <Thread-1> : Acquired lock
<Thread-2> :: seqno = 7
[Sun Mar  3 13:10:19 2013] - <Thread-1> : Released lock
<Thread-1> :: seqno = 8
[Sun Mar  3 13:10:24 2013] - <Thread-2> : Acquired lock
[Sun Mar  3 13:10:24 2013] - <Thread-2> : Released lock
 [Sun Mar  3 13:10:24 2013] - <Thread-1> : Acquired lock
<Thread-2> :: seqno = 9
[Sun Mar  3 13:10:24 2013] - <Thread-1> : Released lock
<Thread-1> :: seqno = 10

Python's standard library module Queue provides support for a thread-safe first-in first-out (FIFO) queue data structure in the Queue class that is critical for solving Producer-Consumer type problems. One can specify the maximum size of the queue when an instance of Queue is created. The following are the commonly used methods in the Queue class:

• empty() - Returns True if the queue is empty

• full() - Returns True if the queue was created with a maximum size and the queue is full

• get() - Remove and return an object from the queue. Thi operation will block if the queue is empty and until some object is put into the queue

• put(obj) - Put the specified object obj into the queue. If the queue was created with a maximum size and the queue is full, this operation will block until some free space becomes available

• qsize() - Returns the number of objects in the queue

The following is the python program named sample-19.py that demonstrates the Producer-Consumer problem using the Queue class:

#
# Name: sample-19.py
#

# ----- Producer-Consumer problem -----

import queue
import threading
import time

# Jobs to process

jobs = ["one", "two", "three", "four", "five"]

# Create queue of size 3

queue = queue.Queue(3)

# Define consumer thread method

def consumer():
    """Consumer method to be invoked in a thread"""
    name = threading.currentThread().getName()
    while True:
        job = queue.get()
        print("[" + time.ctime() + "] <" + name + "> :: Consumed " + job)
    return

# Define producer method

def producer():
    """Producer method to be invoked in a thread"""
    name = threading.currentThread().getName()
    for s in jobs:
        queue.put(s)
        print("[" + time.ctime() + "] <" + name + "> :: Produced " + s)
    return

# Create and start prodcuer and consumer threads

thr1 = threading.Thread(name='Producer', target=producer)
thr2 = threading.Thread(name='Consumer-A', target=consumer)
thr3 = threading.Thread(name='Consumer-B', target=consumer)

thr2.start()
thr3.start()
thr1.start()

Execute the following command:

python sample-19.py

The following is the output:

Output (sample-19.py)

[Sun Mar  3 14:22:23 2013] <Producer> :: Produced one
[Sun Mar  3 14:22:23 2013] <Producer> :: Produced two
[Sun Mar  3 14:22:23 2013] <Consumer-B> :: Consumed one
[Sun Mar  3 14:22:23 2013] <Consumer-B> :: Consumed two
[Sun Mar  3 14:22:23 2013] <Producer> :: Produced three
[Sun Mar  3 14:22:23 2013] <Consumer-A> :: Consumed three
[Sun Mar  3 14:22:23 2013] <Producer> :: Produced four
 [Sun Mar  3 14:22:23 2013] <Consumer-A> :: Consumed four
[Sun Mar  3 14:22:23 2013] <Producer> :: Produced five
[Sun Mar  3 14:22:23 2013] <Consumer-A> :: Consumed five

References

Python Quick Notes :: Part - 1

Python Quick Notes :: Part - 2