Multithreading - Subclassing Thread
Python Multithread
Creating a thread and passing arguments to the threadIdentifying threads - naming and logging
Daemon thread & join() method
Active threads & enumerate() method
Subclassing & overriding run() and __init__() methods
Timer objects
Event objects - set() & wait() methods
Lock objects - acquire() & release() methods
RLock (Reentrant) objects - acquire() method
Using locks in the with statement - context manager
Condition objects with producer and consumer
Producer and Consumer with Queue
Semaphore objects & thread pool
Thread specific data - threading.local()
So far, we've been using a thread by instantiating the Thread class given by the package (threading.py). To create our own thread in Python, we'll want to make our class to work as a thread. For this, we should subclass our class from the Thread class.
First thing we need to do is to import Thread using the following code:
from threading import Thread
Then, we should subclass our class from the Thread class like this:
class MyThread(Thread):
Just for reference, here is a code snippet from the package for the Thread class:
class Thread: ... def start(self): """Start the thread's activity. It must be called at most once per thread object. It arranges for the object's run() method to be invoked in a separate thread of control. This method will raise a RuntimeError if called more than once on the same thread object. """ if not self._initialized: raise RuntimeError("thread.__init__() not called") if self._started.is_set(): raise RuntimeError("threads can only be started once") with _active_limbo_lock: _limbo[self] = self try: _start_new_thread(self._bootstrap, ()) except Exception: with _active_limbo_lock: del _limbo[self] raise self._started.wait() def _bootstrap(self): try: self._bootstrap_inner() except: if self._daemonic and _sys is None: return raise def _bootstrap_inner(self): try: ... try: self.run() except SystemExit: pass except: def run(self): try: if self._target: self._target(*self._args, **self._kwargs) finally: # Avoid a refcycle if the thread is running a function with # an argument that has a member that points to the thread. del self._target, self._args, self._kwargs
As a Thread starts up, it does some basic initialization and then calls its run() method, which calls the target function passed to the constructor. The Thread class represents an activity that runs in a separate thread of control. There are two ways to specify the activity:
- by passing a callable object to the constructor
- by overriding the run() method in a subclass
No other methods (except for the constructor) should be overridden in a subclass. In other words, we only override the __init__() and run() methods of a class.
In this section, we will create a subclass of Thread and override run() to do whatever is necessary:
import threading class MyThread(threading.Thread): def run(self): pass if __name__ == '__main__': for i in range(3): t = MyThread() t.start()
Once a thread object is created, its activity must be started by calling the thread's start() method. This invokes the run() method in a separate thread of control.
Once the thread's activity is started, the thread is considered 'alive'. It stops being alive when its run() method terminates - either normally, or by raising an unhandled exception. The is_alive() method tests whether the thread is alive.
import threading import time class MyThread(threading.Thread): def run(self): time.sleep(5) return if __name__ == '__main__': for i in range(3): t = MyThread() t.start() print 't.is_alive()=', t.is_alive() t.join() print 't.is_alive()=', t.is_alive()
Output:
t.is_alive()= True t.is_alive()= False t.is_alive()= True t.is_alive()= False t.is_alive()= True t.is_alive()= False
As we can see from the output, each of the three thread is alive just after the start but t.is_alive()=False after terminated.
Before we move forward, for our convenience, let's put a logging feature into a place:
import threading import time import logging logging.basicConfig(level=logging.DEBUG, format='(%(threadName)-9s) %(message)s',) class MyThread(threading.Thread): def run(self): logging.debug('running') return if __name__ == '__main__': for i in range(3): t = MyThread() t.start()
Output:
(Thread-1 ) running (Thread-2 ) running (Thread-3 ) running
Because the *args and **kwargs values passed to the Thread constructor are saved in private variables, they are not easily accessed from a subclass. To pass arguments to a custom thread type, we need to redefine the constructor to save the values in an instance attribute that can be seen in the subclass:
import threading import time import logging logging.basicConfig(level=logging.DEBUG, format='(%(threadName)-9s) %(message)s',) class MyThread(threading.Thread): def __init__(self, group=None, target=None, name=None, args=(), kwargs=None, verbose=None): super(MyThread,self).__init__(group=group, target=target, name=name, verbose=verbose) self.args = args self.kwargs = kwargs return def run(self): logging.debug('running with %s and %s', self.args, self.kwargs) return if __name__ == '__main__': for i in range(3): t = MyThread(args=(i,), kwargs={'a':1, 'b':2}) t.start()
Output:
(Thread-1 ) running with (0,) and {'a': 1, 'b': 2} (Thread-2 ) running with (1,) and {'a': 1, 'b': 2} (Thread-3 ) running with (2,) and {'a': 1, 'b': 2}
We overrided the __init__() using:
super(MyThread,self).__init__()
For Python 3, we could have used without any args within the super(), like this:
super().__init__()
Python Multithread
Creating a thread and passing arguments to the threadIdentifying threads - naming and logging
Daemon thread & join() method
Active threads & enumerate() method
Subclassing & overriding run() and __init__() methods
Timer objects
Event objects - set() & wait() methods
Lock objects - acquire() & release() methods
RLock (Reentrant) objects - acquire() method
Using locks in the with statement - context manager
Condition objects with producer and consumer
Producer and Consumer with Queue
Semaphore objects & thread pool
Thread specific data - threading.local()
Python tutorial
Python Home
Introduction
Running Python Programs (os, sys, import)
Modules and IDLE (Import, Reload, exec)
Object Types - Numbers, Strings, and None
Strings - Escape Sequence, Raw String, and Slicing
Strings - Methods
Formatting Strings - expressions and method calls
Files and os.path
Traversing directories recursively
Subprocess Module
Regular Expressions with Python
Regular Expressions Cheat Sheet
Object Types - Lists
Object Types - Dictionaries and Tuples
Functions def, *args, **kargs
Functions lambda
Built-in Functions
map, filter, and reduce
Decorators
List Comprehension
Sets (union/intersection) and itertools - Jaccard coefficient and shingling to check plagiarism
Hashing (Hash tables and hashlib)
Dictionary Comprehension with zip
The yield keyword
Generator Functions and Expressions
generator.send() method
Iterators
Classes and Instances (__init__, __call__, etc.)
if__name__ == '__main__'
argparse
Exceptions
@static method vs class method
Private attributes and private methods
bits, bytes, bitstring, and constBitStream
json.dump(s) and json.load(s)
Python Object Serialization - pickle and json
Python Object Serialization - yaml and json
Priority queue and heap queue data structure
Graph data structure
Dijkstra's shortest path algorithm
Prim's spanning tree algorithm
Closure
Functional programming in Python
Remote running a local file using ssh
SQLite 3 - A. Connecting to DB, create/drop table, and insert data into a table
SQLite 3 - B. Selecting, updating and deleting data
MongoDB with PyMongo I - Installing MongoDB ...
Python HTTP Web Services - urllib, httplib2
Web scraping with Selenium for checking domain availability
REST API : Http Requests for Humans with Flask
Blog app with Tornado
Multithreading ...
Python Network Programming I - Basic Server / Client : A Basics
Python Network Programming I - Basic Server / Client : B File Transfer
Python Network Programming II - Chat Server / Client
Python Network Programming III - Echo Server using socketserver network framework
Python Network Programming IV - Asynchronous Request Handling : ThreadingMixIn and ForkingMixIn
Python Coding Questions I
Python Coding Questions II
Python Coding Questions III
Python Coding Questions IV
Python Coding Questions V
Python Coding Questions VI
Python Coding Questions VII
Python Coding Questions VIII
Python Coding Questions IX
Python Coding Questions X
Image processing with Python image library Pillow
Python and C++ with SIP
PyDev with Eclipse
Matplotlib
Redis with Python
NumPy array basics A
NumPy Matrix and Linear Algebra
Pandas with NumPy and Matplotlib
Celluar Automata
Batch gradient descent algorithm
Longest Common Substring Algorithm
Python Unit Test - TDD using unittest.TestCase class
Simple tool - Google page ranking by keywords
Google App Hello World
Google App webapp2 and WSGI
Uploading Google App Hello World
Python 2 vs Python 3
virtualenv and virtualenvwrapper
Uploading a big file to AWS S3 using boto module
Scheduled stopping and starting an AWS instance
Cloudera CDH5 - Scheduled stopping and starting services
Removing Cloud Files - Rackspace API with curl and subprocess
Checking if a process is running/hanging and stop/run a scheduled task on Windows
Apache Spark 1.3 with PySpark (Spark Python API) Shell
Apache Spark 1.2 Streaming
bottle 0.12.7 - Fast and simple WSGI-micro framework for small web-applications ...
Flask app with Apache WSGI on Ubuntu14/CentOS7 ...
Selenium WebDriver
Fabric - streamlining the use of SSH for application deployment
Ansible Quick Preview - Setting up web servers with Nginx, configure enviroments, and deploy an App
Neural Networks with backpropagation for XOR using one hidden layer
NLP - NLTK (Natural Language Toolkit) ...
RabbitMQ(Message broker server) and Celery(Task queue) ...
OpenCV3 and Matplotlib ...
Simple tool - Concatenating slides using FFmpeg ...
iPython - Signal Processing with NumPy
iPython and Jupyter - Install Jupyter, iPython Notebook, drawing with Matplotlib, and publishing it to Github
iPython and Jupyter Notebook with Embedded D3.js
Downloading YouTube videos using youtube-dl embedded with Python
Machine Learning : scikit-learn ...
Django 1.6/1.8 Web Framework ...
Ph.D. / Golden Gate Ave, San Francisco / Seoul National Univ / Carnegie Mellon / UC Berkeley / DevOps / Deep Learning / Visualization