Coding Questions - 2023

We have two sorted lists, and we want to write a function to merge the two lists into one sorted list:

a = [3, 4, 6, 10, 11, 18]
b = [1, 5, 7, 12, 13, 19, 21]

Here is our code:

a = [3, 4, 6, 10, 11, 18]
b = [1, 5, 7, 12, 13, 19, 21]
c = []

while a and b:
    if a[0] < b[0]:
        c.append(a.pop(0))
    else:
        c.append(b.pop(0))
        
# either a or b could still be non-empty
print c + a + b

The output:

[1, 3, 4, 5, 6, 7, 10, 11, 12, 13, 18, 19, 21]

A little bit more compact version using list.extend():

a = [3, 4, 6, 10, 11, 18]
b = [1, 5, 7, 12, 13, 19, 21]

a.extend(b)
c = sorted(a)
print c

Note that the list.extend() is different from list.append():

[1, 3, 5, 7, 9, 11, [2, 4, 6]]  # a.append(b)
[1, 3, 5, 7, 9, 11, 2, 4, 6]    # a.extend(b)

We'll see two ways of initializing dictionary by solving word frequency problem.

#!/usr/bin/python
ss = """Nory was a Catholic because her mother was a Catholic, 
and Nory's mother was a Catholic because her father was a Catholic, 
and her father was a Catholic because his mother was a Catholic, 
or had been."""

words = ss.split()
d = {}.fromkeys(words,0)

# or we can use this to initialize a dict
d = {x:0 for x in words}

for w in words:
    d[w] += 1
print d

ds = sorted(d.items(), key=lambda x:x[1], reverse=True)
print(ds)

# or we can use this for the key
import operator
ds2 = sorted(d.items(), key=operator.itemgetter(1), reverse=True)
print(ds2)

We initialized the dictionary with 0 using fromkeys(), and the output should look like this:

{'a': 6, 'Catholic': 3, 'and': 2, 'because': 3, 'her': 3, 'had': 1, 'Nory': 1, 'father': 2, 
"Nory's": 1, 'his': 1, 'been.': 1, 'mother': 3, 'was': 6, 'or': 1, 'Catholic,': 3}

Here is another way of initializing a dictionary:

d = {}
for w in ss.split():
    d[w] = d.get(w,0) + 1
print d

The third way of dictionary initialization using collections.defaultdict(int) which is convenient for counting:

from collections import defaultdict
d = defaultdict(int)
for w in words:
    d[w] += 1
print d

Sometimes we may want to construct dictionary whose values are lists.

In the following example, we make a dictionary like {'Country': [cities,...], }:

cities = {'San Francisco': 'US', 'London':'UK',
        'Manchester':'UK', 'Paris':'France',
        'Los Angeles':'US', 'Seoul':'Korea'}

# => {'US':['San Francisco', 'Los Angeles'], 'UK':[,], ...}

from collections import defaultdict
# using collections.defaultdict()
d1 = defaultdict(list) # initialize dict with list
for k,v in cities.items():
    d1[v].append(k)
print(f'd1 = {d1}')

# using dict.setdefault(key, default=None)
d2 = {}
for k,v in cities.items():
       d2.setdefault(v,[]).append(k)
print(f'd2 = {d2}')

Output:

d1 = defaultdict(<type 'list'>, {'US': ['San Francisco', 'Los Angeles'], 'UK': ['London', 'Manchester'], 'France': ['Paris'], 'Korea': ['Seoul']})
d2 = {'US': ['San Francisco', 'Los Angeles'], 'UK': ['London', 'Manchester'], 'France': ['Paris'], 'Korea': ['Seoul']}

Or we can use this code:

#! /usr/bin/python

cities = {'San Francisco': 'US', 'London':'UK',
        'Manchester':'UK', 'Paris':'France',
        'Los Angeles':'US', 'Seoul':'Korea'}

# => {'US':['San Francisco', 'Los Angeles'], 'UK':[,], ...}

a = [v for k,v in cities.items()]
print(a)

ds = {x:[] for x in set(a)}
print(ds)

for k,v in cities.items():
	ds[v].append(k)

print(ds)    

Output:

['UK', 'Korea', 'France', 'UK', 'US', 'US']
{'Korea': [], 'UK': [], 'US': [], 'France': []}
{'Korea': ['Seoul'], 'UK': ['Manchester', 'London'], 'US': ['Los Angeles', 'San Francisco'], 'France': ['Paris']}    

A little bit simpler problem. We have a list of numbers:

L = [1,2,4,8,16,32,64,128,256,512,1024,32768,65536,4294967296]

We want to make a dictionary with the number of digits as the key and list of numbers the value:

 {1: [1, 2, 4, 8], 2: [16, 32, 64], 3: [128, 256, 512], 4: [1024], 5: [32768, 65536], 10: [4294967296]})

The code looks like this:

L = [1,2,4,8,16,32,64,128,256,512,1024,32768,65536,4294967296]

from collections import defaultdict
d = defaultdict(list)
for i in L:
    d[len(str(i))].append(i)
print d
print {k:v for k,v in d.items()}

Output:

defaultdict(<type 'list'>, {1: [1, 2, 4, 8], 2: [16, 32, 64], 3: [128, 256, 512], 4: [1024], 5: [32768, 65536], 10: [4294967296]})
{1: [1, 2, 4, 8], 2: [16, 32, 64], 3: [128, 256, 512], 4: [1024], 5: [32768, 65536], 10: [4294967296]}

More examples: Word Frequency

Using map, filter, reduce, write a code that create a list of (n)**2 for range(10) for even integers:

l = [x for x in range(10) if x % 2 == 0]
print l

m = filter(lambda x:x % 2 == 0, [x for x in range(10)] )
# m = list( filter(lambda x:x % 2 == 0, [x for x in range(10)] ) ) # python3
print m

o = map(lambda x: x**2, m)
# o = list( map(lambda x: x**2, m) ) # python3
print o


p = reduce(lambda x,y:x+y, o)
# import functools . # python3
# p = functools.reduce(lambda x,y:x+y, o) # python3
print p

Output:

[0, 2, 4, 6, 8]
[0, 2, 4, 6, 8]
[0, 4, 16, 36, 64]
120

Q: We have the following code with unknown function f(). In f(), we do not want to use return, instead, we may want to use generator.

for x in f(5):
    print x,

The output looks like this:

0 1 8 27 64

Write a function f() so that we can have the output above.

We may use the following f() to get the same output:

def f(n):
   return [x**3 for x in range(n)]

But we want to use generator not using return.

So, the answer should look like this:

def f(n):
    for x in range(n):
        yield x**3

The yield enables a function to comeback where it left off when it is called again. This is the critical difference from a regular function. A regular function cannot comes back where it left off. The yield keyword helps a function to remember its state.

A generator function is a way to create an iterator. A new generator object is created and returned each time we call a generator function. A generator yields the values one at a time, which requires less memory and allows the caller to get started processing the first few values immediately.

Another example of using yield:

Let's build the primes() function so that I fills the n one at a time, and comes back to primes() function until n > 100.

def isPrime(n):
   if n == 1:
      return False
   for t in range(2,n):
      if n % t == 0:
         return False
   return True

for n in primes():
   print n,

The print out from the for-loop should look like this:

2 3 5 7 11 ... 83 89 97

Here is the primes() function:

def primes(n=1):
   while n < 100:
      # yields n instead of returns n
      if isPrime(n): yield n
      # next call it will increment n by 1
      n += 1

Here is a more practical sample of code which I used for Natural Language Processing(NLP).

Suppose we have a huge data file that has hundred millions of lines. So, it may well exceed our computer's memory. In this case, we may want to take so called out-of-core approach: we process data in batch (partially, one by one) rather than process it at once. This saves us from the memory issue when we deal with big data set.

So, we want to use yield command. In the following sample, we do process three lines at a time.

Here is the code:

def stream_docs(path):
    with open(path, 'rb') as lines:
        for line in lines:
            text, label = line[:-3], line[-3:-1]
            yield text, label
            
def get_minibatch(doc_stream, size):
    docs, y = [], []
    try:
        for _ in range(size):
            text, label = next(doc_stream)
            docs.append(text)
            y.append(label)
    except StopIteration:
        return None, None
    return docs, y

doc_stream = stream_docs(path='./test.txt')

for _ in range(100):
    X_train, y_train = get_minibatch(doc_stream, size=3)
    if not X_train:
        break
    print 'X_train, y_train=', X_train, y_train

The yield makes the stream_docs() to return a generator which is always an iterator:

(note) - iterable: strings, lists, tuples, dictionaries, and sets are all iterable objects (containers) which we can get an iterator from. A range() functiuon also returns iterable object. All these objects have an iter() method which is used to get an iterator.

If we comment out the "yield" line, we get "TypeError: NoneType object is not an iterator" at the next(doc_stream) in "get_minibatch()" function.

Output from the code:

X_train, y_train= ['line-a', 'line-b', 'line-c'] [' 1', ' 2', ' 3']
X_train, y_train= ['line-d', 'line-e', 'line-f'] [' 4', ' 5', ' 6']
X_train, y_train= ['line-g', 'line-h', 'line-i'] [' 7', ' 8', ' 9']
X_train, y_train= ['line-j ', 'line-k ', 'line-k '] ['10', '11', '12']

The input used in the code looks like this:

line-a 1
line-b 2
line-c 3
line-d 4
line-e 5
line-f 6
line-g 7
line-h 8
line-i 9
line-j 10
line-k 11
line-k 12

The digit at the end of each line is used as a class label(y_train), so we want to keep it separate from the rest of the text(X_train).

For more information about yield or generator, please visit:

1. The yield keyword
2. Generator Functions and Expressions

It is used to import a module in a directory, which is called package import.

If we have a module, dir1/dir2/mod.py, we put __init__.py in each directories so that we can import the mod like this:

import dir1.dir2.mod

The __init__.py is usually an empty py file. The hierarchy gives us a convenient way of organizing the files in a large system.

We may want to use str.join rather than appending a number every time.

>>> ''.join([`x` for x in xrange(101)])
'0123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100'
>>> 

Note that the (`) is a backtick not a regiular single quote ('):

>>> type(1)
<type 'int'>
>>> type(`1`)
<type 'str'>

Also tote that we cannot use double quote(") or single quote(') to make n a string:

>>> type("1")
<type 'str'>

>>> ''.join(["n" for n in range(10)])
'nnnnnnnnnn'
>>> ''.join(['n' for n in range(10)])
'nnnnnnnnnn'

>>> n = 1
>>> print `n`
1
>>> print "n"
n

Warning: the usage of the backtick(`) in the example above has been deprecated and no longer works in Python3. So, we may want to use str() instead as shown in the example below.

Note: join() returns a string in which the string elements of sequence have been joined by string separator.

We can use str(x) instead:

>>> ''.join([str(x) for x in range(10)])
'0123456789'

Also, since the xrange() is replaced with range in Python 3.x, we should use range() instead for compatibility.

try:
    with open('filename','r') as f:
        print f.read()
except IOError:
    print "No such file exists"

We need to import os module, and add just one line:

import os
print os.path.expanduser('~')

Output:

/home/k

For example, we have the path like this, /home/k/TEST/PYTHON/p.py:

We can get the dir and file using the following:

First, we need to import os module:

>>> import os

Then, we do:

>>> os.path.dirname('/home/k/TEST/PYTHON/p.py')
'/home/k/TEST/PYTHON'

>>> os.path.basename('/home/k/TEST/PYTHON/p.py')
'p.py'

Or we can get them at once in tuple using os.path.split():

>>> os.path.split('/home/k/TEST/PYTHON/p.py')
('/home/k/TEST/PYTHON', 'p.py')

If we want to combine and make a full path:

>>> os.path.join('/home/k/TEST/PYTHON', 'p.py')
'/home/k/TEST/PYTHON/p.py'

We should be able to answer the questions about the standard library.
Such as "Do you know if there's a standard library for recursive file renaming?",
or "In which library would you use for regular expression?"

1. os: operating system support
   
   os.path: Common pathname manipulations:
   

   >>> import os
   >>> print(os.getcwd())
   C:\Python32
   >>> cur_dir = os.curdir
   >>> print(cur_dir)
   .
   >>> scripts_dir = os.path.join(os.curdir, 'Tools\Scripts')
   >>> print(scripts_dir)
   .\Tools\Scripts
   >>> diff_py = os.path.join(scripts_dir, 'diff.py')
   >>> print(diff_py)
   .\Tools\Scripts\diff.py
   >>> os.path.basename(diff_py)
   ' diff.py'
   >>> os.path.splitext(diff_py)
   ('.\\Tools\\Scripts\\diff', '.py')
   

   
   

   The os.path.join() function constructs a pathname out of one or more partial pathnames. In this case, it simply concatenates strings.

   Other convenient ones are: dirname() and basename(), which are the 1st and 2nd element of split(), respectively:

   >>> import os
   >>> print(os.getcwd())
   C:\TEST\dirA\dirB\dirC
   >>> print(os.path.dirname(os.getcwd()))
   C:\TEST\dirA\dirB
   >>> print(os.path.basename(os.getcwd()))
   dirC
   >>> print(os.path.split(os.getcwd()))
   ('C:\\TEST\\dirA\\dirB', 'dirC')
   >>> pathname = os.path.join(os.getcwd(),'myfile.py')
   >>> pathname
   'C:\\TEST\\dirA\\dirB\\dirC\\myfile.py'
   >>> (dirname, filename) = os.path.split(pathname)
   >>> dirname
   'C:\\TEST\\dirA\\dirB\\dirC'
   >>> filename
   'myfile.py'
   

   
   

   The split function splits a full pathname and returns a tuple containing the path and filename. We could use multi-variable assignment to return multiple values from a function. The os.path.split() function does exactly that. We assign the return value of the split function into a tuple of two variables. Each variable receives the value of the corresponding element of the returned tuple.

   The first variable, dirname, receives the value of the first element of the tuple returned from the os.path.split() function, the file path. The second variable, filename, receives the value of the second element of the tuple returned from the os.path.split() function, the filename.

   os.path also contains the os.path.splitext() function, which splits a filename and returns a tuple containing the filename and the file extension.

   The os.path.expanduser() function :

   >>> print(os.path.expanduser('~'))
   C:\Users\KHong
   

   
   

   will expand a pathname that uses ~ to represent the current user's home directory. This works on any platform where users have a home directory, including Linux, Mac OS X, and Windows. The returned path does not have a trailing slash, but the os.path.join() function doesn't mind:

2. re: Regular expression operations
   Visit Regular Expressions with Python


3. itertools: Functions creating iterators for efficient looping.
   It includes permutations, combinations and other useful iterables.

   >>> [x for x in itertools.permutations('123')]
   [('1', '2', '3'), ('1', '3', '2'), ('2', '1', '3'), ('2', '3', '1'), ('3', '1', '2'), ('3', '2', '1')]
   >>> [x for x in itertools.permutations('123',2)]
   [('1', '2'), ('1', '3'), ('2', '1'), ('2', '3'), ('3', '1'), ('3', '2')]
   >>> 
   

>>> sum(range(1,101))
5050
>>> sum(xrange(1,101))
5050
>>> 

range() returns a list to the sum function containing all the numbers from 1 to 100. But xrange() returns an iterator rather than a list, which makes it more lighter in terms of memory use as shown below.

>>> range(1,5)
[1, 2, 3, 4]
>>> xrange(1,5)
xrange(1, 5)
>>> 

Note: Since the xrange() is replaced with range in Python 3.x, we should use range() instead for compatibility. The range() in Python 3.x just returns iterator. That means it does not produce the results in memory any more, and if we want to get list from range(), we need to force it to do so: list(range(...)).

Python defines several iterator objects to support iteration over general and specific sequence types, dictionaries.

1. An iterable is any Python object capable of returning its elements one at a time. Examples of iterables include sequences like lists, tuples, and strings, as well as other iterable objects like sets and dictionaries.
2. Iterable objects can be used in a for loop because they can produce an iterator.
3. We can check if an object is iterable using the iter() function and catching a TypeError if it's not iterable.

my_list = [1, 2, 3]
iter_obj = iter(my_list)

for element in iter_obj:
    print(element)    

1. An iterator is an object representing a stream of data. It must implement two methods: __iter__() and __next__().
2. The __iter__() method returns the iterator object itself, and the __next__() method returns the next element from the stream.
3. When there are no more elements, the __next__() method should raise the StopIteration exception.
4. We can create an iterator from an iterable using the iter() function.

my_list = [1, 2, 3]
iter_obj = iter(my_list)

print(next(iter_obj))  # Output: 1
print(next(iter_obj))  # Output: 2

Any object with a __next__() method to advance to a next result is considered iterator. Note that if an object has __iter__() method, we call the object iterable.

a = [1,2,3]

try:
    iterator = iter(a)
    print(iterator)
    print(iterator.__next__())  # for python 2, use next() instead
    print(iterator.__next__())
    print(iterator.__next__())
    print(iterator.__next__())
except StopIteration:
    print("StopIteration")    

Output:

<list_iterator object at 0x108ac0610>
1
2
3
StopIteration    

For more info, please visit
http://www.bogotobogo.com/python/python_iterators.php.

Generators allow us to declare a function that behaves like an iterator, i.e. it can be used in a for loop. It's a function type generator, but there is another type of generator that may be more familiar to us - expression type generator used in list comprehension:

>>> # List comprehension makes a list
>>> [ x ** 3 for x in range(5)]
[0, 1, 8, 27, 64]
>>> 
>>> # Generator expression makes an iterable
>>> (x ** 3 for x in range(5))
<generator object <genexpr> at 0x000000000315F678>

With the generator expression, we can just wrap it with list() call:

>>> list(x ** 3 for x in range(5))
[0, 1, 8, 27, 64]

Since every generator is an iterator, we can use next() to get the values:

>>> generator = (x ** 3 for x in range(5))
>>> generator.__next__()   # for python2, use next() instead
0
>>> generator.__next__()
1
>>> generator.__next__()
8
>>> generator.__next__()
27

For more information about generators, please visit:
http://www.bogotobogo.com/python/python_generators.php

Functions as first-class objects?
That means we can pass them around as objects and can manipulate them. In other words, most of the times, this just means we can pass these first-class citizens as arguments to functions, or return them from functions. Everything in Python is a proper object. Even things that are "primitive types" in other languages:

>>> dir (100)
['__abs__', '__add__', '__and__', '__class__', '__cmp__', '__coerce__', 
'__delattr__', '__div__', '__divmod__', '__doc__', '__float__', 
....
 'numerator', 'real']
>>> 

Functions have attributes and can be referenced and assigned to variables.

>>> def one(arg):
	'''I am a function returning arg I received.'''
	return arg

>>> one(1)
1
>>> one
<function one at 0x0284AA70>
>>> One = one
>>> One
<function one at 0x0284AA70>
>>> one.__doc__
'I am a function returning arg I received.'
>>> 

A docstring is the documentation string for a function. We use it as shown below:

function_name.__doc__

We can declare it like this:

def my_function():
    """our docstring"""

or:

def my_function():
    '''our docstring'''

Everything between the triple quotes (with double quotes, """ or with single quotes,''') is the function's docstring, which documents what the function does. A docstring, if it exists, must be the first thing defined in a function. In other words, it should appear on the next line after the function declaration. We don't technically need to give our function a docstring, but we always should. The docstring will be available at runtime as an attribute of the function.

Writing documentation for our program this way makes the code more readable. We can also use comments for clarification of what the code is doing. In general, docstrings are for documentation, comments are for a code reader.

The origin of monkey-patch according to wiki is :
"The term monkey patch seems to have come from an earlier term, guerrilla patch, which referred to changing code sneakily at runtime. The word guerrilla, homophonous with gorilla, became monkey, possibly to make the patch sound less intimidating."

In Python, the term monkey patch only refers to dynamic modifications of a class or module at runtime, motivated by the intent to patch existing third-party code as a workaround to a bug or feature which does not act as we desire.

We have a module called m.py like this:

# m.py
class MyClass:
    def f(self):
        print "f()"

Then, if we run the monkey-patch testing like this:

>>> import m
>>> def monkey_f(self):
	print "monkey_f()"

	
>>> m.MyClass.f = monkey_f
>>> obj = m.MyClass()
>>> obj.f()
monkey_f()
>>> 

As we can see, we did make some changes in the behavior of f() in MyClass using the function we defined, monkey_f(), outside of the module m.

It is a risky thing to do, but sometimes we need this trick, such as testing.

The module pdb defines an interactive source code debugger for Python programs. It supports setting (conditional) breakpoints and single stepping at the source line level, inspection of stack frames, source code listing, and evaluation of arbitrary Python code in the context of any stack frame. It also supports post-mortem debugging and can be called under program control.

Python supports the creation of anonymous functions (i.e. functions that are not bound to a name) at runtime, using a construct called lambda. This is not exactly the same as lambda in functional programming languages such as Lisp, but it is a very powerful concept that's well integrated into Python and is often used in conjunction with typical functional concepts like filter(), map() and reduce().

The following code shows the difference between a normal function definition, func and a lambda function, lamb:

>>> 
>>> def func(x): return x ** 3

>>> print(func(5))
125
>>> 
>>> lamb = lambda x: x ** 3
>>> print(lamb(5))
125
>>> 

As we can see, func() and lamb() do exactly the same and can be used in the same ways. Note that the lambda definition does not include a return statement -- it always contains an expression which is returned. Also note that we can put a lambda definition anywhere a function is expected, and we don't have to assign it to a variable at all.

The lambda's general form is :

lambda arg1, arg2, ...argN : expression using arguments

Function objects returned by running lambda expressions work exactly the same as those created and assigned by defs. However, there are a few differences that make lambda useful in specialized roles:

1. lambda is an expression, not a statement.
   Because of this, a lambda can appear in places a def is not allowed. For example, places like inside a list literal, or a function call's arguments. As an expression, lambda returns a value that can optionally be assigned a name. In contrast, the def statement always assigns the new function to the name in the header, instead of returning is as a result.
2. lambda's body is a single expression, not a block of statements.
   The lambda's body is similar to what we'd put in a def body's return statement. We simply type the result as an expression instead of explicitly returning it. Because it is limited to an expression, a lambda is less general that a def. We can only squeeze design, to limit program nesting. lambda is designed for coding simple functions, and def handles larger tasks.

>>> 
>>> def f(x, y, z): return x + y + z

>>> f(2, 30, 400)
432

We can achieve the same effect with lambda expression by explicitly assigning its result to a name through which we can call the function later:

>>> 
>>> f = lambda x, y, z: x + y + z
>>> f(2, 30, 400)
432
>>> 

Here, the function object the lambda expression creates is assigned to f. This is how def works, too. But in def, its assignment is an automatic must.

For more, please go to Functions lambda

We have more detailed discussion in Classes and Instances - Method: Properties.

In general, properties are more flexible than attributes. That's because we can define functions that describe what is supposed to happen when we need setting, getting or deleting them. If we don't need this additional flexibility, we may just use attributes since they are easier to declare and faster.

However, when we convert an attribute into a property, we just define some getter and setter that we attach to it, that will hook the data access. Then, we don't need to rewrite the rest of our code, the way for accessing the data is the same, whatever our attribute is a property or not.

From the official Python documentation on @classmethod:

classmethod(function)
    Return a class method for function.

    A class method receives the class as implicit first argument, 
    just like an instance method receives the instance. 
    To declare a class method, use this idiom:

class C:
    @classmethod
    def f(cls, arg1, arg2, ...): ...

The @classmethod form is a function decorator.
It can be called either on the class (such as C.f()) or on an instance (such as C().f()). 
The instance is ignored except for its class. If a class method is called for a derived class, 
the derived class object is passed as the implied first argument.

And for the @staticmethod, the python doc describes it as below:

staticmethod(function)
    Return a static method for function.

    A static method does not receive an implicit first argument. 
    To declare a static method, use this idiom:

class C:
    @staticmethod
    def f(arg1, arg2, ...): ...

The @staticmethod form is a function decorator.
It can be called either on the class (such as C.f()) or on an instance (such as C().f()). 
The instance is ignored except for its class.
Static methods in Python are similar to those found in Java or C++.

For more info on static vs class methods, please visit:

@static method vs class method

Iterating the list is not a desirable solution. The right answer should look like this:

>>> dup_list = [1,2,3,4,4,4,5,1,2,7,8,8,10]
>>> unique_list = list(set(dup_list))
>>> print unique_list
[1, 2, 3, 4, 5, 7, 8, 10]
>>> 

Or without using the set:

dup_list = [1,2,3,4,4,4,5,1,2,7,8,8,10]

#d = {}.fromkeys(dup_list,0) 
d = {k:0 for k in dup_list}
print(d)

for x in dup_list:
    d[x] += 1
print(d)

uniq = [ k for k,v in d.items() if v == 1]
print(uniq)    

Output:

{1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 7: 0, 8: 0, 10: 0}
{1: 2, 2: 2, 3: 1, 4: 3, 5: 1, 7: 1, 8: 2, 10: 1}
[3, 5, 7, 10]    

Python provides the following:

1. map(aFunction, aSequence)
2. filter(aFunction, aSequence)
3. reduce(aFunction, aSequence)
4. lambda
5. list comprehension

These functions are all convenience features in that they can be written in Python fairly easily. Functional programming is all about expressions. We may say that the Functional programming is an expression oriented programming.

The syntax of map is:

map(aFunction, aSequence)

The first argument is a function to be executed for all the elements of the iterable given as the second argument. If the function given takes in more than 1 arguments, then many iterables are given.

>>> def cubic(x):
	return x*x*x

>>> items = [x for x in range(11) if x % 2 == 0]
>>> list(map(cubic, items))
[0, 8, 64, 216, 512, 1000]
>>>
>>> list(map(lambda x,y: x*y, [1,2,3], [4,5,6]))
[4, 10, 18]
>>> 

map is similar to list comprehension but is more limited because it requires a function instead of an arbitrary expression.

Just for comparison purpose, in the following example, we will include map as well.

>>> integers = [ x for x in range(11)]
>>> filter(lambda x: x % 2 == 0, integers)
[0, 2, 4, 6, 8, 10]
>>> map(lambda x: x**2, integers)
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
>>> reduce(lambda x, y: x + y, integers)
55
>>> 

In the above example, we defined a simple list of integer values, then we use the standard functions filter(), map() and reduce() to do various things with that list. All of the three functions expect two arguments: A function and a list.
In the first example, filter() calls our lambda function for each element of the list, and returns a new list that contains only those elements for which the function returned "True". In this case, we get a list of all even numbers.
In the second example, map() is used to convert our list. The given function is called for every element in the original list, and a new list is created which contains the return values from our lambda function. In this case, it computes x^2 for every element.
Finally, reduce() is somewhat special. The function for this one must accept two arguments (x and y), not just one. The function is called with the first two elements from the list, then with the result of that call and the third element, and so on, until all of the list elements have been handled. This means that our function is called n-1 times if the list contains n elements. The return value of the last call is the result of the reduce() construct. In the above example, it simply adds the arguments, so we get the sum of all elements.

Putting *args and/or **kwargs as the last items in our function definition's argument list allows that function to accept an arbitrary number of anonymous and/or keyword arguments.
Those arguments are called Keyword Arguments. Actually, they are place holders for multiple arguments, and they are useful especially when we need to pass a different number of arguments each time we call the function.

We may want to use *args when we're not sure how many arguments might be passed to our function, i.e. it allows us to pass an arbitrary number of arguments to your function as shown in the example below:

Let's make a function that sums of all numbers. It should work for bo the inputs: 1,2,3,4,5 as separate args and as a list [1,2,3,4,5]:

def fn(*args):
  ans = 0 
  for x in args:
    ans += x
return ans

print(fn(1,2,3,4,5))
print(fn(*[1, 2, 3, 4, 5]))
print(fn(*range(1,6)))

Note here if we give the list itself instead unpacking('*'), we get an error:

print(fn([1, 2, 3, 4, 5]))    

The error we get is:

TypeError: unsupported operand type(s) for +=: 'int' and 'list'    

That's because we're trying to do 0 + [1,2,3,4,5] which is not allowed.

So, we need to check how the arg is passed to the function with the following code:

def fn(*args):
  print('args={0}'.format(args))

fn(1,2,3,4,5)       # args=(1, 2, 3, 4, 5)
fn(*[1,2,3,4,5])    # args=(1, 2, 3, 4, 5)
fn([1,2,3,4,5])     # args=([1, 2, 3, 4, 5],)

As we can see, fn(*[1,2,3,4,5]) is passing args uppacked, while fn([1,2,3,4,5]) is passing a list as one parameter.

The keyword arguments is a special name=value syntax in function calls that specifies passing by name. It is often used to provide configuration options.

>>> def kargs_function(**kargs):
	  for k,v in kargs.items():
		print (k,v)

>>> kargs_function(**{'uno':'one','dos':'two','tres':'three'})
('dos', 'two')
('tres', 'three')
('uno', 'one')
>>>
>>> kargs_function(dos='two', tres='three', uno='one')
('dos', 'two')
('tres', 'three')
('uno', 'one')
>>> 

For more details, please visit *args and **kwargs - Collecting and Unpacking Arguments.

The content of objects of immutable types cannot be changed after they are created.

To remove a list element, we can use either the del statement if we know exactly which element(s) we are deleting or the remove() method if we do not know.

list.remove(element), del list(index), list.pop(index)

remove() removes the first matching value, not a specific index:

>>> a = [5,6,7,7,8]
>>> a.remove(7)
>>> a
[5, 6, 7, 8]

Both del and pop() work on index while the pop() returns an element:

>>> a = [5,6,7,7,8]
>>> del a[1]
>>> a
[5, 7, 7, 8]

>>> a = [5,6,7,7,8]
>>> a.pop(1)
6
>>> a
[5, 7, 7, 8]

>>> a = [5,6,7,7,8]
>>> a.pop(a.index(6)) # get the index for 6
6
>>> a
[5, 7, 7, 8]

Given a list of string, ['Black', 'holes', 'are', 'where', 'God', 'divided', 'by', 'zero'], print each word in a new line:

>>> s = ['Black', 'holes', 'are', 'where', 'God', 'divided', 'by', 'zero']
>>> print '\n'.join(s)
Black
holes
are
where
God
divided
by
zero

1. Python Coding Questions I
2. Python Coding Questions II
3. Python Coding Questions III
4. Python Coding Questions IV
5. Python Coding Questions V
6. Python Coding Questions VI
7. Python Coding Questions VII
8. Python Coding Questions VIII
9. Python Coding Questions IX
10. Python Coding Questions X

List of codes Q & A

1. Merging two sorted list
2. Get word frequency - initializing dictionary
3. Initializing dictionary with list
4. map, filter, and reduce
5. Write a function f() - yield
6. What is __init__.py?
7. Build a string with the numbers from 0 to 100, "0123456789101112..."
8. Basic file processing: Printing contents of a file - "with open"
9. How can we get home directory using '~' in Python?
10. The usage of os.path.dirname() & os.path.basename() - os.path
11. Default Libraries
12. range vs xrange
13. Iterators
14. Generators
15. Manipulating functions as first-class objects
16. docstrings vs comments
17. using lambdda
18. classmethod vs staticmethod
19. Making a list with unique element from a list with duplicate elements
20. What is map?
21. What is filter and reduce?
22. *args and **kwargs
23. mutable vs immutable
24. Difference between remove, del and pop on lists
25. Join with new line
26. Hamming distance
27. Floor operation on integers
28. Fetching every other item in the list
29. Python type() - function
30. Dictionary Comprehension
31. Sum
32. Truncating division
33. Python 2 vs Python 3
34. len(set)
35. Print a list of file in a directory
36. Count occurrence of a character in a Python string
37. Make a prime number list from (1,100)
38. Reversing a string - Recursive
39. Reversing a string - Iterative
40. Reverse a number
41. Output?
42. Merging overlapped range
43. Conditional expressions (ternary operator)
44. Packing Unpacking
45. Function args
46. Unpacking args
47. Finding the 1st revision with a bug
48. Which one has higher precedence in Python? - NOT, AND , OR
49. Decorator(@) - with dollar sign($)
50. Multi-line coding
51. Recursive binary search
52. Iterative binary search
53. Pass by reference
54. Simple calculator
55. iterator class that returns network interfaces
56. Converting domain to ip
57. How to count the number of instances
58. Python profilers - cProfile
59. Calling a base class method from a child class that overrides it
60. How do we find the current module name?
61. Why did changing list 'newL' also change list 'L'?
62. Constructing dictionary - {key:[]}
63. Colon separated sequence
64. Converting binary to integer
65. 9+99+999+9999+...
66. Calculating balance
67. Regular expression - findall
68. Chickens and pigs
69. Highest possible product
70. Implement a queue with a limited size
71. Copy an object
72. Filter
73. Products
74. Pickle
75. Overlapped Rectangles
76. __dict__
77. Fibonacci I - iterative, recursive, and via generator
78. Fibonacci II - which method?
79. Fibonacci III - find last two digits of Nth Fibonacci number
80. Write a Stack class returning Max item at const time A
81. Write a Stack class returning Max item at const time B
82. Finding duplicate integers from a list - 1
83. Finding duplicate integers from a list - 2
84. Finding duplicate integers from a list - 3
85. Reversing words 1
86. Parenthesis, a lot of them
87. Palindrome / Permutations
88. Constructing new string after removing white spaces
89. Removing duplicate list items
90. Dictionary exercise
91. printing numbers in Z-shape
92. Factorial
93. lambda
94. lambda with map/filter/reduce
95. Number of integer pairs whose difference is K
96. iterator vs generator
97. Recursive printing files in a given directory
98. Bubble sort
99. What is GIL (Global Interpreter Lock)?
100. Word count using collections
101. Pig Latin
102. List of anagrams from a list of words
103. lamda with map, filer and reduce functions
104. Write a code sending an email using gmail
105. histogram 1 : the frequency of characters
106. histogram 2 : the frequency of ip-address
107. Creating a dictionary using tuples
108. Getting the index from a list
109. Looping through two lists side by side
110. Dictionary sort with two keys : primary / secondary keys
111. Writing a file downloaded from the web
112. Sorting csv data
113. Reading json file
114. Sorting class objects
115. Parsing Brackets
116. Printing full path
117. str() vs repr()
118. Missing integer from a sequence
119. Polymorphism
120. Product of every integer except the integer at that index
121. What are accessors, mutators, and @property?
122. N-th to last element in a linked list
123. Implementing linked list
124. Removing duplicate element from a list
125. List comprehension
126. .py vs .pyc
127. Binary Tree
128. Print 'c' N-times without a loop
129. Quicksort
130. Dictionary of list
131. Creating r x c matrix
132. Transpose of a matrix
133. str.isalpha() & str.isdigit()
134. Regular expression
135. What is Hashable? Immutable?
136. Convert a list to a string
137. Convert a list to a dictionary
138. List - append vs extend vs concatenate
139. Use sorted(list) to keep the original list
140. list.count()
141. zip(list,list) - join elements of two lists
142. zip(list,list) - weighted average with two lists
143. Intersection of two lists
144. Dictionary sort by value
145. Counting the number of characters of a file as One-Liner
146. Find Armstrong numbers from 100-999
147. Find GCF (Greatest common divisor)
148. Find LCM (Least common multiple)
149. Draws 5 cards from a shuffled deck
150. Dictionary order by value or by key
151. Regular expression - re.split()
152. Regular expression : re.match() vs. re.search()
153. Regular expression : re.match() - password check
154. Regular expression : re.search() - group capturing
155. Regular expression : re.findall() - group capturin
156. Prime factors : n = products of prime numbers
157. Valid IPv4 address
158. Sum of strings
159. List rotation - left/right
160. shallow/deep copy
161. Converting integer to binary number
162. Creating a directory and a file
163. Creating a file if not exists
164. Invoking a python file from another
165. Sorting IP addresses
166. Word Frequency
167. Printing spiral pattern from a 2D array - I. Clock-wise
168. Printing spiral pattern from a 2D array - II. Counter-Clock-wise
169. Find a minimum integer not in the input list
170. I. Find longest sequence of zeros in binary representation of an integer
171. II. Find longest sequence of zeros in binary representation of an integer - should be surrounded with 1
172. Find a missing element from a list of integers
173. Find an unpaired element from a list of integers
174. Prefix sum : Passing cars
175. Prefix sum : count the number of integers divisible by k in range [A,B]
176. Can make a triangle?
177. Dominant element of a list
178. Minimum perimeter
179. MinAbsSumOfTwo
180. Ceiling - Jump Frog
181. Brackets - Nested parentheses
182. Brackets - Nested parentheses of multiple types
183. Left rotation - list shift
184. MaxProfit
185. Stack - Fish
186. Stack - Stonewall
187. Factors or Divisors
188. String replace in files 1
189. String replace in files 2
190. Using list as the default_factory for defaultdict
191. Leap year
192. Capitalize
193. Log Parsing
194. Getting status_code for a site
195. 2D-Array - Max hourglass sum
196. New Year Chaos - list
197. List (array) manipulation - list
198. Hash Tables: Ransom Note
199. Count Triplets with geometric progression
200. Strings: Check if two strings are anagrams
201. Strings: Making Anagrams
202. Strings: Alternating Characters
203. Special (substring) Palindrome
204. String with the same frequency of characters
205. Common Child
206. Fraudulent Activity Notifications
207. Maximum number of toys
208. Min Max Riddle
209. Poisonous Plants with Pesticides
210. Common elements of 2 lists - Complexity
211. Get execution time using decorator(@)
212. Conver a string to lower case and split using decorator(@)
213. Python assignment and memory location
214. shallow copy vs deep copy for compound objects (such as a list)
215. Generator with Fibonacci
216. Iterator with list
217. Second smallest element of a list
218. *args, **kargs, and positional args
219. Write a function, fn('x','y',3) that returns ['x1', 'y1', 'x2', 'y2', 'x3', 'y3']
220. sublist or not
221. any(), all()
222. Flattening a list
223. Select an element from a list
224. Circularly identical lists
225. Difference between two lists
226. Reverse a list
227. Split a list with a step
228. Break a list and make chunks of size n
229. Remove duplicate consecutive elements from a list
230. Combination of elements from two lists
231. Adding a sublist
232. Replace the first occurence of a value
233. Sort the values of the first list using the second list
234. Transpose of a matrix (nested list)
235. Binary Gap
236. Powerset
237. Round Robin
238. Fixed-length chunks or blocks
239. Accumulate
240. Dropwhile
241. Groupby
242. Simple product
243. Simple permutation
244. starmap(fn, iterable)
245. zip_longest(*iterables, fillvalue=None)
246. What is the correct way to write a doctest?
247. enumerate(iterable, start=0)
248. collections.defaultdict - grouping a sequence of key-value pairs into a dictionary of lists
249. What is the purpose of the 'self' keyword when defining or calling instance methods?
250. collections.namedtuple(typename, field_names, *, rename=False, defaults=None, module=None)
251. zipped
252. What is key difference between a set and a list?
253. What does a class's init() method do?
254. Class methods
255. Permutations and combinations of ['A','B','C']
256. Sort list of dictionaries by values
257. Return a list of unique words
258. hashlib
259. encode('utf-8')
260. Reading in CSV file
261. Count capital letters in a file
262. is vs ==
263. Create a matrix : [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
264. Binary to integer and check if it's the power of 2
265. urllib.request.urlopen() and requests
266. Game statistics
267. Chess - pawn race
268. Decoding a string
269. Determinant of a matrix - using numpy.linalg.det()
270. Revenue from shoe sales - using collections.Counter()
271. Rangoli
272. Unique characters
273. Valid UID
274. Permutations of a string in lexicographic sorted order
275. Nested list
276. Consecutive digit count
277. Find a number that occurs only once
278. Sorting a two-dimensional array
279. Reverse a string
280. Generate random odd numbers in a range
281. Shallow vs Deep copy
282. Transpose matrix
283. Are Arguments in Python Passed by Value or by Reference?
284. re: Is a string alphanumeric?
285. reversed()
286. Caesar's cipher, or shift cipher, Caesar's code, or Caesar shift
287. Every other words
288. re: How can we check if an email address is valid or not?
289. re: How to capture temperatures of a text
290. re.split(): How to split a text.
291. How can we merge two dictionaries?
292. How can we combine two dictionaries?
293. What is the difference between a generator and a list?
294. Pairs of a given array A whose sum value is equal to a target value N
295. Adding two integers without plus
296. isinstance() vs type()
297. What is a decorator?
298. In Python slicing, what does my_list[-3:2:-2] slice do?
299. Revisit sorting dict - counting chars in a text file
300. re: Transforming a date format using re.sub
301. How to replace the newlines in csv file with tabs?
302. pandas.merge
303. How to remove duplicate charaters from a string?
304. Implement a class called ComplexNumber
305. Find a word frequency
306. Get the top 3 most frequent characters of a string
307. Just seen and ever seen
308. Capitalizing the full name
309. Counting Consequitive Characters
310. Calculate Product of a List of Integers Provided using input()
311. How many times a substring appears in a string
312. Hello, first_name last_name
313. String validators
314. Finding indices that a char occurs in a list
315. itertools combinations