Skip to content

Instantly share code, notes, and snippets.

@IdanBanani
Last active January 3, 2026 10:40
Show Gist options
  • Select an option

  • Save IdanBanani/9a5b2807accfaa151567e8b21a4752b6 to your computer and use it in GitHub Desktop.

Select an option

Save IdanBanani/9a5b2807accfaa151567e8b21a4752b6 to your computer and use it in GitHub Desktop.
python3 cheatsheet

Python 3.6-3.12+ Comprehensive Reference Guide

Table of Contents

  1. Getting Started
  2. Object Types & Type System
  3. Numeric Types
  4. Strings
  5. Sequences (Lists & Tuples)
  6. Dictionaries & Sets
  7. File I/O & Serialization
  8. Control Flow & Loops
  9. Functions & Scope
  10. Generators, Comprehensions & Iterators
  11. Modules & Packages
  12. Object-Oriented Programming
  13. Decorators
  14. Context Managers (with/as)
  15. Unicode & Byte Strings
  16. Standard Library Tools
  17. Advanced Iteration (itertools)
  18. Collections & Data Structures
  19. Interactive Python (IPython & Jupyter)
  20. Scientific Computing
  21. Common Mistakes & Best Practices

Getting Started

Script Basics

  • Shebang (hash-bang): Scripts typically begin with #!/usr/bin/env python
    • In UNIX-like systems: Use chmod +x script.py to make executable
    • Direct path: #!/usr/local/bin/python
    • Dynamic lookup (recommended): #!/usr/bin/env python — system finds Python path automatically

Special Variables

  • _ (underscore): Contains the last evaluated value in REPL

    • Must explicitly call the expression first: a = 3; a; _ # gives 3
  • None: Python's null value (equivalent to NULL in C++)

Code Organization

  • Python programs decompose into: modulesstatementsexpressionsobjects
  • Use dir(variable) to list all available methods for any object

Object Types & Type System

Built-in Object Type Overview

Object Type Examples Immutable? Notes
Numbers 1234, 3.1415, 3+4j, 0b111, Decimal(), Fraction() Unlimited precision integers
Strings 'spam', "bos's", b'a\x01c', u'sp\xc4m' Sequences of text
Lists [1, [2, 'three'], 4.5], list(range(10)) Ordered, mutable sequences
Tuples (1, 'spam', 4, 'U'), tuple('spam'), namedtuple Fixed, immutable sequences
Dictionaries {'food': 'spam', 'taste': 'yum'}, dict(hours=10) Key-value mappings
Sets set('abc'), {'a', 'b', 'c'} Unordered unique collections
Files open('eggs.txt'), open(r'C:\ham.bin', 'wb') I/O streams
Booleans True, False Truth values
Program Units functions, modules, classes Code organization

Key Concepts:

  • Immutability: Numbers, Strings, Tuples cannot be modified after creation
  • Mutability: Lists, Dictionaries, Sets are mutable (can be changed in-place)

Type Checking (Use Sparingly!)

# Bad practice - avoids polymorphism
type(L) == list

# Better - use isinstance for type checking
isinstance(L, list)
isinstance(x, (list, tuple))  # Check multiple types

# Best - use duck typing (Pythonic way)
# Don't ask "what is it?", ask "what can it do?"

Numeric Types

Numeric Literals

Literal Interpretation
1234, 24 Integers (unlimited size)
1.23, 1.3e-10, 4E210 Floating-point numbers
0o117, 0x9ff, 0b10101 Octal, hexadecimal, binary
3+4j, 3.0+4.0j, 3J Complex numbers
set('spam'), {1, 2, 3, 4} Sets
Decimal('1.0'), Fraction(1, 3) Decimal and fraction extensions
bool(x), True, False Boolean type and constants

Numeric Display & Representation

# Precision vs. user-friendly display
3.1415 * 2          # Full precision: 6.28300000004
print(3.1415 * 2)   # User-friendly: 6.283

# Formatted output
'%e' % num          # Scientific: '3.333e-01'
'%4.2f' % num       # Fixed-point: '0.33'
'{0:4.2f}'.format(num)  # Format method: '0.33'

Numeric Operations

# Built-in numeric functions
pow(x, y)           # Power: x^y
abs(x)              # Absolute value
round(x, ndigits)   # Round to n decimals
int(x)              # Convert to int (truncates float)
float(x)            # Convert to float
hex(x)              # Convert to hex string: '0xf'
oct(x)              # Convert to octal string: '0o17'
bin(x)              # Convert to binary string: '0b1111'

# Conversion with base
int('0xf', 16)      # Parse hex: 15
int('0o17', 8)      # Parse octal: 15

Math Module (Standard Library)

import math

# Constants
math.pi              # 3.141592...
math.e              # 2.71828...

# Functions
math.sqrt(x)        # Square root
math.sin(x), math.cos(x), math.tan(x)  # Trigonometry
math.floor(x)       # Round towards negative infinity
math.ceil(x)        # Round towards positive infinity
math.trunc(x)       # Discard fractional part
math.min(), math.max()  # Min and max functions

Random Module (Standard Library)

import random

random.random()             # Random float [0.0, 1.0)
random.randint(a, b)        # Random integer [a, b]
random.choice(sequence)     # Random element from sequence
random.shuffle(list)        # Shuffle list in-place

Division Operations

# Floor division (truncating division)
10 / 4              # True division: 2.5 (keeps remainder)
10 // 4             # Floor division: 2 (truncates)
10 // 4.0           # 2.0 (also works on floats)

# Rounding differences
import math
math.floor(2.5)     # 2 (towards negative infinity)
math.floor(-2.5)    # -3 (towards negative infinity!)
math.trunc(2.5)     # 2 (towards zero)
math.trunc(-2.5)    # -2 (towards zero)

Chain Comparisons

# Python allows chained comparisons
X < Y < Z           # True (equivalent to: X < Y AND Y < Z)
X < Y > Z           # Works as expected
1 < 2 < 3.0 < 4     # True
1 == 2 < 3          # Same as: 1 == 2 AND 2 < 3

# Warning: Floating-point comparisons
1.1 + 2.2 == 3.3    # May not be True due to rounding
# 1.1 + 2.2 = 3.3000000000000003
int(1.1 + 2.2) == 3  # This works

Numeric Extensions

  • Complex: 3+4j, operations: +, -, *, /, **
  • Decimal (from decimal import Decimal): Fixed-precision arithmetic
  • Fraction (from fractions import Fraction): Exact rational arithmetic
  • 3rd-party: Matrix and vector types available

Strings

String Basics

S = 'Spam'

# Length and access
len(S)              # 4
S[0]                # 'S' (first)
S[-1]               # 'm' (last)
S[-2]               # 'a' (second from end)

# Slicing
S[1:3]              # 'pa' (excludes end index)
S[0:3]              # 'Spa'
S[1:]               # 'pam' (to end)
S[:-1]              # 'Spa' (everything but last)
S[::2]              # 'Sm' (every other character)
S[::-1]             # 'mapS' (reversed)

# Operations
S + 'xyz'           # Concatenation: 'Spamxyz'
S * 3               # Repetition: 'SpamSpamSpam'
'ab' in 'cabsf'     # Membership: True

String Slicing & Indexing Visualization

Index:   0   1   2   3   4   5   6   7
        | S | L | I | C | E | S | B | C |
Index:  -8  -7  -6  -5  -4  -3  -2  -1

String Methods

Method Example Output Notes
find S.find('pa') 1 Return index or -1 if not found
replace S.replace('pa', 'XYZ') 'SXYZm' Strings are immutable
split 'a,b,c'.split(',') ['a', 'b', 'c'] Split by delimiter
upper S.upper() 'SPAM' Convert to uppercase
lower S.lower() 'spam' Convert to lowercase
strip ' spam '.strip() 'spam' Remove leading/trailing whitespace
rstrip 'spam\n'.rstrip() 'spam' Remove trailing chars (default: whitespace)
lstrip ' spam'.lstrip() 'spam' Remove leading chars
isalpha S.isalpha() True/False All alphabetic?
isdigit '123'.isdigit() True All digits?
startswith S.startswith('S') True/False Starts with substring?
endswith S.endswith('m') True/False Ends with substring?
join ','.join(['a', 'b']) 'a,b' Join iterable with delimiter
count 'aaa'.count('a') 3 Count occurrences
encode 'spam'.encode('utf8') b'spam' Encode to bytes

String Immutability

S = 'spam'
S[0] = 'z'          # Error! Strings are immutable

# Correct approach
S = 'z' + S[1:]     # Create new string: 'zpam'

String Formatting

Old-style % formatting

'%s, %s' % ('eggs', 'SPAM')              # 'eggs, SPAM'
'There are %d %s' % (2, 'birds')         # 'There are 2 birds'

# Formatting with dictionary
'%(qty)d more %(food)s' % {'qty': 1, 'food': 'spam'}  # '1 more spam'

Method-style .format() — Python 3.0+

# By position
'{0},{1},{2}'.format('spam', 'ham', 'eggs')      # 'spam,ham,eggs'

# By keyword
'{motto},{pork},{food}'.format(motto='spam', pork='ham', food='eggs')

# Mixed
'{motto},{0},{food}'.format('ham', motto='spam', food='egg')  # 'spam,ham,egg'

# Relative position (no indices)
'{},{},{}'.format('spam', 'ham', 'eggs')

# Accessing attributes
'My {0.platform}'.format(sys)                    # 'My win32'
'My {map[kind]}'.format(map={'kind': 'laptop'}) # 'My laptop'

F-strings — Python 3.6+

name, age = 'Bob', 40
f'Hello {name}, age {age}'          # f'Hello Bob, age 40'
f'{name.upper()}'                   # f'BOB'
f'{2 ** 8}'                         # f'256'
f'{value:.2f}'                      # f-string formatting

Special String Types

Raw strings — Backslashes not interpreted

myfile = open(r'C:\new\text.dat', 'w')  # \n is not interpreted as newline

Triple-quoted strings — Multi-line text

mantra = """Always look
on the bright
side of life."""

# Common uses:
# 1. Docstrings for functions/classes
# 2. Temporarily disable code (act as block comments)
# 3. Embed quotes easily

X = 1
"""
import os
print(os.getcwd())
"""
Y = 2  # Code in triple quotes is ignored

Byte strings — Python 3.0+

b'a\x01c'           # Byte literals (ASCII only)
'spam'.encode('utf8')  # Encode string to bytes: b'spam'
b'spam'.decode('utf8')  # Decode bytes to string: 'spam'

Regular Expressions (re module)

import re

# Basic matching
match = re.match(r'Hello[ \t]*(.*)world', 'Hello     Python world')
match.group(1)          # 'Python '

# Finding patterns with groups
match = re.match(r'[/:](.*)[/:](.*)[/:](.*)', '/usr/home:lumberjack')
match.groups()          # ('usr', 'home', 'lumberjack')

# Splitting
re.split(r'[/:]', '/usr/home/lumberjack')  # ['', 'usr', 'home', 'lumberjack']

# Find all matches
re.findall(r'\d+', '12 apples, 34 oranges')  # ['12', '34']

# Named groups
m = re.match(r'(?P<first>\w+) (?P<last>\w+)', 'John Doe')
m.group('first')        # 'John'
m.groupdict()           # {'first': 'John', 'last': 'Doe'}

Sequences (Lists & Tuples)

Lists

Creation & Basics

L = [1, 2, 3]
L = [1, [2, 'three'], 4.5]          # Nested lists
L = list(range(10))                 # [0, 1, 2, ..., 9]
L = [None] * 5                      # [None, None, None, None, None]

# Don't use shallow copy for initialization
L = [[None]*3]*2                    # Wrong! Each row shares same list
L = [[None]*3 for _ in range(2)]    # Correct!

Indexing & Slicing

L = [1, 2, 3, 4, 5]
L[0]                    # 1
L[-1]                   # 5 (last)
L[1:3]                  # [2, 3] (excludes end)
L[::2]                  # [1, 3, 5] (every other)
L[::-1]                 # [5, 4, 3, 2, 1] (reversed)
L[1:4:2]                # [2, 4] (from 1 to 4, step 2)

L[:-1]                  # [1, 2, 3, 4] (all but last)
L[1:]                   # [2, 3, 4, 5] (from 1 onward)
L[:3]                   # [1, 2, 3] (up to 3)

Modification Methods

Method Effect Example
L.append(x) Add single element to end L.append(4)
L.extend(iterable) Add all elements from iterable L.extend([4, 5, 6])
L.insert(i, x) Insert at position L.insert(0, 'first')
L.remove(x) Remove first occurrence by value L.remove(2)
L.pop([i]) Remove and return element (default: last) L.pop() / L.pop(0)
L.clear() Remove all elements L.clear()
L.sort() Sort in-place L.sort()
L.reverse() Reverse in-place L.reverse()
L.copy() Shallow copy L2 = L.copy()
L.count(x) Count occurrences L.count(2)
L.index(x) Return index of first occurrence L.index(2)

⚠️ CRITICAL: Never re-assign mutable to itself with in-place methods:

L = L.append(1)  # WRONG! append() returns None
L.append(1)      # CORRECT!

Append vs. Extend

L = [1, 2, 3]
L.append([4, 5])        # [1, 2, 3, [4, 5]] — adds list as single element
L.extend([4, 5])        # [1, 2, 3, 4, 5] — unpacks iterable

# extend is equivalent to
L[len(L):] = [4, 5]     # Slice assignment for extension

Slice Assignment — Insert, Replace, Delete

L = [1, 2, 3]
L[1:2] = [4, 5]         # [1, 4, 5, 3] — replacement
L[1:1] = [6, 7]         # [1, 6, 7, 4, 5, 3] — insertion (no elements in range)
L[1:3] = []             # Delete slice
del L[1:]               # Delete from position onward

List Comprehensions

[x**2 for x in range(5)]                            # [0, 1, 4, 9, 16]
[x for x in range(10) if x % 2 == 0]                # [0, 2, 4, 6, 8]
[[x, x**2] for x in range(3)]                       # [[0, 0], [1, 1], [2, 4]]
[x*2 for x in 'SPAM']                               # ['SS', 'PP', 'AA', 'MM']

# Nested comprehensions (inner loop on right)
[x+y for x in 'abc' for y in 'lmn']                 # ['al', 'am', 'an', 'bl', ...]

# With conditions
[x for x in range(10) if x % 2 == 0 if x > 5]       # [6, 8]

# With dict iteration
[k for (k, v) in mydict.items() if v == V]

Performance: List comprehensions are ~2x faster than for loops!

List of Zeros

[0] * 10                # [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

Iteration

for x in [1, 2, 3]:
    print(x, end=' ')   # 1 2 3  #end= controls output separator

Return a Copy

arr[:]                  # Slice copy of list
arr.copy()              # Also creates shallow copy

Tuples

Basics

T = (1, 2, 3, 4)        # Tuple
T = 1, 2, 3, 4          # Parentheses optional
T = (40,)               # Single-element tuple (comma REQUIRED!)
T = (40)                # NOT a tuple — just integer 40!

len(T)                  # 4
T[0]                    # 1
T[1:3]                  # (2, 3)
T + (5, 6)              # (1, 2, 3, 4, 5, 6)
T * 2                   # (1, 2, 3, 4, 1, 2, 3, 4)

Properties

  • Immutable: Cannot modify after creation
  • Indexing: Supports same indexing/slicing as lists
  • Methods: T.index(x) — return index of value; T.count(x) — count occurrences

Unpacking

a, b, c = (1, 2, 3)     # Tuple unpacking
x, *rest = [1, 2, 3, 4]  # x=1, rest=[2, 3, 4] (Python 3+)
first, *middle, last = range(5)  # First=0, middle=[1,2,3], last=4

Single-Level Immutability

T = (1, [3, 4], 3)
T[1][0] = 2             # Works! Lists inside tuples are mutable
T[0] = 5                # Error! Cannot reassign tuple element

# Tuple is only one-level-deep immutable

Sorting Tuples

T = (3, 5, 1, 6, 2)
L = list(T)
L.sort()
T = tuple(L)

Dictionaries & Sets

Dictionaries

Creation & Basics

D = {}                                          # Empty dict
D = {'food': 'Spam', 'quantity': 4}
D = dict(name='Bob', job='dev', age=40)        # Keyword args
D = dict(zip(['a', 'b', 'c'], [1, 2, 3]))      # From two lists
D = {k: v for (k,v) in zip(['a','b'], [1,2])}  # Dict comprehension

# Accessing
D['food']               # 'Spam'
D.get('missing', 'default')  # 'default' (safe access)
D['missing']            # KeyError!

Key Membership Testing

'food' in D             # True
'missing' in D          # False

# Safe access patterns
if 'key' in D:
    value = D['key']
else:
    value = 0

# Or use get()
value = D.get('key', 0)

Dictionary Methods

Method Purpose Example
D.keys() Return view of keys list(D.keys())
D.values() Return view of values list(D.values())
D.items() Return view of (key, value) tuples list(D.items())
D.get(key, default) Fetch with default D.get('x', 0)
D.pop(key, default) Remove and return value D.pop('key')
D.update(D2) Merge another dict D.update(D2)
D.setdefault(key, default) Get value or set default D.setdefault('x', []).append(1)
D.clear() Remove all items D.clear()
D.copy() Shallow copy D2 = D.copy()
D.popitem() Remove and return any (key, value) pair D.popitem()

Important Notes

  • Keys must be immutable: Strings, numbers, tuples allowed; lists, dicts, sets not allowed
  • Dict lookup is O(1): Use dicts instead of lists for fast membership testing
  • Python 3.7+: Dicts maintain insertion order (CPython 3.6+ too)

Nested Dictionaries

D = {'cto': {'name': 'Bob', 'age': 40}}
D['cto']['name']        # 'Bob'

rec = {'name': {'first': 'Bob', 'last': 'Smith'},
       'jobs': ['dev', 'mgr'],
       'age': 40.5}
rec['jobs'][-1]         # 'mgr'
rec['name']['first']    # 'Bob'

Dict vs. List Lookup

# Slower (O(n))
if x in [1, 2, 3]:
    pass

# Faster (O(1))
if x in {1, 2, 3}:
    pass

Dict Comprehensions

{x: x**2 for x in range(5)}                 # {0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
{k: v for k, v in zip(['a', 'b'], [1, 2])} # {'a': 1, 'b': 2}

Collections: OrderedDict — Python 2.7+

from collections import OrderedDict

D = OrderedDict([('z', 1), ('a', 2), ('b', 3)])
# Maintains insertion order even in Python 3.0-3.6

Collections: defaultdict — Python 2.5+

from collections import defaultdict

# Automatically creates values for missing keys
D = defaultdict(list)
D['key'].append(1)      # Works even if 'key' doesn't exist

# Count words example
s = [('yellow', 1), ('blue', 2), ('yellow', 3)]
d = defaultdict(list)
for k, v in s:
    d[k].append(v)
# d = {'yellow': [1, 3], 'blue': [2]}

Dictionary Sorting

D = dict(zip(['a', 'c', 'b'], [1, 2, 3]))
Ks = list(D.keys())     # ['a', 'c', 'b'] (note: D.keys() returns view)
Ks.sort()               # ['a', 'b', 'c']
for key in Ks:
    print(key, D[key])

Sets

Creation & Basics

S = set('spam')         # {'s', 'p', 'a', 'm'} — unordered
S = {'a', 'b', 'c'}     # Literal syntax
S = set()               # Empty set (note: {} is empty dict!)

'p' in S                # True
len(S)                  # 4

Set Operations

X = set('spam')
Y = set('ham')

X & Y                   # {'a', 'm'} — intersection
X | Y                   # Union
X - Y                   # {'s', 'p'} — difference
X > Y                   # X is superset? False
X < Y                   # X is subset? False

Set Comprehensions — Python 2.7+

{x**2 for x in range(5)}                    # {0, 1, 4, 9, 16}
{x for x in range(10) if x % 2 == 0}        # {0, 2, 4, 6, 8}

File I/O & Serialization

Opening Files

# Reading
f = open('data.txt', 'r')        # Read mode (default)
f = open('data.txt', 'rb')       # Binary read

# Writing
f = open('data.txt', 'w')        # Write mode (overwrites)
f = open('data.txt', 'a')        # Append mode

# With statement (auto-closes) — RECOMMENDED
with open('data.txt', 'r') as f:
    content = f.read()

Reading Files

# Read entire file
text = f.read()         # Single string (NOT one line!)
lines = f.readlines()   # List of lines (with \n)

# Read line by line
line = f.readline()     # Single line with \n
char = f.read(N)        # N characters

# Iterate through lines
for line in open('myfile.txt'):
    print(line, end='')

# Reset position
f.seek(0)               # Go to beginning
f.tell()                # Current position

Writing Files

f.write(string)         # Write string
f.writelines(list)      # Write list of strings
f.flush()               # Force write to disk (buffering)
f.seek(offset)          # Change position for next operation
f.truncate([size])      # Truncate to size

⚠️ Note: Output files are buffered. Use flush() or close() to ensure data is written.

File Context Manager

# Safe pattern - auto-closes
myfile = open(r'~/data.txt', 'r')
try:
    for line in myfile:
        print(line, end=' ')
finally:
    myfile.close()   # Optional in Python (auto-closes), but good habit

File Conversion

# Python relies on conversion functions to read from files
lines = open('file.txt').readlines()
lines = [line.rstrip() for line in lines]  # Remove \n

# int() ignores \n
lin = open('d.txt', 'r')
s = lin.readline()     # Returns '89\n'
i = int(s)             # 89 (ignores \n)

Pickle Module — Object Serialization

import pickle

# Save object
D = {'a': 1, 'b': 2}
F = open('database', 'wb')      # 'wb' required for pickle
pickle.dump(D, F)
F.close()

# Load object
F = open('database', 'rb')
E = pickle.load(F)              # Automatically converts to dict

Shelve Module — Key-Value Object Storage

import shelve

bob = Person("bob smith")
sue = Person("sue jones", job='dev', pay=1000)
tom = Manager("tom jones", 50000)

db = shelve.open('persondb')
for obj in (bob, sue, tom):
    db[obj.name] = obj
db.close()

# Later...
db = shelve.open('persondb')
person = db['bob']     # Retrieve object by key

Control Flow & Loops

if/elif/else

if x < y:
    print('less')
elif x > y:
    print('greater')
else:
    print('equal')

# Ternary expression
result = value_if_true if condition else value_if_false
A = Y if x else Z       # Assign Y if x is true, else Z

while Loops

while condition:
    statements
else:  # Runs if loop exits normally (not via break)
    statements

# Example: Find prime
x = y // 2
while x > 1:
    if y % x == 0:
        print(y, 'has factor', x)
        break
    x -= 1
else:
    print(y, 'is prime')

for Loops

for x in [1, 2, 3]:
    print(x)

for x in 'spam':
    print(x)

for key in D:           # Iterate dict keys
    print(key, D[key])

# Loop with counter
for offset, item in enumerate(['a', 'b', 'c']):
    print(f'{offset}: {item}')

# Zip iterables
for a, b in zip([1, 2, 3], ['a', 'b', 'c']):
    print(a, b)

# Range
for i in range(10):             # 0 to 9
for i in range(1, 11):          # 1 to 10
for i in range(0, 10, 2):       # 0, 2, 4, 6, 8

Loop Control

break               # Exit loop immediately
continue            # Skip to next iteration
pass                # Do nothing (placeholder)

range() & zip()

range(10)           # 0 to 9
range(1, 11)        # 1 to 10
range(0, 20, 2)     # 0, 2, 4, ..., 18 (step=2)

# zip combines iterables
list(zip([1, 2], ['a', 'b']))  # [(1, 'a'), (2, 'b')]

# In Python 3.0+, zip returns iterator; use list() to force

Boolean & Logic

# All objects have boolean value
# False: empty objects, 0, None
# True: non-empty objects, non-zero, any other value

# and/or return actual values (not just True/False)
result = X and Y    # Returns Y if X is True, else X
result = X or Y     # Returns X if X is True, else Y

# Example: default value
value = uservalue or default_value

# Conditional assignment
result = (X and Y) or Z  # Y if X is true, else Z
X = A or B or C or None  # First non-empty object

# in operator for membership
'p' in set('spam')
'ham' in ['eggs', 'spam', 'ham']

input() Function

# Get user input from console
reply = input('Enter text: ')

# Example: input loop with validation
while True:
    reply = input('Enter text:')
    if reply == 'stop':
        break
    elif not reply.isdigit():
        print('bad' * 8)
    else:
        print(int(reply) ** 2)
print('Bye')

Functions & Scope

Function Definition & Basics

def func(a, b, c=1):            # Default argument
    """Docstring for function"""
    return a + b + c

# Calling
func(1, 2)              # Uses default c=1
func(1, 2, c=5)         # Keyword argument

Arguments

Positional & Keyword

def f(a, b, c=3):
    return a + b + c

f(1, 2)             # Positional
f(1, b=2)           # Mixed
f(a=1, b=2, c=3)    # All keyword

*args (Variable Positional Arguments)

def f(*args):
    print(args)     # args is a tuple

f(1, 2, 3)          # args = (1, 2, 3)

# Unpacking
values = [1, 2, 3]
f(*values)          # Same as f(1, 2, 3)

**kwargs (Keyword Arguments) — Python 2.0+

def f(**kwargs):
    print(kwargs)   # kwargs is a dict

f(a=1, b=2)         # kwargs = {'a': 1, 'b': 2}

# Unpacking
d = {'a': 1, 'b': 2}
f(**d)              # Same as f(a=1, b=2)

Mixed Arguments Order (must follow this order)

def f(a, *args, b=None, **kwargs):
    pass

# Order: positional, *args, keyword-only, **kwargs

Keyword-Only Arguments — Python 3.0+

def f(a, *, b=None, c=None):  # b and c must be passed by keyword
    pass

f(1, b=2)           # OK
f(1, 2)             # Error! Can't pass 2 as positional

Scope (LEGB Rule)

# LEGB: Local → Enclosing → Global → Built-in

x = 1               # Global

def outer():
    x = 2           # Enclosing
    
    def inner():
        x = 3       # Local
        print(x)    # 3
    inner()

# Access global
def modify_global():
    global x
    x = 99          # Changes global x

# Access enclosing (Python 3+)
def outer():
    x = 2
    
    def inner():
        nonlocal x
        x = 3       # Changes enclosing x
    inner()
    print(x)        # 3

Default Mutable Arguments Gotcha ⚠️

# WRONG: default is created once
def append_to(element, to=[]):
    to.append(element)
    return to

append_to(1)        # [1]
append_to(2)        # [1, 2] (shared list!)

# CORRECT: use None
def append_to(element, to=None):
    if to is None:
        to = []
    to.append(element)
    return to

Multiple Return Values

def multi():
    return 1, 2, 3      # Returns tuple

a, b, c = multi()

Lambda (Anonymous Functions)

f = lambda x, y: x + y
f(2, 3)             # 5

# With defaults
f = lambda x, y=2: x + y
f(3)                # 5

# In comprehensions
funcs = [lambda x: x**2, lambda x: x**3]
funcs[0](2)         # 4

Docstrings

def func():
    """This is a docstring.
    
    Multi-line allowed.
    """
    pass

func.__doc__        # Access docstring
help(func)          # Display with help system

Recursive Functions

def mysum(L):
    if not L:
        return 0
    else:
        return L[0] + mysum(L[1:])

# One-liner alternative
def mysum(L):
    return 0 if not L else L[0] + mysum(L[1:])

Generators, Comprehensions & Iterators

List Comprehensions

[x**2 for x in range(5)]                            # [0, 1, 4, 9, 16]
[x for x in range(10) if x % 2 == 0]                # [0, 2, 4, 6, 8]
[[x, x**2] for x in range(3)]                       # [[0, 0], [1, 1], [2, 4]]

# Nested comprehensions
[x+y for x in 'abc' for y in 'lmn']                 # ['al', 'am', 'an', 'bl', ...]

# With conditions
[x for x in range(10) if x % 2 == 0 if x > 5]       # [6, 8]

# With dict iteration
[k for (k, v) in mydict.items() if v == V]

Performance: Comprehensions are 2x faster than for loops!

Dict Comprehensions

{x: x**2 for x in range(5)}                         # {0: 0, 1: 1, 2: 4, ...}
{k: v for k, v in zip(['a', 'b'], [1, 2])}         # {'a': 1, 'b': 2}

Set Comprehensions — Python 2.7+

{x**2 for x in range(5)}                            # {0, 1, 4, 9, 16}

Generator Expressions — Python 2.4+

G = (x**2 for x in range(5))    # Note: parentheses, not brackets
next(G)         # 0
next(G)         # 1

# Force all results
list(G)         # [0, 1, 4, 9, 16]

# File processing
lines = (line.rstrip() for line in open('file.txt'))

Generator Functions (yield) — Python 2.2+

def gen_squares(n):
    for i in range(n):
        yield i ** 2

G = gen_squares(5)
next(G)         # 0
next(G)         # 1

# Or iterate
for val in gen_squares(5):
    print(val)

Benefits:

  • Lazy evaluation (compute on-demand)
  • Memory efficient for large datasets
  • Can represent infinite sequences

Built-in Iterator Functions

map() — Python 2.0+

# Python 3.0+: returns iterator, not list
list(map(abs, [-1, -2, -3]))                    # [1, 2, 3]
list(map(lambda x, y: x+y, [1, 2], [3, 4]))     # [4, 6]

# In Python 3.0+, two-argument form acts like zip
list(map(None, [1, 2], [3, 4]))                 # Use zip() instead!

filter() — Python 2.0+

list(filter(lambda x: x > 0, [-1, 0, 1, 2]))    # [1, 2]
list(filter(bool, ['spam', '', 1]))             # ['spam', 1]

reduce() — functools module — Python 3.0+

from functools import reduce

reduce(lambda x, y: x+y, [1, 2, 3, 4])          # 10

zip() — Python 2.0+

list(zip([1, 2], ['a', 'b']))                   # [(1, 'a'), (2, 'b')]

# Multiple arguments
list(zip([1, 2], ['a', 'b'], ['x', 'y']))       # [(1, 'a', 'x'), (2, 'b', 'y')]

enumerate() — Python 2.3+

for i, v in enumerate(['a', 'b', 'c']):
    print(i, v)     # 0 a, 1 b, 2 c

Built-in Aggregation Functions

sum([1, 2, 3, 4])       # 10
min([3, 1, 4, 1, 5])    # 1
max([3, 1, 4, 1, 5])    # 5
any([False, False, True])  # True
all([1, 2, 3])          # True

Iterators & Iteration Protocol

# Get an iterator from an object
L = [1, 2, 3]
i = iter(L)
i.__next__()            # Returns first element (Python 3)
i.next()                # Python 2

# Manual iteration
I = iter(L)
while True:
    try:
        X = next(I)
    except StopIteration:
        break
    print(X ** 2)

Built-in Iterator Functions

itertools module (more in separate section)

import itertools

itertools.chain('abc', 'def')                   # Chain iterables
itertools.combinations([1, 2, 3], 2)            # Combinations
itertools.permutations([1, 2, 3])               # Permutations
itertools.product([1, 2], ['a', 'b'])           # Cartesian product
itertools.groupby(data, key)                    # Group consecutive items

Multiple vs. Single Pass Iterator

# Single pass iterator
R = range(3)
I1 = iter(R)
next(I1)                # Works

# range() itself is not an iterator
R = range(3)
next(R)                 # Error! range is iterable, not iterator

Modules & Packages

Import Basics

import module           # Import module
from module import name # Import specific name
from module import *    # Import all (discouraged)
import module as m      # Alias
from module import name as n  # Alias specific

Module Search Path

1. Home directory of program
2. PYTHONPATH directories (if set)
3. Standard library directories
4. .pth file contents (in site-packages)
5. site-packages directory (third-party packages)
import sys
sys.path                # View search path
sys.path.append('/custom/path')  # Add directory

Module Caching

# Modules are imported once and cached
import a
a.x = 99
import a              # a is not re-imported, x is still 99

# Force reload (Python 3.4+)
import importlib
importlib.reload(a)   # Re-execute a's code

Packages

mypackage/
    __init__.py       # Makes directory a package
    module1.py
    subpkg/
        __init__.py
        module2.py
import mypackage.module1
from mypackage import module1
from mypackage.subpkg import module2

Special Attributes

__name__            # Current module name (e.g., '__main__' for top-level)
__file__            # Module's filename
__doc__             # Module/function docstring
__dict__            # Namespace dictionary

Main Guard

if __name__ == '__main__':
    # Code only runs when executed directly, not imported
    test_code()

Dynamic Imports

# Reload a module after code changes
import importlib
importlib.reload(module_name)

# Note: Only works for Python modules, not C extensions

Object-Oriented Programming

Class Basics

class MyClass:
    class_var = 10      # Class attribute
    
    def __init__(self, name):
        self.name = name    # Instance attribute
    
    def method(self):
        return self.name

obj = MyClass('Alice')
obj.method()            # 'Alice'

Special Methods (Magic Methods / Dunder Methods)

Object Lifecycle

__new__(cls, *args, **kwargs)      # Object creation (before __init__)
__init__(self, *args, **kwargs)    # Object initialization
__del__(self)                      # Object destruction

String Representation

__str__(self)                      # str(obj), print(obj) — user-friendly
__repr__(self)                     # repr(obj) — developer-friendly
__format__(self, format_spec)      # f-strings, format()

Containers

__len__(self)                      # len(obj)
__getitem__(self, key)             # obj[key]
__setitem__(self, key, value)      # obj[key] = value
__delitem__(self, key)             # del obj[key]
__contains__(self, item)           # item in obj
__iter__(self)                     # for x in obj (returns iterator)
__next__(self)                     # next(iterator)

Operators

__add__(self, other)               # obj + other
__sub__(self, other)               # obj - other
__mul__(self, other)               # obj * other
__truediv__(self, other)           # obj / other (Python 3)
__floordiv__(self, other)          # obj // other
__mod__(self, other)               # obj % other
__pow__(self, other)               # obj ** other

# Right-hand operations
__radd__(self, other)              # other + obj
__rsub__(self, other)              # other - obj

# In-place operations
__iadd__(self, other)              # obj += other
__isub__(self, other)              # obj -= other

Comparisons

__eq__(self, other)                # obj == other
__ne__(self, other)                # obj != other
__lt__(self, other)                # obj < other
__le__(self, other)                # obj <= other
__gt__(self, other)                # obj > other
__ge__(self, other)                # obj >= other

Attributes

__getattr__(self, name)            # obj.undefined_attr (fallback)
__setattr__(self, name, value)     # obj.attr = value
__delattr__(self, name)            # del obj.attr
__getattribute__(self, name)       # obj.attr (called for all access)

Callable Objects

__call__(self, *args, **kwargs)    # obj(args)

Intercepting Slices

__getitem__(self, index)           # obj[key] or obj[slice]
__setitem__(self, index, value)    # obj[key] = value
__delitem__(self, index)           # del obj[key]

# When L[1:2] is called, a slice object is passed as second argument
class Indexer:
    data = [1, 2, 3, 4]
    def __getitem__(self, index):
        if isinstance(index, int):      # regular indexing
            return self.data[index]
        else:                           # slice
            return self.data[index.start:index.stop]

Comprehensive Operator Overloading Table

Method Implements Called for
__init__ Constructor X = class(args)
__del__ Destructor Object reclamation of X
__add__ Operator+ X+Y, X+=Y if no __iadd__
__or__ Bitwise or (bitwise or)
__repr__, __str__ Printing, conversions print(X), repr(X), str(X)
__call__ Function calls X(*args, **kwargs)
__getattr__ Attribute fetch x.undefined
__setattr__ Attribute assignment X.any = value
__delattr__ Attribute deletion del X.any
__getattribute__ Attribute fetch X.any
__getitem__ Indexing, slicing X[key], X[i:j], for loops
__setitem__ Index and slice assignment X[key] = value, X[i:j] = iterable
__delitem__ Index and slice deletion del X[key], del X[i:j]
__len__ Length len(x)
__bool__ Boolean tests bool(x), truth tests
__lt__, __gt__ Comparisons <, >, <=, >=, ==, !=
__le__, __ge__
__eq__, __ne__
__radd__ Right-side Operators Other + X
__iadd__ In-place augmented operators X += (or else __add__)
__iter__, __next__ Iteration contexts for loops, comprehensions, map(F, X)
__contains__ Membership test item in X
__index__ Integer value hex(X), bin(X), oct(X)
__enter__, __exit__ Context manager with obj as var
__get__, __set__, __delete__ Descriptor attributes X.attr, X.attr = value, del X.attr
__new__ Creation Object creation, before __init__

Inheritance

class Parent:
    def method(self):
        return 'parent'

class Child(Parent):
    def method(self):
        return 'child'

c = Child()
c.method()          # 'child'

super() — Python 2.6+ (improved in 3.0+)

# Python 3.0+ (simplified)
class Child(Parent):
    def method(self):
        super().method()        # Call parent method

# Python 2.x (explicit)
class Child(Parent):
    def method(self):
        Parent.method(self)     # Manual call
        # or
        super(Child, self).method()

Multiple Inheritance

class A:
    def method(self): return 'A'

class B:
    def method(self): return 'B'

class C(A, B):      # MRO: C → A → B → object
    pass

C().method()        # 'A' (leftmost parent)

# Check MRO
C.__mro__           # Method Resolution Order

Pseudoprivate Attributes

class C1:
    def meth1(self, value):
        self.__X = 88           # Converted to _C1__X

I = C1()
I.meth1(88)
print(I.__dict__)   # {_C1__X: 88}  automatically prefixed

Properties — Intercept Attribute Access — Python 2.2+

class Person:
    def __init__(self, name):
        self._name = name
    
    @property
    def name(self):
        return self._name
    
    @name.setter
    def name(self, value):
        self._name = value
    
    @name.deleter
    def name(self):
        del self._name

p = Person('Alice')
p.name              # Calls getter
p.name = 'Bob'      # Calls setter

Static & Class Methods — Python 2.2+

class MyClass:
    class_var = 0
    
    def instance_method(self):
        pass
    
    @staticmethod
    def static_method():
        return 'static'
    
    @classmethod
    def class_method(cls):
        return cls.class_var

MyClass.static_method()         # Works without instance
MyClass.class_method()          # Receives class, not instance

Slots — Memory Optimization — Python 2.2+

class Limited:
    __slots__ = ['name', 'age']  # Only these attributes allowed
    
obj = Limited()
obj.name = 'Alice'  # OK
obj.job = 'dev'     # AttributeError!

Iteration Support

getitem for indexing

class StepperIndex:
    def __getitem__(self, i):
        return self.data[i]

X = StepperIndex()
X.data = "spam"
for item in X:
    print(item, end=' ')    # s p a m

iter and next — Preferred approach

class Squares:
    def __init__(self, start, stop):
        self.value = start - 1
        self.stop = stop
    
    def __iter__(self):
        return self
    
    def __next__(self):
        if self.value == self.stop:
            raise StopIteration
        self.value += 1
        return self.value ** 2

for x in Squares(1, 5):
    print(x)        # 1 4 9 16 25

Multiple Iterators on One Object

class SkipObject:
    def __init__(self, wrapped):
        self.wrapped = wrapped
    
    def __iter__(self):
        return SkipIterator(self.wrapped)

class SkipIterator:
    def __init__(self, wrapped):
        self.wrapped = wrapped
        self.offset = 0
    
    def __next__(self):
        if self.offset >= len(self.wrapped):
            raise StopIteration
        item = self.wrapped[self.offset]
        self.offset += 2
        return item

iter with yield

class Squares:
    def __init__(self, start, stop):
        self.start = start
        self.stop = stop
    
    def __iter__(self):
        for value in range(self.start, self.stop + 1):
            yield value ** 2

Membership & Iteration

# __contains__ is preferred for "in" operator
class MyClass:
    def __contains__(self, item):
        return item in self.data

# __iter__ is preferred for iteration
# __getitem__ is fallback for both

# Order of preference for "in": __contains__ > __iter__ > __getitem__

Attribute Access Interception

# __getattr__ called when attribute not found elsewhere
class Empty:
    def __getattr__(self, attrname):
        if attrname == 'age':
            return 40
        else:
            raise AttributeError(attrname)

X = Empty()
X.age           # 40
X.name          # AttributeError: name

# __setattr__ called when setting attributes (WATCH FOR INFINITE LOOP!)
class Control:
    def __setattr__(self, attr, value):
        if attr == 'age':
            self.__dict__[attr] = value + 10    # Use __dict__ to avoid recursion!
        else:
            raise AttributeError(attr + " not allowed")

# __delattr__ is similar to __setattr__ (watch for infinite loop)

Right-side Operations

class adder:
    def __add__(self, val):
        return self.val + val
    
    def __radd__(self, val):
        return self.__add__(val)
    
    # Or shorthand
    __radd__ = __add__

In-place Operations

class adder:
    def __iadd__(self, val):
        self.val += val
        return self

Making Objects Callable

class Callee:
    def __call__(self, *arg, **argv):
        print("called", arg, argv)

c = Callee()
c(1, 2, 3, x=2, y=7)    # called (1, 2, 3) {'x': 2, 'y': 7}

String Representation

# __str__ for users, __repr__ for developers
class MyClass:
    def __repr__(self):
        return '[MyClass: %s]' % self.data
    
    def __str__(self):
        return 'MyClass(%s)' % self.data

X = MyClass()
X               # calls __repr__
print(X)        # calls __str__ if defined, else __repr__

Class Decorators & Introspection

# Add instance counter to class
def count(aClass):
    aClass.numInstances = 0
    return aClass

@count
class Spam: pass

@count
class Sub: pass

Spam.numInstances          # Access counter

Documentation & Introspection

# dir() - list all attributes
dir(sys)        # All attributes in sys module

# __doc__ - access docstring
class MyClass:
    """This is my class"""
    pass

MyClass.__doc__         # "This is my class"

# help() - display documentation
help(MyClass)

Decorators

Function Decorators Basics

# Decorator is a callable that returns a callable
def decorator(func):
    def wrapper(*args, **kwargs):
        print('before')
        result = func(*args, **kwargs)
        print('after')
        return result
    return wrapper

@decorator
def my_func():
    pass

# Equivalent to: my_func = decorator(my_func)

Class Decorators

# Using a class to wrap functions
class decorator:
    def __init__(self, func):
        self.func = func
    
    def __call__(self, *args):
        # Do something with self.func and args
        return self.func(*args)

@decorator
def func(arg):
    # do something
    pass

Factory Pattern for Class Decorators

def decorator(cls):
    class Wrapper:
        def __init__(self, *args):
            self.wrapped = cls(*args)
        
        def __getattr__(self, name):
            # Intercepts all attribute access!
            return getattr(self.wrapped, name)
    
    return Wrapper

@decorator
class C:
    def __init__(self, x, y):
        self.attr = 'spam'

x = C(6, 7)             # Calls Wrapper(6, 7)
print(x.attr)           # Runs Wrapper.__getattr__, prints 'spam'

Multiple Decorator Layers

@A
@B
@C
def f(...):
    pass

# Same as: f = A(B(C(f)))

Practical Examples

Tracing Calls

class tracer:
    def __init__(self, func):
        self.calls = 0
        self.func = func
    
    def __call__(self, *args):
        self.calls += 1
        print('call %s to %s' % (self.calls, self.func.__name__))
        return self.func(*args)

@tracer
def spam(a, b, c):
    print(a + b + c)

@property Decorator (Special Case)

class Student(object):
    @property
    def score(self):
        return self._score
    
    @score.setter
    def score(self, value):
        if not isinstance(value, int):
            raise ValueError("score must be an integer")
        if value < 0 or value > 100:
            raise ValueError("score must between 0-100")
        self._score = value

Context Managers (with/as)

Basic Usage

# Syntax: with/as is optional
with expression [as variable]:
    with-block

# Example: file handling
with open('file.txt') as f:
    content = f.read()  # Auto-closes

# Multiple context managers — Python 2.7, 3.1+
with open('data') as fin, open('res', 'w') as fout:
    for line in fin:
        if 'some key' in line:
            fout.write(line)

Context Management Protocol

# Implement __enter__ and __exit__

class MyContext:
    def __enter__(self):
        print('enter')
        return self
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        print('exit')
        # exc_type, exc_val, exc_tb are exception details
        # Return False to propagate exception, True to suppress
        return False

with MyContext() as ctx:
    pass

How It Works

  1. __enter__ is called and its return value assigned to variable in as clause
  2. Code in nested with block is executed
  3. If exception in with block: __exit__(type, value, traceback) called with exception details
  4. If no exception: __exit__(None, None, None) called

Unicode & Byte Strings

Encoding & Decoding

# Encoding: strings → raw bytes
S = 'ni'
S.encode('utf16')           # b'\xff\xfen\x00i\x00'
len(S.encode('utf16'))      # 6 bytes

# Decoding: raw bytes → strings
b'spam'.decode('utf8')      # 'spam'

Byte Types

# bytes - immutable byte sequence
b'xxxx'                     # Byte literals (ASCII only)

# bytearray - mutable byte sequence
ba = bytearray(b'hello')
ba[0] = ord('H')            # Change first byte

# Used for image/audio/binary data
# Open in binary mode: 'wb', 'rb'

Standard Library Tools

sys Module

import sys

sys.argv            # Command-line arguments
sys.path            # Module search path
sys.stdout          # Standard output
sys.stdin           # Standard input
sys.exit([code])    # Exit program
sys.version         # Python version info
sys.getrefcount(x)  # Reference count of object

os Module

import os

os.getcwd()                 # Current directory
os.chdir(path)              # Change directory
os.listdir(path)            # List files
os.path.exists(path)        # Check if exists
os.path.isfile(path)        # Is regular file?
os.path.isdir(path)         # Is directory?
os.path.join(a, b)          # Join paths
os.path.dirname(path)       # Directory part
os.path.basename(path)      # Filename part
os.makedirs(path)           # Create directories
os.remove(path)             # Delete file
os.rename(old, new)         # Rename file

pathlib — Object-oriented paths — Python 3.4+

from pathlib import Path

p = Path('file.txt')
p.exists()
p.is_file()
p.read_text()
p.write_text('content')
p.parent                    # Parent directory
p.name                      # Filename
p.stem                      # Filename without extension

datetime Module

from datetime import datetime, timedelta

now = datetime.now()
d = datetime(2024, 1, 1)
delta = timedelta(days=1)
future = now + delta

json Module

import json

json.dumps(obj)             # Object to JSON string
json.loads(json_str)        # JSON string to object

with open('file.json') as f:
    data = json.load(f)

with open('file.json', 'w') as f:
    json.dump(data, f)

copy Module

import copy

shallow = copy.copy(obj)
deep = copy.deepcopy(obj)

operator Module

import operator

operator.add(1, 2)          # 3
operator.itemgetter(0)      # Equivalent to lambda x: x[0]
operator.attrgetter('attr') # Equivalent to lambda x: x.attr

subprocess Module

import subprocess

# Execute shell command
subprocess.call('echo $HOME', shell=True)

# Get command output
proc = subprocess.Popen(["cat", '/tmp/bax'], stdout=subprocess.PIPE)
(out, err) = proc.communicate()

weakref Module

import weakref

# Weak references don't prevent object reclamation
# Useful for caches of large objects
weakref.ref(obj)

Advanced Iteration (itertools)

Module Overview

import itertools

# Python 2: many functions return lists
# Python 3: most return iterators (using itertools!)

Functions

accumulate(iterable[, func]) — Accumulated sums

# Returns accumulated sums or results of other binary functions
# If func supplied, should be function of two arguments
# Roughly equivalent to:
def accumulate(iterable, func=operator.add):
    it = iter(iterable)
    try:
        total = next(it)
    except StopIteration:
        return
    yield total
    for element in it:
        total = func(total, element)
        yield total

# Example
list(itertools.accumulate([1, 2, 3, 4]))    # [1, 3, 6, 10]

*chain(iterables) — Chain multiple iterables

# Returns elements from first iterable until exhausted, then next, etc.
# Note: argument is *iterable (multiple args), not single list!

itertools.chain('abc', 'def')               # a, b, c, d, e, f
# WON'T work: itertools.chain(['abc', 'def'])
# Use itertools.chain.from_iterable() instead

for item in itertools.chain([1, 2], [3, 4]):
    print(item)  # 1, 2, 3, 4

combinations(iterable, r) — r-length combinations

# Return r length subsequences of elements from input iterable
# Without replacement

itertools.combinations('ABCD', 2)           # AB AC AD BC BD CD
itertools.combinations([1, 2, 3], 2)        # (1,2), (1,3), (2,3)

combinations_with_replacement(iterable, r) — Combinations with replacement

# Return r length subsequences allowing individual elements repeated

itertools.combinations_with_replacement('ABC', 2)  # AA AB AC BB BC CC

permutations(iterable, r=None) — r-length permutations

# Return successive r length permutations of elements in iterable

itertools.permutations('ABCD', 2)           # AB AC AD BA BC BD CA CB CD DA DB DC

compress(data, selectors) — Filter with selector

# Make iterator that filters elements from data 
# returning only those with corresponding element in selectors that evaluates to True
# Stops when either iterable exhausted

itertools.compress('ABCDEF', [1, 0, 1, 0, 1, 1])  # A C E F

# Roughly equivalent to:
def compress(data, selectors):
    return (d for d, s in zip(data, selectors) if s)

count(start=0, step=1) — Infinite counter

# Make iterator that returns evenly spaced values starting with "start"
# Often used as argument to map() to generate consecutive data points
# Also used in zip() to add sequence numbers

itertools.count(10)                         # 10, 11, 12, 13, ...
itertools.count(2.5, 0.5)                   # 2.5, 3.0, 3.5, ...

cycle(iterable) — Repeat indefinitely

# Make iterator returning elements from iterable and saving a copy
# When iterable exhausted, return elements from saved copy
# Repeats indefinitely

itertools.cycle('ABCD')                     # A B C D A B C D ...

dropwhile(predicate, iterable) — Drop until predicate false

# Make iterator that drops elements from iterable as long as predicate is true
# Afterwards, returns every element
# Note: iterator doesn't produce output until predicate first becomes false

itertools.dropwhile(lambda x: x < 5, [1, 4, 6, 4, 1])  # 6, 4, 1

filterfalse(predicate, iterable) — Filter false elements

# Make iterator that filters elements from iterable 
# returning only those for which predicate is false
# If predicate is None, return items that are false

itertools.filterfalse(lambda x: x % 2, range(10))  # 0 2 4 6 8

groupby(iterable, key=None) — Group consecutive items

# Make iterator that returns consecutive keys and groups
# key is function computing key value for each element
# Similar to Unix "uniq"

[k for k, g in itertools.groupby('AAAABBBCCAABBB')]  # A B C A B
[[k, list(g)] for k, g in itertools.groupby('AAAABBBCCD')]  # [['A', 'AAAA'], ...]

# Returns groupby object with (key, group_iterator) tuples

islice(iterable, stop) & islice(iterable, start, stop[, step])

# Returns selected elements from the iterable

itertools.islice('ABCDEFG', 2)              # A, B
itertools.islice('ABCDEFG', 2, 4)           # C, D
itertools.islice('ABCDEFG', 2, None)        # C, D, E, F, G
itertools.islice('ABCDEFG', 0, None, 2)     # A, C, E, G

*product(iterables, repeat=1) — Cartesian product

# Cartesian product of input iterables
# Roughly equivalent to: (x,y) for x in A for y in B

itertools.product('ABCD', 'xy')             # Ax Ay Bx By Cx Cy Dx Dy
itertools.product(range(2), repeat=3)       # 000 001 010 011 100 101 110 111

repeat(object[, times]) — Repeat forever or n times

# Make iterator that returns object over and over again
# Repeats indefinitely unless times argument is specified

itertools.repeat('A', 3)                    # A, A, A
itertools.repeat('A')                       # A, A, A, A, ... (infinite)

starmap(function, iterable) — Unpack tuple arguments

# Make iterator that computes function using arguments obtained from iterable
# Used instead of map() when argument parameters already grouped in tuples
# Distinction: function(a, b) vs function(*c)

itertools.starmap(pow, [(2, 5), (3, 2), (10, 3)])  # 32 9 1000

takewhile(predicate, iterable) — Take until predicate false

# Make iterator that returns elements as long as predicate is true
# Once predicate becomes false, we stop

itertools.takewhile(lambda x: x < 5, [1, 4, 6, 4, 1])  # 1 4

tee(iterable, n=2) — Independent iterators

# Return n independent iterators (in a tuple) from a single iterable

c, d = itertools.tee([1, 2, 3], 2)
# Then c and d can iterate through list independently

*zip_longest(iterables, fillvalue=None) — Zip with fill

# Make iterator that aggregates elements from iterables
# If iterables are uneven length, missing values filled with fillvalue
# Iteration continues until longest iterable exhausted

itertools.zip_longest('abcd', 'xy', fillvalue='-')  # ax by c- d-

Collections & Data Structures

Module Overview

from collections import namedtuple, deque, Counter, OrderedDict, defaultdict

namedtuple — Lightweight class with named fields

Point = namedtuple('Point', ['x', 'y'])
p = Point(11, y=22)
p[0] + p[1]                 # 33 (indexable)
x, y = p                    # unpacked
p.x + p.y                   # 33 (attribute access)

# Useful for unpacking CSV/sqlite3 results

deque — Double-ended queue

from collections import deque

d = deque([1, 2, 3])
d.append(4)                 # Add to right
d.appendleft(0)             # Add to left

d.pop()                     # Remove from right
d.popleft()                 # Remove from left

d.extend([4, 5, 6])         # Extend right
d.extendleft([...])         # Extend left

d.rotate(n=1)               # Rotate n steps right
d.rotate(-1)                # Rotate left

d.count(x)                  # Count occurrences
d.remove(value)             # Remove first occurrence
d.clear()                   # Clear all
d.reverse()                 # Reverse in-place

# Thread-safe, memory efficient, O(1) operations on both ends

Counter — Tally occurrences

from collections import Counter

cnt = Counter()
for word in ['red', 'blue', 'red', 'green', 'blue', 'blue']:
    cnt[word] += 1

cnt                         # Counter({'blue': 3, 'red': 2, 'green': 1})

# Methods
cnt.elements()              # Iterator over elements repeating each
cnt.most_common([n])        # n most common elements and counts
cnt.subtract([iterable])    # Subtract counts
cnt.update([iterable])      # Add counts

OrderedDict — Maintain insertion order

from collections import OrderedDict

D = OrderedDict([('z', 1), ('a', 2), ('b', 3)])
# Remembers order of element insertion

D.popitem(last=True)        # Return and remove specified key
# last=True: remove last (LIFO), last=False: remove first (FIFO)

defaultdict — Auto-create missing values

from collections import defaultdict

# Automatically creates values for missing keys
D = defaultdict(list)
D['key'].append(1)          # Works even if 'key' doesn't exist

# Example: count words grouped by color
s = [('yellow', 1), ('blue', 2), ('yellow', 3)]
d = defaultdict(list)
for k, v in s:
    d[k].append(v)
# d = {'yellow': [1, 3], 'blue': [2]}

Interactive Python (IPython & Jupyter)

IPython — Enhanced Python Interpreter

# Tab completion
# Type and press Tab to complete names

# Introspection with ?
obj?                        # Display info about object
function?                   # Show docstring and signature

Magic Commands

# Magic commands prefixed with %

%timeit statement           # Time execution of statement
%time statement             # Report execution time of single statement
%debug                      # Activate interactive debugger
%pdb                        # Inspect stack frames on exception
%pwd                        # Show current working directory
%paste                      # Execute code from clipboard
%cpaste                     # Prompt for code to paste and execute
%quickref                   # Quick reference card
%magic                      # Display all magic commands
%hist                       # Print command input history
%reset                      # Delete all variables
%page OBJECT                # Pretty-print object through pager
%prun statement             # Profile code with cProfile
%who, %whos                 # Display variables in namespace
%xdel variable              # Delete variable and clear references
%matplotlib                 # Configure matplotlib integration
%load                       # Load code from file
%run filename.py            # Run Python script in IPython session

Jupyter Notebook

# Browser-based interactive Python
# Similar features to IPython
# Cell-based execution
# Rich output (plots, tables, etc.)

Scientific Computing

SciPy — Scientific Python

from scipy import integrate, linalg, optimize, signal, sparse, special, stats

# scipy.integrate - numerical integration, ODE solvers
# scipy.linalg - linear algebra (extends numpy.linalg)
# scipy.optimize - function optimizers, root finding
# scipy.signal - signal processing
# scipy.sparse - sparse matrices, sparse linear system solvers
# scipy.special - special math functions (SPECFUN Fortran library wrapper)
# scipy.stats - probability distributions, statistical tests

scikit-learn — Machine Learning

# Classification: SVM, nearest neighbors, random forest, logistic regression
# Regression: Lasso, ridge regression
# Clustering: k-means, spectral clustering

Common Mistakes & Best Practices

Assignment Creates References, Not Copies ⚠️

a = [1, 2, 3]
b = [0, a, 4]           # [0, [1, 2, 3], 4]
a[0] = 0
b                       # [0, [0, 2, 3], 4] — shared reference!

# Fix: use slice or copy()
a = [1, 2, 3]
b = [0, a[:], 4]        # [0, [1, 2, 3], 4]
a[0] = 0
b                       # [0, [1, 2, 3], 4] — independent copy

Shallow vs. Deep Copy

import copy

L1 = [1, 2, [3, 4]]
L2 = L1[:]              # Shallow copy
L2[2][0] = 99           # Affects L1[2] (shared inner list)

L2 = copy.deepcopy(L1)  # Deep copy
L2[2][0] = 99           # Doesn't affect L1

Default Mutable Arguments ⚠️

# WRONG: default is created once
def append_to(element, to=[]):
    to.append(element)
    return to

append_to(1)            # [1]
append_to(2)            # [1, 2] (shared list!)

# CORRECT: use None
def append_to(element, to=None):
    if to is None:
        to = []
    to.append(element)
    return to

List Multiplication Gotcha

L = [1, 2]
X = L * 2               # [1, 2, 1, 2]
Y = [L] * 2             # [[1, 2], [1, 2]] — both point to same list!

# Correct:
Y = [L[:] for _ in range(2)]    # Independent copies

Cyclic Data Structures

L = ['g']
L.append(L)
L                       # ['g', [...]] — prints [...] to avoid infinite recursion

# Avoid creating cycles if possible

String Immutability

S = 'spam'
S[0] = 'z'              # Error! Strings are immutable

# Correct:
S = 'z' + S[1:]         # Create new string

Type Checking — Anti-Pattern

# AVOID
type(x) == list         # Bad practice

# BETTER
isinstance(x, list)     # More Pythonic

# BEST - Use duck typing
# Don't ask "what is it?", ask "what can it do?"

Membership Testing Performance

# Slow (O(n))
if x in [1, 2, 3]:
    pass

# Fast (O(1))
if x in {1, 2, 3}:
    pass

Boolean Context

# False values
if not []:              # Empty list
if not 0:               # Zero
if not None:            # None
if not '':              # Empty string

# True values
if [1]:                 # Non-empty
if 1:                   # Non-zero

Loop else Clause (Useful!)

# else runs only if loop exits normally (without break)
for x in range(10):
    if x == 999:
        break
else:
    print('No break occurred')  # Prints

# Find prime with else
x = y // 2
while x > 1:
    if y % x == 0:
        print('has factor', x)
        break
    x -= 1
else:
    print('is prime')            # Runs if break didn't occur

Slice Assignment (Powerful but Tricky)

L = [1, 2, 3]
L[1:2] = [4, 5]         # [1, 4, 5, 3] — replace
L[1:1] = [6, 7]         # [1, 6, 7, ...] — insert
L[1:3] = []             # Remove slice
del L[1:]               # Delete from position onward

List vs. Dictionary Lookup

# Dictionary is much faster for membership testing
# Use dict for fast lookups, not lists

# Slow
if x in [1, 2, 3, 4, 5, ...]:
    pass

# Fast
if x in {1, 2, 3, 4, 5, ...}:
    pass

LEGB Scope Rule

# Local → Enclosing → Global → Built-in

# Use global to modify module-level variables
# Use nonlocal (Python 3+) to modify enclosing function variables

Comprehension Order (Inner Loop on Right)

# Inner loop is on the RIGHT
[x+y for x in 'abc' for y in 'lmn']
# Equivalent to:
result = []
for x in 'abc':
    for y in 'lmn':
        result.append(x+y)

Exception Handling Best Practices

# GOOD: catch specific exceptions
try:
    num = int(reply)
except ValueError:
    print('Invalid number')
except:                         # Only if absolutely necessary
    pass
else:
    print(num ** 2)             # Runs if no exception

# Don't use bare except unless absolutely necessary

Summary of Python Versions (3.6-3.12+)

Feature 3.6 3.7 3.8 3.9 3.10 3.11 3.12
F-strings
Walrus operator :=
Match statement
Dict ordering CPython
Type hints
Positional-only params
Async/await 3.5+
OrderedDict
defaultdict
namedtuple
itertools
Counter
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment