Alexis Kestler
Python is one of the most popular programming languages today due to its easy syntax, readability, and huge community support. It has a vast ecosystem of libraries and modules that make developers‘ lives easier when building applications. In this comprehensive guide, we will explore some of the most useful Python libraries and modules that every developer should know.
Collections
The collections module provides specialized container data types that provide alternatives to Python‘s general purpose built-in containers like dictionaries, lists, sets, and tuples.
defaultdict
The defaultdict is a subclass of the dict class that provides a default value for a dictionary key that does not exist. This avoids raising a KeyError exception when you try to access a non-existent key.
from collections import defaultdict
dict1 = defaultdict(int)
dict1[‘key1‘] = 1
print(dict1[‘key1‘]) # 1
print(dict1[‘key2‘]) # 0 - default valuedeque
The deque (double-ended queue) is a list-like container that allows fast appends and pops from both ends. This makes it useful for implementing queues and breadth-first tree searches.
from collections import deque
q = deque([1,2,3])
q.append(4)
q.appendleft(0)
print(q) # deque([0, 1, 2, 3, 4])Counter
Counter counts the occurrences of hashable objects. It can be useful for tallying, averaging, and statistics.
from collections import Counter
c = Counter([‘a‘, ‘b‘, ‘c‘, ‘a‘])
print(c) # Counter({‘a‘: 2, ‘b‘: 1, ‘c‘: 1})namedtuple
The namedtuple assigns names, as well as the numerical index, to each member of a tuple-like object. This allows for more readable and self-documenting code.
from collections import namedtuple
Point = namedtuple(‘Point‘, [‘x‘, ‘y‘])
p = Point(1, 2)
print(p.x, p.y) # 1 2CSV
The csv module implements classes for reading and writing tabular data in CSV format. CSV (comma separated values) is a common file format for exporting data from spreadsheets and databases.
import csv
with open(‘data.csv‘) as f:
reader = csv.reader(f)
for row in reader:
print(row)
headers = [‘Name‘, ‘Age‘]
rows = [
[‘John‘, 30],
[‘Mary‘, 28]
]
with open(‘data.csv‘, ‘w‘) as f:
writer = csv.writer(f)
writer.writerow(headers)
for row in rows:
writer.writerow(row) JSON
The json module provides an API similar to the csv module for parsing and generating JSON (JavaScript Object Notation) data. JSON is a lightweight data interchange format that is easy for humans to read and write, and easy for machines to parse and generate.
import json
person_dict = {‘name‘: ‘John‘, ‘age‘: 30}
person_json = json.dumps(person_dict)
print(person_json) # {"name": "John", "age": 30}
person_dict = json.loads(person_json)
print(person_dict) # {‘name‘: ‘John‘, ‘age‘: 30}Random
The random module provides a fast pseudo-random number generator for various distributions. It is useful for games, simulations, testing, security, and privacy applications.
import random
print(random.randint(1,10)) # random int between 1-10
print(random.random()) # random float between 0-1
print(random.choice([1, 2, 3])) # random element from listTkinter
Tkinter is Python‘s de-facto GUI framework that is built into the standard library. It provides a simple yet powerful interface for creating desktop applications.
import tkinter as tk
root = tk.Tk()
label = tk.Label(root, text="Hello World")
label.pack()
root.mainloop()Tkinter is easy to learn and great for building simple interfaces. For more complex GUIs, PyQT and wxPython are popular alternatives.
Requests
Requests allows you to send HTTP requests extremely easily. No need to manually add query strings, encode data, or handle redirects. Safe SSL connections are the default.
import requests
response = requests.get(‘https://api.github.com‘)
print(response.status_code)
response = requests.post(‘https://httpbin.org/post‘, data={‘key‘:‘value‘})
print(response.json())Requests simplifies RESTful API integrations. It‘s one of the most installed Python packages of all time for good reason.
BeautifulSoup
BeautifulSoup is a robust library for scraping data from HTML and XML documents. It creates a parse tree from page source that can be used to extract data in a structured way.
from bs4 import BeautifulSoup
soup = BeautifulSoup(html, ‘html.parser‘)
# Select all anchor tags
links = soup.find_all(‘a‘)
for link in links:
print(link.text)BeautifulSoup is commonly used for web scraping. It handles malformed markup gracefully and provides a simple API for navigating and searching a parse tree.
NumPy
NumPy adds support for large, multi-dimensional arrays and matrices along with a collection of high-level mathematical functions to operate on these arrays. It is the fundamental package for scientific computing in Python.
import numpy as np
array = np.array([1, 2, 3])
print(array.shape) # (3,)
matrix = np.array([[1, 2], [3, 4]])
print(matrix.shape) # (2, 2)NumPy is the backbone of computational libraries like Pandas, SciPy, Matplotlib, scikit-learn, Keras and more. It provides an efficient interface for working with dense data buffers.
Pandas
Pandas provides easy to use data structures and data analysis tools for working with tabular and time series data. It is built on top of NumPy and makes use of its fast array processing capabilities.
import pandas as pd
df = pd.DataFrame({
‘Name‘: [‘John‘, ‘Mary‘],
‘Age‘: [30, 25]
})
print(df)
# Name Age
# 0 John 30
# 1 Mary 25Pandas allows you to read, filter, transform and aggregate data with just a few lines of code. It is a must-have tool for data scientists and analysts.
Matplotlib
Matplotlib is the most popular Python library for producing 2D graphics and visualizations. It provides an object oriented API that allows you to build plots by adding one element at a time.
from matplotlib import pyplot as plt
plt.plot([1,2,3],[4,5,1])
plt.xlabel(‘x‘)
plt.ylabel(‘y‘)
plt.title(‘Demo Plot‘)
plt.show()From simple line charts to complex statistical visualizations, Matplotlib provides the building blocks to customize elegant visuals.
TensorFlow
TensorFlow is Google‘s open source framework for high performance numerical computation and deep learning. It uses data flow graphs to express complex mathematical computations that can be run efficiently across GPUs and clusters.
import tensorflow as tf
x = tf.constant(5)
y = tf.constant(2)
z = tf.add(x, y)
print(z) # Tensor("Add:0", shape=(), dtype=int32)
sess = tf.Session()
output = sess.run(z)
print(output) # 7TensorFlow powers large scale production machine learning applications at Google and many other companies. The eager execution API makes getting started with TensorFlow intuitive.
Keras
Keras is a high level API for building neural networks that runs on top of TensorFlow. It provides useful abstractions to reduce boilerplate and let you focus on optimizing your models.
from keras.models import Sequential
from keras.layers import Dense
model = Sequential([
Dense(10, input_shape=(5,)),
Dense(1)
])
model.compile(optimizer=‘sgd‘, loss=‘mse‘)
x_data = ... # input
y_true = ... # labels
model.fit(x_data, y_true, epochs=10)Keras makes prototyping, training, and testing deep learning models quick and efficient. It is favored for its ease of use and syntactic simplicity.
scikit-learn
Scikit-learn provides a unified interface for machine learning algorithms in Python. It builds on top of NumPy, SciPy and Matplotlib to provide easy to use tools for classification, regression, clustering, dimensionality reduction and model selection.
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target)
clf = RandomForestClassifier()
clf.fit(X_train, y_train)
print(clf.score(X_test, y_test))scikit-learn makes it quick and convenient to experiment with popular machine learning algorithms on your data. It is well documented and has an active community.
Django
Django is a full featured web framework for Python inspired by the Ruby on Rails framework. It provides a batteries included approach for developing server side web applications by providing an ORM, templating, authentication, URL routing and more.
# urls.py
from django.urls import path
from . import views
urlpatterns = [
path(‘‘, views.index)
]
# views.py
from django.http import HttpResponse
def index(request):
return HttpResponse(‘Hello World‘) Django helps take the complexity out of web development by providing solutions to common tasks out of the box. Its admin interface allows non-engineers to edit and manage content easily.
Flask
Flask is a microframework for Python based on Werkzeug and Jinja2. It is a minimalist approach to web development which provides just the core components needed for building web apps.
from flask import Flask
app = Flask(__name__)
@app.route(‘/‘)
def index():
return ‘Hello World!‘
if __name__ == ‘__main__‘:
app.run()Flask offers simplicity, flexibility and fine grained control. It gets out of your way so you decide how to glue components together. Flask is ideal for creating APIs, microservices and web apps.
Conclusion
Python ships with many useful modules and packages right out of the box. Third party libraries like NumPy, Pandas, scikit-learn, TensorFlow and Keras provide specialized capabilities that enable Python to be a versatile language for development and data science.
Web frameworks like Django and Flask allow you to build web applications of any scale and complexity quickly. Python has one of the largest and most active open source communities.
There are many more excellent Python libraries for tasks like web scraping (scrapy, BeautifulSoup), async processing (asyncio), GUI development (PyQt, wxPython), image processing (OpenCV, Pillow) and natural language processing (NLTK, spaCy).
I hope this guide gave you a taste of what‘s possible with Python. The language makes it easy to get started with these tools, even for beginners. Python‘s growth shows no signs of slowing down. Now is a great time to learn Python and level up your development skills!
