In: computer, programming, language.

Backlinks: javascript-lang.

Python (programming language)

Appeared: February 1991
Typing: duck, dynamic, strong typing; gradual
Implementations: CPython, PyPy, Stackless Python, MicroPython, CircuitPython, IronPython, Jython
Dialects: Cython, RPython, Starlark

Python is an interpreted high-level general-purpose programming language. Its design philosophy emphasizes code readability with its notable use of significant indentation.

Personal

I discovered Python in 2007 and fell in love instantly. I was working on a small game and Python made it really easy.
A year later I had a job where I used my Python skills a lot and became proficient. I used Python everywhere since then.
I did all kinds of projects with Python, anything you can imagine really, it’s a versatile language.

For X years in a row, Python was the most wanted language on Stackoverflow, which has to mean something:

• https://insights.stackoverflow.com/survey/2020#technology-most-loved-dreaded-and-wanted-languages-wanted

• https://insights.stackoverflow.com/survey/2019#technology-_-most-loved-dreaded-and-wanted-languages

• https://insights.stackoverflow.com/survey/2018#technology-_-most-loved-dreaded-and-wanted-languages

Python is no #1 in Tiobe index since 2021:
https://tiobe.com/tiobe-index/

Versions

CPython

PyPy

Stackless

Stackless Python programming language
https://github.com/stackless-dev/stackless
https://stackless.com

MicroPython

https://micropython.org
https://github.com/micropython/micropython
https://github.com/mcauser/awesome-micropython

Pyodide

Pyodide is a Python distribution for the browser and Node.js based on WebAssembly
https://github.com/pyodide/pyodide
https://pyodide.org
https://blog.pyodide.org

RustPython

Python Interpreter written in Rust
Compiled to WASM, runs in the browser
https://github.com/RustPython/RustPython

Brython

Python 3 implementation for client-side web programming
https://brython.info
https://pypi.org/project/brython

Libraries

Checking

Ruff

Extremely fast Python linter, written in Rust
Ruff can be used to replace Flake8 (plus a variety of plugins)
https://github.com/charliermarsh/ruff

Debugging and Profiling

Articles about Python memory profiling 📖 📚 ::

• https://medium.com/zendesk-engineering/hunting-for-memory-leaks-in-python-applications-6824d0518774

• https://medium.com/@narenandu/profiling-and-visualization-tools-in-python-89a46f578989

• https://pluralsight.com/blog/tutorials/how-to-profile-memory-usage-in-python

• https://towardsdatascience.com/making-python-programs-blazingly-fast-c1cd79bd1b32

Austin

Frame stack sampler for CPython written in pure C
Samples are collected by reading the CPython interpreter virtual memory space in order to retrieve information about the currently running threads along with the stack of the frames that are being executed
Make powerful statistical profilers that have minimal impact on the target application and that don’t require any instrumentation

• https://github.com/p403n1x87/austin

Guppy

Heap analysis toolset

• https://github.com/zhuyifei1999/guppy3

• https://zhuyifei1999.github.io/guppy3

line-profiler & kernprof

Line-by-line profiling of functions
kernprof is a convenient script for running either lineprofiler or the Python standard library’s cProfile or profile modules, depending on what is available

• https://github.com/pyutils/line_profiler

• https://pypi.org/project/line-profiler

Memory profiler

Monitor memory consumption of a process as well as line-by-line analysis of memory consumption for python programs
Pure Python module which depends on the psutil module

• https://github.com/pythonprofilers/memory_profiler

Memray

Track memory allocations in Python code, in native extension modules, and in the Python interpreter itself
Generate several different types of reports to help you analyze the captured memory usage data
https://github.com/bloomberg/memray

Pympler

Measure, monitor and analyze the memory behavior of Python objects in a running Python application
Written entirely in Python, with no ext dependencies

• https://pythonhosted.org/Pympler

• https://pythonhosted.org/Pympler/muppy.html

• https://github.com/pympler/pympler

Pyroscope

Continuous Profiling Platform. Debug performance issues down to a single line of code.
> pyroscope exec python manage.py runserver # If using Python

• https://github.com/pyroscope-io/pyroscope

• https://pyroscope.io

• https://twitter.com/PyroscopeIO

py-spy

Sampling profiler for Python programs
Visualize what your Python program is spending time on without restarting the program or modifying the code in any way
Is extremely low overhead: it is written in Rust for speed and doesn’t run in the same process as the profiled Python program.

• https://github.com/benfred/py-spy

• https://benfrederickson.com/profiling-native-python-extensions-with-py-spy

Snoop

A powerful set of Python debugging tools, based on PySnooper
https://github.com/alexmojaki/snoop

Scraping

AutoScraper

Smart, automatic web scraper

• https://github.com/alirezamika/autoscraper

CoCrawler

Versatile web crawler built using modern tools and concurrency

• https://github.com/cocrawler/cocrawler

Grab

Web scraping framework

• https://grab.readthedocs.io

• https://github.com/lorien/grab

HtmlMatch

Automatic data scraping

• https://github.com/santinic/htmlmatch

Pattern

Web mining (Google, Twitter and Wikipedia API, a web crawler, a HTML DOM parser), natural language processing (part-of-speech taggers, n-gram search, sentiment analysis, WordNet), machine learning (vector space model, clustering, SVM), network analysis andvisualization

• https://github.com/clips/pattern

• https://github.com/clips/pattern/wiki

PySpider

• https://github.com/binux/pyspider

Ruia

Async Python 3 web scraping micro-framework based on asyncio

• https://python-ruia.org

• https://github.com/python-ruia

Scrapy

Framework for extracting the data you need from websites

• https://scrapy.org

• https://github.com/scrapy/scrapy

Image

EXIF read/ write:

• https://github.com/hMatoba/Piexif

• https://github.com/ianare/exif-py

• https://github.com/python-pillow/Pillow

Articles

The origins of Python

Lambert Meertens
https://inference-review.com/article/the-origins-of-python

Python behind the scenes: How variables are implemented in CPython

https://tenthousandmeters.com/blog/python-behind-the-scenes-5-how-variables-are-implemented-in-cpython

echo ‘a = b’ | python -m dis
1 0 LOAD_NAME 0 (b)
2 STORE_NAME 1 (a)
4 LOAD_CONST 0 (None)
6 RETURN_VALUE

Python behind the scenes: How the Python import system works

https://tenthousandmeters.com/blog/python-behind-the-scenes-11-how-the-python-import-system-works

import m
m = __import__(‘m’, globals(), locals(), None, 0)

import a.b.c
a = __import__(‘a.b.c’, globals(), locals(), None, 0)

from a.b import f, g
a_b = __import__(‘a.b’, globals(), locals(), (‘f’, ‘g’), 0)

from m import *
m = __import__(‘m’, globals(), locals(), (‘*’,), 0)

from … import f
m = __import__(‘’, globals(), locals(), (‘f’,), 2)

Writing Python extensions in Assembly

https://tonybaloney.github.io/posts/extending-python-with-assembly.html

Asynchronous web scraping: Scaling for the moon

https://scrapecrow.com/asynchronous-web-scraping.html
Take a look at asynchronous python for scraping
Sync, async, async with throttler, async limiter with max rate

When your data doesn’t fit in memory: the basic techniques

Technique #1: Compression
technique #2: Chunking, loading all the data one chunk at a time
technique #3: Indexing, when you need a subset of the data
https://pythonspeed.com/articles/data-doesnt-fit-in-memory/

About memory in Python

• https://realpython.com/python-memory-management/

• https://pythonspeed.com/articles/python-object-memory/

• https://pythonspeed.com/articles/data-doesnt-fit-in-memory/

• https://stackabuse.com/basics-of-memory-management-in-python/

• https://geeksforgeeks.org/memory-management-in-python/

• https://fugue.co/blog/diagnosing-and-fixing-memory-leaks-in-python.html

Python type checkers to keep your code clean

Mypy, Pytype, Pyright/ Pylance, and Pyre/ Pysa can help you keep your type-hinted Python code bug-free
https://infoworld.com/article/3575079/4-python-type-checkers-to-keep-your-code-clean.html

Learn

• http://norvig.com/python-lisp.html

• https://docs.python-guide.org

• https://github.com/gto76/python-cheatsheet

• https://github.com/keon/algorithms

• https://github.com/satwikkansal/wtfpython

• https://github.com/trekhleb/learn-python

• https://github.com/vinta/awesome-python

• https://greenteapress.com/wp/think-python-2e

• https://hyperpolyglot.org/scripting

• https://learnpythonthehardway.org/book

• https://rosettacode.org/wiki/Category:Python

• https://tutorialspoint.com/python

• https://w3schools.com/python

Scientific Computing with Python, 300 hours, certification:
https://freecodecamp.org/learn/scientific-computing-with-python/

Machine Learning with TensorFlow, 300 hours, certification:
https://freecodecamp.org/learn/machine-learning-with-python/

Awesome

• https://github.com/jhermann/awesome-python-talks

• https://github.com/krzjoa/awesome-python-data-science

• https://github.com/lk-geimfari/awesomo/blob/master/languages/PYTHON.md

• https://github.com/mcauser/awesome-micropython

• https://github.com/mjhea0/awesome-flask

• https://github.com/TheAlgorithms/Python

• https://github.com/timofurrer/awesome-asyncio

• https://github.com/uhub/awesome-python

References

• https://python.org

• https://wikipedia.org/wiki/Python_%28programming_language%29