Compared with Node.js, Python has more "batteries" included and we can write functional script without external dependencies. This is a great boom for deployment.
It's also easier to create a single binary with all dependencies bundled for distribution, see python-settings#Packaging Application/Freezing.
Welcome to Python.org
Python Insider
Python Tutorials – Real Python
Python Training by Dan Bader – dbader.org
VSCode's Python Interactive mode is AMAZING! - YouTube
Advanced Visual Studio Code for Python Developers – Real Python
filiplajszczak/awesome-zen-of-python: awesome list of so called python philosophy resources
The Zen Of Python Is A Joke And Here Is Why (You Should Not Take It Too Seriously) - DEV Community
Clean Code in Python | TestDriven.io
All python code are executed statements, no such thing as declaration
Where is Python heading in 2018/19? When is version 4 coming out and what new features and improvements can we expect in next two years? - Quora
The creator of Python on how the programming language is learning from TypeScript - TechRepublic
Why is Python So Popular Among Programmers?
The Hitchhiker’s Guide to Python! source ❗!important
BeginnersGuide - Python Wiki
vinta/awesome-python: A curated list of awesome Python frameworks, libraries, software and resources
trananhkma/fucking-awesome-python: awesome-python with and
Awesome Python
Python Central | Python Programming Examples, Tutorials and Recipes
trekhleb/learn-python: 📚 Playground and cheatsheet for learning Python. Collection of Python scripts that are split by topics and contain code examples with explanations.
TheAlgorithms/Python: All Algorithms implemented in Python
Why Python? - DEV Community 👩💻👨💻
What Can I Do With Python? – Real Python
Python 3.7 beginner's cheat sheet | Opensource.com
Welcome to Python Cheatsheet! — pysheeet
An A-Z of useful Python tricks
From Anaconda Python to PyPy: Know your Python distributions | InfoWorld
How we rolled out one of the largest Python 3 migrations ever – Dropbox Tech Blog
Russell Keith-Magee - Keynote - PyCon 2019 - YouTube Black swan of Python
Changelog/Features History
A reverse chronology of some Python features
Python 3.11: Cool New Features for You to Try – Real Python better error messages; performance improvements; better async syntax; TOML; ExceptionGroup
Python 3.10: Cool New Features for You to Try – Real Python structural pattern matching; type unions, aliases guards;
Python 3.9: Cool New Features for You to Try – Real Python type hinting generics in standard collections; union on dict
Cool New Features in Python 3.8 – Real Python := operator; positional-only parameters; = in f-strings
Cool New Features in Python 3.7 – Real Python dataclasses; asyncio
Cool new features in Python 3.6 – dbader.org
Python on Windows
Episode #243 Python on Windows is OK, actually - [Talk Python To Me Podcast]
Steve Dower - Python on Windows is Okay, Actually - PyCon 2019 - YouTube
- Windows store (Python 3.7+)
3. Using Python on Windows — Python documentation
Python 3.8 - Microsoft Store Apps
This provides python3.exe already on PATH
- Installer (any version)
How To Install Python 3 on Windows {Quickstart}
The installer has a "Add Python to PATH" option
%LOCALAPPDATA%\Programs\Python\Python37-32;%LOCALAPPDATA%\Programs\Python\Python37-32\Scripts
Adding to path may cause an app with embedded
This provides py.exe, python.exe
.NET app uses whichever Python distribution on PATH
This is different from embedded Python (c.f. SublimeText), https://docs.python.org/3/using/windows.html#the-embeddable-package
This however may be a plus for our app as we can use Intel Distribution for Python
Runtime/Distributions
it's impossible to install multiple distros without coming to a mess, pick one
PythonDistributions - Python Wiki
Download Python | Python.org CPython, the official Python implementation
Intel® Distribution for Python | Intel® Software optimized for Intel CPUs
IronPython.net .NET Core, Python 2.7 only
Don't hardcode python3.7 as executable name
Python version should be a requirement to be satisfied (by pinning particular version or venv), python or python3 can be assumed to be the correct version
Learn
Perfect Your Python Development Setup (Learning Path) – Real Python
Getting started with... Python - Stack Overflow Blog
simeonfranklin.com - Python Fundamentals PDF 2013
4 day comprehensive tutorial, target audiences are not advanced
Python in Plain English
The 35 Words You Need to Python | yawpitchroll ❗!important
Learn Python By Example - PythonForBeginners.com
Python tutorial - beginner Python tutorial
9 Free Online Courses for Python Beginners
PY4E - Python for Everybody
Programming for Everybody (Getting Started with Python) | Coursera
Python for Everybody - Full University Python Course - YouTube
Python Tutorial for Beginners - YouTube freeCodeCamp, 2:52:12
trinket - Python 3 playground, with Skulpt
python-utils.com
The best Python websites and resources - The MagPi MagazineThe MagPi Magazine
Everything About Python — Beginner To Advanced - FinTechExplained - Medium
Learning Python: Best free books, tutorials and videos - TechRepublic
Practical Business Python -
Data Structures & Algorithms in Python - Self Paced
Preface - Python resources for everybody
Intermediate to Advanced resources - Python resources for everybody
Python Basics – Real Python
Intermediate Python Tutorials – Real Python
Learn Python Programming | Python Tutorial
Python Tutorials - Python Tutorial
Automate with Python – Full Course for Beginners - YouTube
What Does It Take To Be An Expert At Python? - YouTube
Expert Python Tutorials - YouTube
Tech With Tim
anthony explains - YouTube intermediate topics
Talk Python Training - Python tutorials and courses for developers I've bought some courses here
Python Practice Problems: Get Ready for Your Next Interview – Real Python
Python Tutorial for Beginners - Learn Python Programming from Scratch - DataFlair ❗!important
Learn Python through the Master Guide - Python Notes for Beginner to Advanced Learners - DataFlair ❗!important
Python Tutorial - Python for Beginners [Full Course] - YouTube 2019
Python For Data Science Full Course - 9 Hours | Data Science With Python | Python Training | Edureka - YouTube 2020
Python Tutorials – Real Python I've bought some books here
Python Programming Language: Guides, Tutorials, and Downloads
Teaching Python and more with open educational resources | Opensource.com
Pynative Python Tutorials: Learn Python With Examples and Exercises
Blog - Python Simplified
Python For Beginners | Python.org
The Python Tutorial — Python documentation
The Python Standard Library — Python documentation
The Python Language Reference — Python documentation
How To Python - Explore the World of Python Programming
Write Pythonic Code Like a Seasoned Developer - [Talk Python Training - Python tutorials and courses for developers] From Humble Bundle
Write More Pythonic Code (Learning Path) – Real Python
Intro to Computer Science | Udacity
Python for Everybody | Coursera
Python 3 Programming | Coursera
Introduction to Scripting in Python | Coursera
Introduction to Computer Science and Programming Using Python | edX
Introduction to Computing using Python | edX
Introduction to Python for Data Science | edX
Composing Programs
Programming for Everybody (Getting Started with Python) - Online Course
Data Science Courses: R & Python Analysis Tutorials | DataCamp
Python Programming For Data Science Course | Dataquest
Modern Python LiveLessons: Big Ideas and Little Code in Python
Modern Python LiveLessons: Big Ideas and Little Code in Python(英文字幕 CC)_哔哩哔哩 (゜-゜)つロ 干杯~-bilibili
Python Core and Advanced | Udemy
Learn Python Programming
Python3 Tutorial: Python Online Course
How to Think Like a Computer Scientist — How to Think Like a Computer Scientist: Learning with Python 3 3rd edition
Table of Contents — How to Think like a Computer Scientist: Interactive Edition
Computer Science Circles | 01000011 01010011 01000011
Learn Python for Data Science - Online Course
Python Central | Python Programming Examples, Tutorials and Recipes
Learn Python - Free Interactive Python Tutorial
Python 201 | The Mouse Vs. The Python
Python 101 | The Mouse Vs. The Python
Learn Python, it's CAKE (Beginners) | Udemy
Python for Data Science Course - Free Course
Python Fundamentals Training - YouTube !good Python 2.6
Google's Python Class - Google for Education Python 2.5
Introducing "Dead Simple Python" - DEV Community 👩💻👨💻
How to Do a Binary Search in Python – Real Python
All You Need To Know on APIs With Python | by James Briggs | Towards Data Science working with RESTful API
The Mouse Vs. The Python - Python Programming from the Frontlines
Python Programming Tutorials
BeginnersGuide - Python Wiki
Learning Python: From Zero to Hero – freeCodeCamp
IBM developerWorks : Linux : Technical library Charming Python, mostly Python 2
Hidden features of Python - Stack Overflow
Advanced Python Features
mdipierro/nlib: The book "Annotated Algorithms in Python" and the nlib.py library
Problem Solving with Algorithms and Data Structures using Python — Problem Solving with Algorithms and Data Structures
Python Resources – Python Tips
Books
PythonBooks - Learn Python the easy way !
Python 3 Patterns, Recipes and Idioms — Python 3 Patterns, Recipes and Idioms
A Byte of Python - GitBook
PY4E - Python for Everybody 2016, Python 3
Dive Into Python 3
Intermediate Python — Python Tips
Object-Oriented Programming in Python
Welcome to Python 101! — Python 101 documentation
Think Python 2e – Green Tea Press
Think Python
Think DSP – Green Tea Press
AllenDowney/ThinkDSP: Think DSP: Digital Signal Processing in Python, by Allen B. Downey.
Python 201 | Leanpub
Cover - 100 Page Python Intro
Intermediate Python | Leanpub
Python Data Science Handbook | Python Data Science Handbook
Introduction · A Byte of Python
Python for Informatics: Exploring Information
Nick Coghlan’s Python Notes
High Performance Python [Book] Python 2
Python Practice Book — Python Practice Book Python 2.7
Dive Into Python Python 2
python 3.7 极速入门教程 9 最佳 python 中文工具书籍下载
Al Sweigart
Invent with Python free books by Al Sweigart
Get the Best Python Books for Free - YouTube interview with Al Sweigart
Automate the Boring Stuff with Python 2nd edition, 2019
Hacking Secret Ciphers with Python 2013
Cracking Codes with Python 2nd edition of "Hacking Secret Ciphers with Python" 2018
The Big Book of Small Python Projects 2021, mostly single file console apps
The Recursive Book of Recursion 2022
Beyond the Basic Stuff with Python 2020
Video
PyVideo.org
anthonywritescode - YouTube
ArjanCodes - YouTube
mCoding - YouTube
Harvard CS50’s Introduction to Programming with Python – Full University Course - YouTube 15:57:47
Real Python - YouTube
6.0001 Introduction to Computer Science and Programming in Python. Fall 2016 - YouTube
Python 3 Basics Tutorial Series - YouTube
Python Basics - Jose Portilla - YouTube
Python Tutorials - Ardit Sulce - YouTube
Python (all parts in one) - YouTube Python 3, CodeSchool.org, compares 2.x and 3.x, quite good
Learn Python - Full Course for Beginners [Tutorial] - YouTube Python 3, ❗!important
Python – Intermediate and Advanced Features - YouTube Python 3, Real Python
A new video series for beginners to learn Python programming - Open Source Blog by Microsoft
Python for Beginners - YouTube
More Python for Beginners - YouTube
Python for Beginners | Channel 9
microsoft/c9-python-getting-started: Sample code for Channel 9 Python for Beginners course
PyCon Australia - YouTube
PyCon 2017 - YouTube
PyCon 2018 - YouTube
PyCon 2018 - YouTube
PyCon 2019 - YouTube
Blogs
miguelgrinberg.com
Python Tutorials – Real Python
Python Tutorials – dbader.org
Python Programming
Python Adventures | Welcome to the Jungle!
Python Tips – Your daily dose of bite sized python tips
Python | Abu Ashraf Masnun
Python · Abu Ashraf Masnun
Python Archives – Polyglot.Ninja()
30 Days of Python 👨💻 - Day 1 - Introduction - DEV
Master Python through building real-world applications
Reference
Overview — Python documentation
FrontPage - Python Wiki
History
Python 2.7 & Python 3: A Sacred Love Story - YouTube
Lesson Directory | Programming Historian
Governance
Mariatta Wijaya - What is a Python Core Developer? - PyCon 2018 - YouTube
Episode #153 How Python Evolves - [Talk Python To Me Podcast]
PEP 13 – Python Language Governance | peps.python.org
PEP 8000 – Python Language Governance Proposal Overview | peps.python.org
PEP 8016 – The Steering Council Model | peps.python.org adopted as PEP 13
Python Enhancement Proposals (PEP)
PEP-Explorer find and filter Python Enhancement Proposals
PEP 0 – Index of Python Enhancement Proposals (PEPs) | peps.python.org
PEP 572 – Assignment Expressions | peps.python.org
IDE
Which IDE is used for Python programming in software companies? - Quora
Top 8 IDEs for Raspberry Pi - Open Source For You
7 sweet Python IDEs you might have missed | InfoWorld
raspbian - Are there any Python IDEs for Raspberry Pi 3 with step-through capability? - Raspberry Pi Stack Exchange
prompt-toolkit/ptpython: A better Python REPL
Home - the bpython interpreter
Code With Mu a simple Python editor for beginner programmers, supports MicroPython
Getting started with Mu, a Python editor for beginners | Opensource.com
IDLE — Python documentation
Getting Started With Python IDLE – Real Python
PyCharm with Linter and Mypy integrated
PyCharm for Productive Python Development (Guide) – Real Python
NINJA IDE | Ninja-ide Is Not Just Another IDE
Thonny, Python IDE for beginners view stack, variable and AST
Wing Python IDE
Vim as a Python IDE - Martin Brochhaus - YouTube
Kite - AI Autocomplete and Docs for Python
Python - Visual Studio Marketplace
Spyder IDE
ŷhat | Data Science Operations an Python IDE written with Electron
IPython
see
data-analytics#jypyter
Jupyter and the future of IPython — IPython
IPython Documentation — IPython documentation
IPython Documentation - Jupyter Notebook Viewer
IPython Cookbook - IPython Cookbook, Second Edition (2018)
IPython Cookbook - IPython Cookbook, Second Edition (2018) source code
IPython Cookbook - IPython Minibook, Second Edition (2015)
Terminology
| Python | Java |
|---|---|
| object | Object |
| type | Class |
| attributes | member/field |
| try-except-raise | try-catch-throw |
c.f. JavaScript objects are inherently a dict, non-local declarations are made global
Builtin types
Common Python Data Structures (Guide) – Real Python
Learn Programming with Python — An Introduction | by Richard Quinn | The Startup | Medium
Understanding Data Types in Python | Python Data Science Handbook
list []: mutabletuple (): immutable,()are optionaldict {key: value}: map of key, value pairs,items(),keys(),values()set(list), {}: un-ordered collection of unique items in listzip(lists): return the columns of matrix formed using the lists as row
insertion-order preservation of dict object is part of language spec since 3.7 with the adoption of PyPy's dict implementation in 3.6.
type and object are instances of each other and themselves
module objects represents .py files, created by import, each global variables in file added as attributes
Lists in Python - Towards Data Science
5 Python Slicing Tricks That Will Make Your Code More Elegant | TechToFreedom
Seven Intermediate-Level Tips and Tricks for Python Lists | by Richard Quinn | Python In Plain English | Medium
Python List Vs. Tuple: An In-Depth Comparison | by Chetan Ambi | Towards AI
Python List Vs Tuple - Python Simplified
Python '!=' Is Not 'is not': Comparing Objects in Python – Real Python
is/is not test for identity, invokes (id())
==/!= test for equality, invokes (__eq__())
small integer (-5 to 256 inclusive) and short strings are interned, use intern(a) to get the interned address
Interning in CPython - Blog | luminousmen
3. Data model — Python documentation
graphlib — Functionality to operate with graph-like structures — Python documentation
Dictionaries in Python – Real Python
5 Expert Tips to Skyrocket Your Dictionary Skills in Python 🚀 | by Eirik Berge | Jul, 2021 | Towards Data Science
array — Efficient arrays of numeric values — Python documentation
collections — Container datatypes — Python documentation
A tactile guide to Python Collections - Towards Data Science
Python's collections: A Buffet of Specialized Data Types – Real Python
Data Structures in Python | Different Types of Data Structures in Python
How to Implement a Python Stack – Real Python
The Python math Module: Everything You Need to Know – Real Python
Operators
How to Use the Python or Operator – Real Python
The Walrus Operator: Python 3.8 Assignment Expressions – Real Python
Martin Heinz | The Unknown Features of Python's Operator Module
operator — Standard operators as functions — Python documentation
3 and Function Overloading in Custom Python Classes – Real Python dunder functions
Exceptions
Built-in Exceptions — Python 3 documentation
8. Errors and Exceptions — Python 3 documentation
Exception Handling Tips in Python ⚠ Write Better Python Code Part 7 - YouTube
Never use catch all except, Ctrl+C generates KeyboardInterrupt
Using except: is a HUGE mistake - YouTube
You can catch Exception (non-system-exiting and user exception)
try:
json = orjson.load(jsonstr)
# read value from `json`
except (orjson.JSONDecodeError, KeyError, ValueError) as err:
print(f"{type(err).__name__}: {err}")
finally:
# cleanup
pass
Packing/unpacking
Packing/unpacking is similar to JavaScript's destructuring and rest/spread operator.
They can be used whenever assignment is performed.
Unpacking Nested Data Structures in Python – dbader.org
a, *b, c = (1, 2, 3, 4)
# a, b, c = (1, [2, 3], 4)
for (key, value) in dictionary.items():
print(key, value)string
Strings and Character Data in Python – Real Python
Unicode & Character Encodings in Python: A Painless Guide – Real Python
unicodedata — Unicode Database — Python documentation
codecs — Codec registry and base classes — Python documentation
Python Idioms: Multiline Strings
textwrap — Text wrapping and filling — Python documentation
22 Pythonic Tricks for Working with Strings | by Richard Quinn | Python In Plain English | Medium
string formatting
The 4 Major Ways to Do String Formatting in Python – dbader.org
Python String Formatting Best Practices – Real Python
Python 3 F-Strings: The Advanced Guide (2021) - Miguel Brito
2.4.3. Formatted string literals — Python documentation f"", Python 3.6+
Python 3's f-Strings: An Improved String Formatting Syntax (Guide) – Real Python
PEP 498 – Literal String Interpolation | peps.python.org
PEP 502 – String Interpolation - Extended Discussion | peps.python.org
4.7.2. printf-style String Formatting "" % ()
6.1.3. Format String Syntax — Python documentation string.format()
6.1.3.1. Format Specification Mini-Language — Python documentation
"{key}={value:.3}".format(**{'key': 'foo', 'value': 3.14})6.1.4. Template strings — Python documentation string.Template
PEP 292 – Simpler String Substitutions | peps.python.org
Python Template String Formatting Method | by Vinicius Monteiro | Towards Data Science
Pathlib
pathlib — Object-oriented filesystem paths — Python 3 documentation
Python 3's pathlib Module: Taming the File System – Real Python
How to Get a List of All Files in a Directory With Python – Real Python
Should You Use Python pathlib or os? | by Adam Green | Better Programming
Starting With Python's pathlib Module - YouTube
defaultdict
collections.defaultdict — Container datatypes — Python documentation
defaultdict — Missing Keys Return a Default Value — PyMOTW 3
Python defaultdict | freeCodeCamp Guide
The first param of defaultdict is a callable to be called to generate the value when a key is first accessed.
Note this function does not take parameter, for more complex behavior overrite the __missing__(key) method.
dictionary - python: defaultdict with non-default argument - Stack Overflow
from collections import defaultdict
base_dict = { 'Alice': 'Chocolate' }
ice_cream = defaultdict(lambda: 'Vanilla', base_dict)
print(ice_cream['Alice']) # 'Chocolate'
print(ice_cream['Joe']) # 'Vanilla'>>> from collections import defaultdict
>>> s = 'mississippi'
>>> d = defaultdict(int)
>>> for k in s:
... d[k] += 1
...
>>> d.items()
[('i', 4), ('p', 2), ('s', 4), ('m', 1)]colors = ['red', 'green', 'red', 'blue', 'green', 'red']
# manual init
d = {}
for color in colors:
if color not in d:
d[color] = 0
d[color] += 1
# dict.get()
d = {}
for color in colors:
d[color] = d.get(color, 0) + 1
# defaultdict
d = defaultdict(int)
for color in colors:
d[color] += 1ChainMaps
Lookup values according to the chain, no need to merge dicts
collections — Container datatypes — Python documentation
ChainMap — Search Multiple Dictionaries — PyMOTW 3
ChainMap in Python - GeeksforGeeks
namedtuple
collections.namedtuple — Container datatypes — Python documentation
namedtuple — Tuple Subclass with Named Fields — PyMOTW 3
Namedtuple in Python - GeeksforGeeks
Writing Clean Python With Namedtuples – dbader.org
Python namedtuple - JournalDev
Understand how to use NamedTuple and Dataclass in Python with benchmarks
Use _make() to create from iterator, _asdict() to convert to a dict
deque
When you're removing from the head of a list, use deque
collections — Container datatypes — Python documentation
Python's deque: Implement Efficient Queues and Stacks – Real Python
deque — Double-Ended Queue — PyMOTW 3
Deque in Python - GeeksforGeeks
dataclasses
dataclasses — Data Classes — Python documentation
The Ultimate Guide to Data Classes in Python 3.7 – Real Python @dataclass dramatically ease class creation
from dataclasses import dataclass
@dataclass(kw_only=True, slots=True, frozen=True)
class Person:
name: str
_password: str(init=False, repr=False)
id: str = field(init=False, default_factory=random_id)
tags: list[str] = field(default_factory=list)
def __post_init__(self) -> None:
self._password = gen_password(name)Introducing Python 3.7's Dataclasses | Linux Journal
Understanding Python Dataclasses — Part 1 – Mindorks – Medium
Dataclasses In Python - DEV Community 👩💻👨💻
Python3.7 dataclass guide
JSON Encoding Python Dataclasses
Which Python @dataclass is best? Feat. Pydantic, NamedTuple, attrs... - YouTube
Attrs, Pydantic, or Python Data Classes: Which One Is The Best? - YouTube
Why I prefer attrs over dataclasses - YouTube __slots__ by default
konradhalas/dacite: Simple creation of data classes from dictionaries. with type checking
Dataclass Wizard — Dataclass Wizard documentation
rnag/dataclass-wizard: A simple, yet elegant, set of wizarding tools for interacting with Python dataclasses.
PEP 681 – Data Class Transforms | peps.python.org
typing — Support for type hints — Python 3.11.0b3 documentation
Raymond Hettinger - Dataclasses: The code generator to end all code generators - PyCon 2018 - YouTube
This Is Why Python Data Classes Are Awesome - YouTube
Python dataclasses will save you HOURS, also featuring attrs - YouTube
python - Data Classes vs typing.NamedTuple primary use cases - Stack Overflow
datetime
Deciphering Glyph :: Dates And Times And Types
glyph/DateType: A type wrapper for the standard library datetime that supplies stricter checks, such as making 'datetime' not substitutable for 'date', and separating out Naive and Aware datetimes into separate, mutually-incompatible types.
docstring
27.2. pydoc — Documentation generator and online help system — Python documentation
Charming Python: pydoc and distutils modules
pydoc — Online Help for Modules — PyMOTW 3
Documenting Python Code: A Complete Guide – Real Python
see
dev-testing.md#python
Internals
Python Developer’s Guide
Articles in tag "Python internals"
Ten thousand meters - Python behind the scenes
CPython Internals Book – Real Python
Python as C++’s limiting case - Brandon Rhodes - code::dive 2018 - YouTube
Demystifying Python’s Internals - presented by Sebastiaan Zeeff - YouTube repo
Get context with stack frame:
frame = inspect.currentframe()
caller_locals = frame.f_back.f_localsdis — Disassembler for Python bytecode — Python documentation
Lifecycle of a Python Code - CPython's Execution Model - DEV Community 👩💻👨💻
Use dis.dis() to pretty print a function's byte code; see adaptive, show_caches arguments
Dragon taming with Tailbiter, a bytecode compiler for Python
PyBites – Under the Hood: Python Comparison Breakdown
zrax/pycdc: C++ python bytecode disassembler and decompiler
rocky/python-xdis: Python cross-version bytecode library and disassembler
pycdc compared with uncompyle6 · rocky/python-uncompyle6 Wiki
rocky/python-xasm: Python cross version bytecode/wordcode assembler
rocky/python-uncompyle6: A cross-version Python bytecode decompiler
Python lists remember what you did to them - YouTube
Your Guide to the CPython Source Code – Real Python
Slightly Advanced Python: Some Python Internals - YouTube Python 2.4, class construction, attr lookup, bytecode (assembly)
Advanced Python or Understanding Python - YouTube Python 2.4
Memory Management
Memory Management — Python documentation
Memory Management in Python – Real Python
A Short Overview of CPython's Memory Management - DEV Community 👩💻👨💻
Pointers in Python: What's the Point? – Real Python
Python memoryview()
Garbage Collector with Pandas DataFrame | Apratim Biswas | Dec 2020 | Towards Data Science
Global Interpreter Lock/GIL
What is the Python Global Interpreter Lock (GIL)? – Real Python
Dabeaz: The Python GIL Visualized
PEP 684 – A Per-Interpreter GIL | peps.python.org
PEP 554 – Multiple Interpreters in the Stdlib | peps.python.org Python API for interpreters
PEP 489 – Multi-phase extension module initialization | peps.python.org for extensions sharing global states to support multiple interpreters
Talks - Eric Snow: A Per-Interpreter GIL: Concurrency and Parallelism with Subinterpreters - YouTube
PEP 703 – Making the Global Interpreter Lock Optional in CPython | peps.python.org
Python moves to remove the GIL and boost concurrency | InfoWorld
Talks - Łukasz Langa: Working Around the GIL with asyncio - YouTube
python is removing the GIL! (PEP 703) (advanced) anthony explains #550 - YouTube
Understanding the Python GIL - YouTube
Talks - Alireza Farhidzadeh: Getting Around the GIL: Parallelizing Python for Better Performance - YouTube
PyVideo.org · to GIL or not to GIL: the Future of Multi-Core (C)Python slides
Python is NOT Single Threaded (and how to bypass the GIL) - YouTube
Keynote: Multithreaded Python without the GIL - presented by Sam Gross - YouTube
Syntax
syntactic sugar - Tall, Snarky Canadian
The Python return Statement: Usage and Best Practices – Real Python
typing.Protocols
added in 3.8
PEP 544 – Protocols: Structural subtyping (static duck typing) | peps.python.org
Protocol Or ABC In Python - When to use which one? - YouTube
- ABC: nominal typing, derived class has to import abc, error at object construction
- Prototype: structural/duck typing, better, smaller interface, abstract 3rd library, error at object usage
Protocol name can be Iterable, PathLike, Ducky
Deciphering Glyph :: I Want A New Duck structural vs nominal typing
Deciphering Glyph :: Interfaces and Protocols
Protocol Types in Python 3.8
Protocols and structural subtyping - mypy documentation
Functions
Glossary parameter — Python documentation
Understanding '*', '*args', '**' and '**kwargs' - Agiliq Blog | Django web app development
Python args and kwargs: Demystified – Real Python
Stop Abusing *args and **kwargs in Python | by Eden Au | Better Programming | Medium
Python Inner Functions—What Are They Good For? – Real Python
Positional-Only Arguments – Real Python
/ indicates end of positional-only arguments
`*`` indicates start of keyword-only arguments
def headline(text, /, border="~", *, width=50):
return f" {text} ".center(width, border)text is positional-only
border can be specified both with and without the keyword
width must be specified using the keyword
Parameters
- positional parameter:
a - keyword parameter:
b=default - variable positional parameter:
*args - variable keyword parameter:
**kwargs
Calling
arguments can always be passes as keyword arguments regardless of their definition
*list/*tuple can unpack list/tuple as positional arguments
**dict can unpack dict as keyword arguments
Default for mutable parameter
Python Pitfall: Mutable Default Arguments | by Don Cross | Towards Data Science
Function parameter are attributes of function object instance
Default value is only applied upon function definition
def foo(a, b=[]): # wrong,
# default list persists in `foo` object after 1st call
b.append(a)
print(a, b)
def foo(a, b=None): # correct
if b is None: b = []
b.append(a)
print(a, b)Overloading (single/multiple dispatch)
Providing Multiple Constructors in Your Python Classes – Real Python
functools — singledispatch
functools — singledispatchmethod
multipledispatch · PyPI
multimethod · PyPI
Variable Namespace
Only functions, class and modules create namespaces
variables first used in flow controls are actually hoisted to their scope
LEGB: Local -> Enclosed -> Global -> Built-in
Python Scope & the LEGB Rule: Resolving Names in Your Code – Real Python
Assignment (lvalue)
if object not found in local scope, creates it
use nonlocal (enclosed)/global keywords to override
Reference (rvalue/function)
need nonlocal/global keywords when referencing non-local variables
nonlocal statement in Python
Classes
3. Data model — Python documentation
Understanding Python Class Instantiation
Python object creation sequence - Eli Bendersky's website
Classes in Python. Understanding Object Oriented… | by Sadrach Pierre, Ph.D. | Towards Data Science
Enriching Your Python Classes With Dunder (Magic, Special) Methods – dbader.org
Python Dunder Methods: The Ugliest Awesome Sauce - CodeSolid.com
3 practical Python tools: magic methods, iterators and generators, and method magic | Opensource.com
Using Magic Methods in Python - Towards Data Science
Magic Methods in Python, by example - Towards Data Science
5 Pairs of Magic Methods in Python That You Should Know | by Yong Cui | Better Programming | Medium
getitem and setitem in Python - GeeksforGeeks for indexed attributes
python - Difference between @staticmethod and @classmethod - Stack Overflow classmethod may be overridden by subclass
type(obj) vs. obj.class in Python, and changing an object's class. - YouTube
OOP
Talks - Hynek Schlawack: Subclassing, Composition, Python, and You - YouTube author of attrs
Why COMPOSITION is better than INHERITANCE - detailed Python example - YouTube
Object-Oriented Programming (OOP) in Python 3 – Real Python
Inheritance and Composition: A Python OOP Guide – Real Python
Object-Oriented Programming (OOP) in Python 3 – Real Python
Object-Oriented Programming in Python
Learn object-oriented programming with Python | Opensource.com
Python Metaclasses ~ The Python Corner
super, Python's most misunderstood feature. - YouTube Method Resolution Order (MRO), also applicable to attribute
Object-oriented programming in Python - YouTube
Inherit classes from object to signal usage of new style classes
Avoid multiple inheritance with exception to mix-ins (for non-overridden functions not representing IS-A relationship)
attribute/descriptor/@property
Use @property declaration to abstract member access or derived/computed attribute (simpler then __getattribute__/__setattr__)
The @property Decorator in Python: Its Use Cases, Advantages, and Syntax
the python @property decorator (beginner - intermediate) anthony explains #005 - YouTube cached_property
python typing: @property (beginner - intermediate) anthony explains #117 - YouTube
Python: Declaring Dynamic Attributes
Python Attribute Access and the Descriptor Protocol
8 things in Python you didn't realize are descriptors - YouTube 3.0, Descriptor, __getattribute__(),__getattr__()
python descriptors! (advanced) anthony explains #519 - YouTube
attrs: Classes Without Boilerplate — attrs documentation annotation and frozen class, more powerful than dataclasses
python-attrs/attrs: Python Classes Without Boilerplate
Deciphering Glyph :: The One Python Library Everyone Needs
__file__
Prints the path the module is form
__repr__
__repr__ is supposed to reconstruct the object via eval()
object == eval(repr(object))
Python String Conversion 101: Why Every Class Needs a “repr” – dbader.org
When Should You Use .repr() vs .str() in Python? – Real Python
Can someone explain what repr() does a little more thoroughly please? | Codecademy
python - Difference between str and repr? - Stack Overflow
__slots__
can be used in
dataclassdeclarator easily in 3.10
UsingSlots - Python Wiki
Python | Use of __slots__ - GeeksforGeeks
Use named tuple rather than __dict__ for faster access, cannot add new attributes
A Simple & Effective Way To Improve Python Class Performance - YouTube
slots breaks multiple inheritance
some libraries may depends of __dict__
Python __slots__ and object layout explained - YouTube
Abstract Base Classes
30.8. abc — Abstract Base Classes — Python documentation
abc — Abstract Base Classes — PyMOTW 3
Python Tutorial: 'The ABC' of Abstract Base Classes
Interfaces, Mixins and Building Powerful Custom Data Structures in Python | Red’s Digressions
ABC can have concrete method and fields
Arithmetic
8/3 = 2.666666
8//3 = 2
-8//3.0 = -3 # rounded down!!int(float) is rounding, use floor() for C-like casting
use open delimiters to override indentation rules
Context Managers
contextlib — Utilities for with-statement contexts — Python documentation
contextlib — Context Manager Utilities — PyMOTW 3
27. Context Managers — Python Tips 0.1 documentation
Context Manager in Python - GeeksforGeeks
with statement in Python - GeeksforGeeks
This can be implemented by a class with __enter__() and __exit__().
However it is better with @contextmanager decorator
@contextmanager
def context_manager_example():
print("__enter__")
try:
yield
finally:
print("__exit__")
with context_manager_example():
print("Run operations with the context")Context Managers and the “with” Statement in Python – dbader.org
Python Context Managers and the "with" Statement (enter & exit) - YouTube
Python Context Managers and the "with" Statement – Real Python
Context Managers and Python's with Statement – Real Python
Martin Heinz - The Magic of Python Context Managers
Introduction to Context Managers in Python
Python with Context Managers
Context Managers — Python Tips documentation
Context Managers in Python — Go Beyond “with open() as file” | by Yong Cui | Better Programming | Medium
Ignore errors
Python: “ignored” context manager - Ruslan Osipov
with ignored(OSError):
os.remove('somefile.tmp')Structural Pattern Matching (~= switch)
Python 3.10: Cool New Features for You to Try – Real Python
The real purpose of Python's match statement, feat. CSTs - YouTube
PEP 634 – Structural Pattern Matching: Specification | peps.python.org
PEP 635 – Structural Pattern Matching: Motivation and Rationale | peps.python.org
PEP 636 – Structural Pattern Matching: Tutorial | peps.python.org
"Structural pattern matching" for Python, part 1 [LWN.net]
"Structural pattern matching" for Python, part 2 [LWN.net]
Python Switches to Match-Case - DEV Community
A Closer Look At Structural Pattern Matching // New In Python 3.10! - YouTube
The Hottest New Feature Coming In Python 3.10 - Structural Pattern Matching / Match Statement - YouTube
Decorators
Primer on Python Decorators – Real Python
Python Decorators: A Step-By-Step Introduction – dbader.org
Python | functools.wraps() function - GeeksforGeeks usually used with functools.wrap()
Why we should use Python Decorator more often | by Donald Le | Jul, 2020 | Towards Data Science with arguments
How to Use Decorators in Python, by example | by Stephen Fordham | Towards Data Science
Decorators — Python 3 Patterns, Recipes and Idioms
Level up your code with Python decorators | by Alexander Bailey | Towards Data Science
Tutorial: Geir Arne Hjelle - Introduction to Decorators: Power Up Your Python Code - YouTube
10 Python Decorators To Take Your Code To The Next Level | by Ahmed Besbes | Towards Data Science
Python's Instance, Class, and Static Methods Demystified – Real Python
Instance vs. Static vs. Class Methods in Python: The Important Differences @staticmethod, @classmethod
Python staticmethod() - Python Standard Library
Python classmethod() - Python Standard Library
Instance vs. Static vs. Class Methods in Python: The Important Differences
Py in 5: Decorators - DEV Community 👩💻👨💻
Python 修饰器的函数式编程 | | 酷 壳 - CoolShell
Are you using Python with APIs? Learn how to use a retry decorator!
Decorator inside Python class. Let’s suppose we need a decorator that… | by Vadim Pushtaev | Medium
Python Decorators: The Complete Guide - YouTube ❗!important
def logging_decorator(func: Callable[..., Any]):
@functools.wraps(func)
def wrapper(*args: Any, **kwargs: Any):
print(f'calling {func.__name__}')
value = func(*args, **kwargs)
print(f'called {func.__name__}')
return value
return wrapperTutorial: Geir Arne Hjelle - Introduction to Decorators: Power Up Your Python Code - YouTube
Bytepawn - Marton Trencseni – Python decorator patterns
Bytepawn - Marton Trencseni – Useful Python decorators for Data Scientists
Debugging
pdb — The Python Debugger — Python documentation
Your Guide to the Python Print Function – Real Python
Understanding the Python Traceback – Real Python
Working with pdb to Debug Python Code | DigitalOcean
Debugging memory usage in a live Python web app – dbader.org
How to use the Python debugger | InfoWorld
python -m pdb script.pyinducer/pudb: Full-screen console debugger for Python
robdmc/behold: A Debugging Tool
Iterables/Generator
iterable: object with __iter__() which returns an iterator (better use iter(object) to get the iterator); for container class a new iterator is returned for each iter(object) call
iterator: object with _ _next__() which returns a value, can call next() on it; it raises StopIteration exception when there are no more items; iterator.__iter__() returns itself
generator: syntax sugar for declaring iterator and iterable as a function with yield expression, allows bidirection communication with send()
You can iterate over a list as many times as you want but you can iterate over an iterator only once.
for-in loop creates the iterable and catch StopIteration automatically.
The Iterator Protocol: How for Loops Work in Python - Trey Hunner ❗!important, see "The Iterator Protocol"
Learn To Loop The Python Way: Iterators And Generators Explained | Hackaday ❗!important
Introduction to Python Generators – Real Python
Iterables vs. Iterators vs. Generators » nvie.com
Glossary iterable — Python documentation
Built-in Types — Python documentation iterator
Built-in Types — Python documentation generator
Expressions — Python documentation Yield expressions
Iterators & Iterables in Python. Introduction to Iterators and Iterables | by Sadrach Pierre, Ph.D. | Towards Data Science
async iterator in 3.10
PEP 492 – Coroutines with async and await syntax | peps.python.org
PEP 525 – Asynchronous Generators | peps.python.org
Tour of Python Itertools - Martin Heinz
A Deep Dive Into Iterators and Itertools in Python - YouTube
compress- one iterator to another eliminating elements that fail a bool expressionaccumulate- likefunctools.reducebut returns all intermediate valuescycle- so cool, create a never ending repeating iterabletee- multiple references to one iterabledivide- divides iterable into sub-iterablespartition- split into two based on a predicate bool expressionside_effect- attach a side effect function to an iterable that gets called with each elementcollapse- like flattensplit_at- multiple iterables splitting at divider items, specified with predicatebucket- multiple iterables based on multi-return-value expressionmap_reduce- specify 3 functions: key function (for categorizing), value function (for transforming) and finally reduce function (for reducing).sort_togetherseekablefilter_exceptunique_to_each
itertools — Functions creating iterators for efficient looping — Python documentation recipes
itertools — Iterator Functions — PyMOTW 3
Itertools in Python 3, By Example – Real Python
A gentle introduction to itertools · Justin Duke
Python 201: An Intro to itertools | The Mouse Vs. The Python
Tour of Python Itertools - Towards Data Science
dictionary - python groupby behaviour? - Stack Overflow
5 Advanced Functions in Itertools To Simplify Iterations in Python
7 Python Iterators You (Maybe) Didn’t Know About | by Branislav Holländer | Towards Data Science
Python Generators
Understanding Python's "yield" Keyword
Generators - Python Wiki
How do Generators Work? | Towards Data Science
Python Generators - YouTube
What Are Python Generators? – dbader.org
Generator Expressions in Python: An Introduction – dbader.org
Iterator Chains as Pythonic Data Processing Pipelines – dbader.org
Iterables vs. Iterators vs. Generators » nvie.com
Use More Iterators » nvie.com
Generator Tricks for Systems Programmers
Generator Tricks for Systems Programmers - Version 2.0
Generators: The Final Frontier
Cleaning Up in a Python Generator Can Be Dangerous
Concurrency
threading in Python is not effective due to GIL, focus on multiprocessing
Is Python Really Scalable? - Bobby - Medium
threading vs multiprocessing in python - YouTube
Ray – Fast and Simple Distributed Computing
Modern Parallel and Distributed Python: A Quick Tutorial on Ray | by Robert Nishihara | Towards Data Science
10x Faster Parallel Python Without Python Multiprocessing | by Robert Nishihara | Towards Data Science
How to Run Python Functions in Parallel Using Ray API | by Konstantinos Patronas | Better Programming
python concurrent.futures.ProcessPoolExecutor: Performance of .submit() vs .map() - Stack Overflow
David Beazley - Python Concurrency From the Ground Up: LIVE! - PyCon 2015 - YouTube
Super Fast Python – making you awesome at concurrency ❗!important
Multiprocessing in Python: The Complete Guide
Threading vs Multiprocessing in Python
ProcessPoolExecutor in Python: The Complete Guide
Parallel Nested For-Loops in Python
Concurrency in Python Tutorial - Tutorialspoint
A Curious Course on Coroutines and Concurrency PDF
Python Concurrency: The Tricky Bits -
Async Python: The Different Forms of Concurrency · Abu Ashraf Masnun
Asynchronous Python – Hacker Noon
Python & Async Simplified - Aeracode
Getting Started With Async Features in Python – Real Python concurrency model, queue, asyncio, aiohttp
Speeding Up Python with Concurrency, Parallelism, and asyncio | TestDriven.io
Async Python Tutorial: Foundations for those with no prior async experience - YouTube
Fast & Asynchronous in Python - Towards Data Science
Hands-on Python 3 Concurrency With the asyncio Module – Real Python
import asyncio: Learn Python's AsyncIO - YouTube
Python 3 Concurrency – The concurrent.futures Module | The Mouse Vs. The Python ThreadPoolExecutor
An Intro to Threading in Python – Real Python
Speed Up Your Python Program With Concurrency – Real Python
ThreadPoolExecutor | Python Adventures
Python: A quick introduction to the concurrent.futures module | Abu Ashraf Masnun
Python threading and subprocesses explained | InfoWorld
Threading in Python | Linux Journal
python concurrent.futures.ProcessPoolExecutor: Performance of .submit() vs .map() - Stack Overflow
Ultimate Python Multithreading Guide - Programming Geeks Club
Raymond Hettinger, Keynote on Concurrency, PyBay 2017 - YouTube
Raymond Hettinger Keynote — PyBay 2017 Keynote documentation
Threading Tutorial #1 - Concurrency, Threading and Parallelism Explained - YouTube
Threading Tutorial #2 - Implementing Threading in Python 3 (Examples) - YouTube
chryswoods.com | Parallel Programming with Python
Parallel Processing in Python
Simple parallel map in python
Python and fast HTTP clients
Python Parallel Computing (in 60 Seconds or less) – dbader.org
Parallelisation In Python — An Alternative Approach
Here’s how you can get a 2–6x speed-up on your data pre-processing with Python ProcessPoolExecutor
17.2. multiprocessing — Process-based parallelism — Python documentation
multiprocessing — Manage Processes Like Threads — PyMOTW 3
python - Dead simple example of using Multiprocessing Queue, Pool and Locking - Stack Overflow
Multiprocessing in Python | Linux Journal
zeehio/parmap: Easy to use map and starmap python equivalents
easier to use than map(), starmap(), with partial support
Subprocess
subprocess — Subprocess management — Python 3 documentation
Python's os and subprocess Popen Commands
Launching External Processes in Python | Linux Journal
How to Execute Shell Commands with Python - Parametric Thoughts
import subprocess
p1 = subprocess.Popen('ls -a', shell=True, stdin=None,
stdout=subprocess.PIPE, stderr=subprocess.PIPE,
universal_newlines=True)
p2 = subprocess.Popen('sort -r', shell=True, stdin=p1.stdout)
p1.stdout.close()
out, err = p2.communicate() # blocks
# poll
while True:
output = process.stdout.readline()
print(output.strip())
# Do something
return_code = process.poll()
if return_code is not None:
print('RETURN CODE', return_code)
# Process has finished, read rest of the output
for output in process.stdout.readlines():
print(output.strip())
breaksh
full-fledged subprocess replacement
sh documentation
amoffat/sh: Python process launching
Functional
Python3 is more function in that the built-in map(), filter() returns iterator instead of list (as in Python2's itertool)
see functools and itertool
list comprehensions is a syntactic sugar to map()
Comprehending the ‘Comprehensions’ in Python - Towards Data Science
list(map(lambda x: x**2, [1,2,3,4,5]))
[x**2 for x in [1,2,3,4,5]]
list(filter(lambda x: x>2, [1,2,3,4,5]))
[x for x in [1,2,3,4,5] if x>2]
list(map(pow, [2, 3, 4], [10, 11, 12])) # takes N-th element from each seq for each call
mymax = lambda x, y: x if x > y else y
[mymax(x,y) for (x,y) in [(1,4),(8,3),(6,5)]]Partial:
import functools
def log(message, subsystem):
"""Write the contents of `message` to the specified `subsystem`."""
print("%s: %s".format(subsystem, message))
...
server_log = functools.partial(log, subsystem='server')
server_log('Unable to open socket')Don't use class as attribute container:
import functools
def greet(greeting, target):
return "{greet}! {name}".format(greet=greeting, name=target)
hola = functools.partial(greet, "hola")Functional Python | ActiveState ❗!important
Functional Programming HOWTO — Python documentation
Why does Functional Python Programming matters: Interview insights
Python - Functional Programming [Gerardnico]
Functional Programming with Python Python2
PyFunctional by EntilZha
Underscore.py by serkanyersen
Top 3 Python Functions You Don’t Know About (Probably) map(), filter(), reduce()
functools — Higher-order functions and operations on callable objects — Python documentation
functools — Tools for Manipulating Functions — PyMOTW 3
Martin Heinz | Functools - The Power of Higher-Order Functions in Python
PyToolz API Documentation — Toolz documentation
pytoolz/toolz: A functional standard library for Python.
Python 3 - Function Overloading with singledispatch - Mouse Vs Python
Mutable And Immutable Objects | Python For The Lab
Charming Python, Python 2
Charming Python: Functional programming in Python, Part 1
Charming Python: Functional programming in Python, Part 2
Charming Python: Functional programming in Python, Part 3
# Same syntax as PySpark
class FunctionalWrapper(object):
def __init__(self, data):
self.data = data
def map(self, function):
"""Call `map` on the items in `data` using the provided `function`"""
return FunctionalWrapper(map(function, self.data))
def reduce(self, function):
"""Call `reduce` on the items in `data` using the provided `function`"""
return reduce(function, self.data)
def filter(self, function):
"""Call `filter` on the items in `data` using the provided `function`"""
return FunctionalWrapper(filter(function, self.data))
def __eq__(self, other):
return (isinstance(other, self.__class__)
and self.__dict__ == other.__dict__)
def __getattr__(self, name): return getattr(self.data, name)
def __getitem__(self, k): return self.data.__getitem__(k)
def __repr__(self): return 'FunctionalWrapper({0})'.format(repr(self.data))
def __str__(self): return 'FunctionalWrapper({0})'.format(str(self.data))def memoize(f):
cache = {}
def inner(*args, **kwargs):
if args not in cache:
cache[args] = f(*args, **kwargs)
return cache[args]
inner.__name__ = 'memoized_' + f.__name__
return inner
# alternatively
class memoize:
def __init__(self, f):
self.f = f
self.dict = {}
def __call__(self, *args):
if not args in self.dict:
self.dict[args] = self.f(*args)
return self.dict[args]
@memoize
def find_path(source, graph=graph, path=[]):
...
How to make your code faster by using a cache in Python ~ The Python Corner
Tips and Tricks
Deciphering Glyph :: Modularity for Maintenance
7 Easter Eggs in Python. Countless Ways to Entertain Yourself at… | by Eden Au | Towards Data Science
Master the 5 Ways to Use Underscores in Python! | by Eirik Berge | Geek Culture | Jul, 2021 | Medium
5 Tips To Achieve Low Coupling In Your Python Code - YouTube
What are the 10 best features of Python? - Quora
Can you amaze me with a Python trick? - Quora
6 essential Python libraries for Python programming | InfoWorld
15 Python tips and tricks, so you don’t have to look them up on Stack Overflow
Python progression path - From apprentice to guru - Stack Overflow
Code Like a Pythonista: Idiomatic Python
Transforming Code into Beautiful, Idiomatic Python - YouTube
Python Readability Series - Julio Merino
Martin Heinz - Writing More Idiomatic and Pythonic Code
Martin Heinz - Python Tips and Trick, You Haven't Already Seen
Martin Heinz - Python Tips and Trick, You Haven't Already Seen, Part 2
Martin Heinz - Ultimate Setup for Your Next Python Project repo
30 Magical Python Tricks to Write Better Code | by Felix Antony | Towards Data Science
12 Python Tips and Tricks For Writing Better Code | by Pavel Tech | Towards Data Science
8 Advanced Python Tricks Used by Seasoned Programmers
Five Advanced Python Features. Curly brace scopes, autovivification… | by James Briggs | Jul, 2020 | Towards Data Science
The Most Undervalued Standard Python Library - Towards Data Science
Python tricks 101, what every new programmer should know.
Python Application Layouts: A Reference – Real Python
Top Python Tips & Tricks - FinTechExplained - Medium
Martin Heinz - Python Pitfalls - Expecting The Unexpected
Python HOWTOs — Python documentation
PEP8: The Style Guide for Python Code
styleguide | Style guides for Google-originated open-source projects
Popular Python recipes « ActiveState Code
faif/python-patterns
Bash to Python Converter – zwischenzugs
How to Use sorted() and sort() in Python – Real Python
One-liners
Gist:
| python3 -c 'import sys; for line in sys.stdin.read() print(line)'
Powerful Python One-Liners - Python Wiki
Sony Pictures Imageworks - Open Source - PYP (Python Power at the Prompt)
The Pyed Piper (pyp) Tutorial - YouTube
zenlc2000/pyp3: Pyed Piper tool by Toby Rosen at Sony Imageworks converted to Python 3
piep — piep 0.8.0 documentation
ksamuel/Pyped: Let you apply a Python expression to a command output like Perl or Awk would do
Tar Vulnerability
[Python-Dev] tarfile and directory traversal vulnerability
Unpatched 15-year old Python bug allows code execution in 350k projects
Tarfile: Exploiting the World With a 15-Year-Old Vulnerability
Project
Automating Every Aspect of Your Python Project - Towards Data Science
Martin Heinz - Automating Every Aspect of Your Python Project repo
Ultimate Setup for Your Next Python Project - Towards Data Science
Derivatives
vindarel/languages-that-compile-to-python: List of languages that compile to python
virtual machine - Why aren't there other programming languages that compile to Python bytecode? - Software Engineering Stack Exchange Python bytecode format is not stable
Security
PyCQA/bandit: Bandit is a tool designed to find common security issues in Python code.
#perfmatters
PythonSpeed - Python Wiki
Ship better Python software, faster
Writing Faster Python 3 - presented by Sebastian Witowski - YouTube code
#Writing Faster Python - Sebastian Witowski
10 hard-core coding tips for faster Python | InfoWorld
A Python Optimization Anecdote | Dropbox Tech Blog
Workshop: Advanced Python Software Development
Deciphering Glyph :: You Should Compile Your Python And Here’s Why
Making Python Programs Blazingly Fast - Towards Data Science
Ten Tricks To Speed Up Your Python Codes - Towards Data Science
10 Techniques to Speed Up Python Efficiency | Towards Data Science | Towards Data Science
Python mmap: Improved File I/O With Memory Mapping – Real Python
Fastest Way to Flatten a List in Python - Chris Conlan
Loops in Python – comparison, and performance - DEV Community 👩💻👨💻
IPython Cookbook - Chapter 4 : Profiling and Optimization
IPython Cookbook - Chapter 5 : High-Performance Computing
Profiling in Python: How to Find Performance Bottlenecks – Real Python
Python 3.12 Preview: Support For the Linux perf Profiler – Real Python ❗!important
Python decorator to measure the execution time of methods
Faster CPython
official project to make CPython fast
Guido And Microsoft Want To Make Python x2 Faster
Faster CPython at PyCon, part one [LWN.net]
Faster CPython at PyCon, part two [LWN.net]
faster-cpython · GitHub
How we are making Python 3.11 faster - presented by Mark Shannon - YouTube
Making Python 5x FASTER with Guido van Rossum and Mark Shannon - Talk Python To Me - YouTube
Talks - Mark Shannon: How we are making CPython faster. Past, present and future. - YouTube
Talks - Brandt Bucher: Inside CPython 3.11's new specializing, adaptive interpreter - YouTube quickening (specialized instructions in JIT compiler)
brandtbucher/specialist: Visualize CPython 3.11's specializing, adaptive interpreter. :fire:
PEP 659 – Specializing Adaptive Interpreter | peps.python.org
Runtime
6 Python interpreters to try in 2022 | Opensource.com
PyPy runtime with JIT
What is PyPy? Faster Python without pain | InfoWorld
PyPy Status Blog: PyPy for low-latency systems
Experiments with new low latency PyPy garbage collector in a thread.
Run Your Python Code as Fast as C | by Marcel Moosbrugger | Apr, 2021 | Towards Data Science
facebookincubator/cinder: Instagram's performance oriented fork of CPython.
Using IPython for parallel computing — ipyparallel dev documentation
PyParallel experimental fork of Python 3 that removes GIL
Pythran — Pythran documentation
serge-sans-paille/pythran: a claimless python to c++ converter
Pyjion - A Python JIT Compiler
Pyjion main documentation
tonybaloney/Pyjion: Pyjion - A JIT for Python based upon CoreCLR
Pyston | Python Performance
pyston/pyston: A faster and highly-compatible implementation of the Python programming language.
only supports 2.7 and lacked the performance improvements originally intended, abandoned by Dropbox; forked from 3.8 in 2021
Pyston 2.2 Goes Open Source Again 2021-05
Programming languages: 'Faster Python' Pyston takes a step forward | ZDNet
Pyston To Merge With CPython?
Compiler
Nuitka the Python Compiler — Nuitka the Python Compiler documentation
kayhayen/Nuitka: Official mirror of Nuitka as from http://nuitka.net
Welcome to mypyc documentation! — mypyc documentation use type hints
mypyc/mypyc: Compile type annotated Python to fast C extensions
pip install mypy
mypyc .../module.py
python -c "import module"Exaloop
Welcome to Codon - Codon
exaloop/codon: A high-performance, zero-overhead, extensible Python compiler using LLVM
Python-based compiler achieves orders-of-magnitude speedups | MIT News | Massachusetts Institute of Technology
C-binding
Cython: C-Extensions for Python compiled Python C, sacrificing Python features
Cython Offers the Ease of Python, the Speed of C++ - The New Stack
Use Cython to get more than 30X speedup on your Python code
An Introduction to Just Enough Cython to be Useful | Peter Baumgartner
An Introduction to Cython, the Secret Python Extension with Superpowers
PEP 579 – Refactoring C functions and methods | peps.python.org
Buffer Protocol — Python documentation
Iterator Protocol — Python documentation
Argument Clinic How-To — Python documentation preprocessor for CPython C files
Python/C API Reference Manual — Python 3 documentation
CFFI documentation better than Ctypes in standard library
Building a Python C Extension Module – Real Python
Interfacing Python and C: The CFFI Module – dbader.org
Extending Python With C Libraries and the “ctypes” Module – dbader.org
Interfacing Python and C: Advanced “ctypes” Features – dbader.org
Cython tutorial: How to speed up Python | InfoWorld
Importing with ctypes in Python: fighting overflows – Python Tips
Python Bindings: Calling C or C++ From Python – Real Python
Intro — pybind11 documentation
pybind/pybind11: Seamless operability between C++11 and Python
Ivan Smirnov - pybind11 - seamless operability between C++11 and Python - YouTube
Python wrappers for C++ with pybind11 — LSST DM Developer Guide Current documentation
C++ in Python the Easy Way! - YouTube easier than Boost.Python
Boost.Python
Using C++ with Python 3 in 2018 - Keith Whitley - Medium build with Docker
Riverbank | Software | SIP | What is SIP?
SIP Reference Guide
Rust binding
Introduction - PyO3 user guide
Introduction - PyO3 user guide
PyO3/pyo3: Rust bindings for the Python interpreter
Writing Python Extensions In Rust Using PyO3
Rust in Python: Fixing Regular Expressions - YouTube
Speed up your Python using Rust - Red Hat Developer
How to use Rust with Python, and Python with Rust | InfoWorld
Why you should use Python and Rust together | Opensource.com
dgrunwald/rust-cpython: Rust <-> Python bindings
getsentry/milksnake: A setuptools/wheel/cffi extension to embed a binary data in wheels
Evolving Our Rust With Milksnake | Product Blog • Sentry
PyO3/setuptools-rust: Setuptools plugin for Rust support
vigneshwer dhinakaran - Pumping up Python modules using Rust - PyCon 2018 - YouTube slide
Go binding
Python and Go : Part I - gRPC
Python and Go : Part II - Extending Python With Go
Python and Go : Part III - Packaging Python Code
Python and Go : Part IV - Using Python in Memory
profiling
Python Timer Functions: Three Ways to Monitor Your Code – Real Python
import codetiming
with codetiming.Timer(text="item: {:0.4f} s") as timer:
# code to be profiled
...start = time.perf_counter()
# code to be profiled
elapsed = time.perf_counter() - startPython 101: An Intro to Benchmarking your code - The Mouse Vs. The Python
Python 102: How to Profile Your Code - The Mouse Vs. The Python
The Python Profilers — Python documentation
Profiling Python Code » ADMIN Magazine
Profiling python code · The COOP Blog
Not just CPU: writing custom profilers for Python
python -m cProfile filename
python3.7 -X importtime import_typing.py
perf stat -r 1000 python3.7 import_typing.pyimport cProfile, pstats, StringIO
pr = cProfile.Profile()
pr.enable()
# ... do something ...
pr.disable()
s = StringIO.StringIO()
ps = pstats.Stats(pr, stream=s).sort_stats(‘cumulative’)
ps.print_stats()
print s.getvalue()
# or run single statement
cProfile.run('sum([i * 2 for i in range(10000)])')
cProfile.run('sum((i * 2 for i in range(10000)))')Diagnose slow Python code. (Feat. async/await) - YouTube
# 3.8+
import cProfile
import pstats
with cProfile.Profile() as profiler:
# code to be profiled
...
stats = pstats.Stats(profiler)
stats.sort_stats(pstats.SortKey.TIME)
stats.print_stats()
stats.dump_stats(filename='output.prof')
# snakeviz output.proftimeit — Measure execution time of small code snippets — Python documentation
# https://dbader.org/blog/python-reverse-string
>>> import timeit
>>> s = 'abcdefghijklmnopqrstuvwxyz' * 10
>>> timeit.repeat(lambda: reverse_string1(s))
[0.6848115339962533, 0.7366074129968183, 0.7358982900041156]
>>> timeit.repeat(lambda: reverse_string2(s))
[5.514941683999496, 5.339547180992668, 5.319950777004124]
>>> timeit.repeat(lambda: reverse_string3(s))
[48.74324739299482, 48.637329410004895, 49.223478018000606]nschloe/perfplot: Performance plots for Python code
rkern/line_profiler: Line-by-line profiling for Python
The Python Performance Benchmark Suite — Python Performance Benchmark Suite documentation
Python pyperf module — pyperf documentation
benfred/py-spy: Sampling profiler for Python programs
An awesome new Python profiler: py-spy! - Julia Evans
Profiling With Cpu Sampling And Speedscope
profilehooks - Decorators for profiling Python functions
Timing Functions With Decorators – Real Python
Python Timer Functions: Three Ways to Monitor Your Code – Real Python
pythonprofilers/memory_profiler: Monitor Memory usage of Python code
# ipython
%load_ext memory_profiler
%memit sum([n*n for n in range(1_000_000)])
%memit sum(n*n for n in range(1_000_000))Fil: A memory profiler for Python
Reduce your Python program’s memory usage with Fil