DevOps

Listed by section, in editorial order. Click a column header to re-sort the whole list.

Press / to search. Tags open their category. Click a row for details.

Search and filter

Results

Row number Tags

DevOps Tools 16 projects

Server setup goes in YAML playbooks that Ansible runs, unless you'd rather write Python with pyinfra. Neither Python DevOps tool needs an agent: SSH is enough.
Python interface to Amazon Web Services.
boto/github.com/boto/boto3 /9,924 stars /
Universal Command Line Interface for Amazon Web Services; the PyPI package is v1, in maintenance mode, while v2 ships as AWS's bundled installer.
aws/github.com/aws/aws-cli /17,334 stars /
Microsoft Azure SDK for Python, published as per-service packages.
Azure/github.com/Azure/azure-sdk-for-python /5,624 stars /
Google Cloud client libraries for Python, published as per-service packages.
googleapis/github.com/googleapis/google-cloud-python /5,414 stars /
A radically simple IT automation platform.
ansible/github.com/ansible/ansible /71,158 stars /
A multi-distribution package that handles early initialization of a cloud instance.
canonical/github.com/canonical/cloud-init /3,847 stars /
Turns Python code into shell commands and runs them on your servers.
pyinfra-dev/github.com/pyinfra-dev/pyinfra /6,059 stars /
Infrastructure automation and management system.
saltstack/github.com/saltstack/salt /15,727 stars /
A simple, Pythonic tool for remote execution and deployment.
fabric/github.com/fabric/fabric /15,520 stars /
A Python serverless microframework for AWS.
aws/github.com/aws/chalice /11,059 stars /
A cross-platform process and system utilities module.
giampaolo/github.com/giampaolo/psutil /11,311 stars /
Sentry SDK for Python.
getsentry/github.com/getsentry/sentry-python /2,218 stars /
Supervisor process control system for UNIX.
Supervisor/github.com/Supervisor/supervisor /9,154 stars /
A real-time monitor and web admin for Celery task queues.
mher/github.com/mher/flower /7,239 stars /
A full-fledged subprocess replacement for Python.
amoffat/github.com/amoffat/sh /7,247 stars /
A deduplicating archiver with compression and encryption.
borgbackup/github.com/borgbackup/borg /13,892 stars /

Distributed Computing 5 projects

From a laptop to a cluster, Python distributed computing comes down to the workload: joblib runs loops, Dask scales pandas, Ray scales ML, and PySpark runs SQL.
Apache Spark Python API.
apache/github.com/apache/spark /44,365 stars /
A flexible parallel computing library for analytic computing.
dask/github.com/dask/dask /13,937 stars /
A unified framework for scaling AI and Python applications.
ray-project/github.com/ray-project/ray /44,155 stars /
Parallel computing and disk-based caching for Python functions.
joblib/github.com/joblib/joblib /4,414 stars /
Python bindings for MPI.
mpi4py/github.com/mpi4py/mpi4py /933 stars /

Task Queues 5 projects

Two questions pick a Python task queue: is your app async, and what's your broker? Taskiq is for asyncio. Celery takes RabbitMQ or Redis, RQ Redis, Huey SQLite.
An asynchronous task queue/job queue based on distributed message passing.
celery/github.com/celery/celery /29,091 stars /
Simple job queues for Python.
rq/github.com/rq/rq /10,698 stars /
A fast and reliable background task processing library for Python 3.
Bogdanp/github.com/Bogdanp/dramatiq /5,335 stars /
A little task queue with multi-process, multi-thread, or greenlet workers.
coleifer/github.com/coleifer/huey /6,057 stars /
Distributed task queue with native asyncio support and pluggable brokers.
taskiq-python/github.com/taskiq-python/taskiq /2,369 stars /

Messaging 4 projects

Instead of a Python messaging library per broker, an asyncio service can run on FastStream. Otherwise, use confluent-kafka, pika, or paho-mqtt.
Confluent's Python client for Apache Kafka, built on librdkafka.
confluentinc/github.com/confluentinc/confluent-kafka-python /526 stars /
Pure-Python RabbitMQ/AMQP 0-9-1 client library.
pika/github.com/pika/pika /3,890 stars /
The Eclipse Paho MQTT client for Python.
eclipse-paho/github.com/eclipse-paho/paho.mqtt.python /2,432 stars /
A framework for building async services over Kafka, RabbitMQ, NATS, MQTT, and Redis.
ag2ai/github.com/ag2ai/faststream /5,387 stars /

Job Schedulers 6 projects

A function that runs every hour fits in APScheduler inside your app. A pipeline of tasks outgrows a Python scheduler and needs Airflow, Prefect, or Dagster.
A light but powerful in-process task scheduler that lets you schedule functions.
agronholm/github.com/agronholm/apscheduler /7,675 stars /
Python job scheduling for humans.
dbader/github.com/dbader/schedule /12,269 stars /
Temporal SDK for durable, fault-tolerant workflows that survive crashes and retries.
temporalio/github.com/temporalio/sdk-python /1,240 stars /
An orchestration platform for the development, production, and observation of data assets.
dagster-io/github.com/dagster-io/dagster /16,350 stars /
Airflow is a platform to programmatically author, schedule and monitor workflows.
apache/github.com/apache/airflow /47,311 stars /
A modern workflow orchestration framework that makes it easy to build, schedule and monitor robust data pipelines.
PrefectHQ/github.com/PrefectHQ/prefect /24,099 stars /

Logging 4 projects

Keep the built-in logging module until you need what a Python logging library adds: structlog's key-value logs in JSON, or Loguru's logger, ready on import.
(Python standard library) Logging facility for Python.
python/docs.python.org/3/library/logging.html /77,691 stars /
Structured logging made easy.
hynek/github.com/hynek/structlog /4,982 stars /
A library that aims to bring enjoyable logging to Python.
Delgan/github.com/Delgan/loguru /24,156 stars /
The observability platform for Python, from the makers of Pydantic.
pydantic/github.com/pydantic/logfire /4,527 stars /

Network Virtualization 3 projects

Netmiko is where to start with Python network automation libraries; when many vendors need one API, move to NAPALM. Scapy works a layer down, on raw packets.
A brilliant packet manipulation library.
secdev/github.com/secdev/scapy /12,659 stars /
Multi-vendor library to simplify CLI connections to network devices.
ktbyers/github.com/ktbyers/netmiko /4,312 stars /
Cross-vendor API to manipulate network devices.
napalm-automation/github.com/napalm-automation/napalm /2,519 stars /

Contribute

Know a project that belongs here?

Tell us what it does and why it stands out.

Submit a project Star the repository