DevOps

Listed by section, in editorial order. Click a column to re-sort the whole list.

Press / to search. Tap a tag to filter. Click any row for details.

Search and filter

Results

Row number Tags

DevOps Tools 16 projects

Server setup goes in YAML playbooks that Ansible runs, unless you'd rather write Python with pyinfra. Neither Python DevOps tool needs an agent: SSH is enough.
Python interface to Amazon Web Services.
boto/github.com/boto/boto3 / /2,490,204,445 downloads/month
Universal Command Line Interface for Amazon Web Services; the PyPI package is v1, in maintenance mode, while v2 ships as AWS's bundled installer.
aws/github.com/aws/aws-cli / /86,935,177 downloads/month
Microsoft Azure SDK for Python, published as per-service packages.
Azure/github.com/Azure/azure-sdk-for-python /
Google Cloud client libraries for Python, published as per-service packages.
googleapis/github.com/googleapis/google-cloud-python /
A radically simple IT automation platform.
ansible/github.com/ansible/ansible / /8,083,011 downloads/month
A multi-distribution package that handles early initialization of a cloud instance.
canonical/github.com/canonical/cloud-init /
Turns Python code into shell commands and runs them on your servers.
pyinfra-dev/github.com/pyinfra-dev/pyinfra / /221,135 downloads/month
Infrastructure automation and management system.
saltstack/github.com/saltstack/salt / /134,501 downloads/month
A simple, Pythonic tool for remote execution and deployment.
fabric/github.com/fabric/fabric / /7,658,678 downloads/month
A Python serverless microframework for AWS.
aws/github.com/aws/chalice / /615,903 downloads/month
A cross-platform process and system utilities module.
giampaolo/github.com/giampaolo/psutil / /301,427,100 downloads/month
Sentry SDK for Python.
getsentry/github.com/getsentry/sentry-python / /134,431,474 downloads/month
Supervisor process control system for UNIX.
Supervisor/github.com/Supervisor/supervisor / /7,736,192 downloads/month
A real-time monitor and web admin for Celery task queues.
mher/github.com/mher/flower / /8,465,484 downloads/month
A full-fledged subprocess replacement for Python.
amoffat/github.com/amoffat/sh / /6,887,475 downloads/month
A deduplicating archiver with compression and encryption.
borgbackup/github.com/borgbackup/borg / /106,555 downloads/month

Distributed Computing 5 projects

From a laptop to a cluster, Python distributed computing comes down to the workload: joblib runs loops, Dask scales pandas, Ray scales ML, and PySpark runs SQL.
Apache Spark Python API.
apache/github.com/apache/spark / /48,973,685 downloads/month
A flexible parallel computing library for analytic computing.
dask/github.com/dask/dask / /27,725,418 downloads/month
A unified framework for scaling AI and Python applications.
ray-project/github.com/ray-project/ray/ / /14,355,446 downloads/month
Parallel computing and disk-based caching for Python functions.
joblib/github.com/joblib/joblib / /164,976,416 downloads/month
Python bindings for MPI.
mpi4py/github.com/mpi4py/mpi4py / /577,803 downloads/month

Task Queues 5 projects

Two questions pick a Python task queue: is your app async, and what's your broker? Taskiq is for asyncio. Celery takes RabbitMQ or Redis, RQ Redis, Huey SQLite.
An asynchronous task queue/job queue based on distributed message passing.
celery/github.com/celery/celery / /45,393,499 downloads/month
Simple job queues for Python.
rq/github.com/rq/rq / /9,459,384 downloads/month
A fast and reliable background task processing library for Python 3.
Bogdanp/github.com/Bogdanp/dramatiq / /1,765,935 downloads/month
A little task queue with multi-process, multi-thread, or greenlet workers.
coleifer/github.com/coleifer/huey / /13,204,437 downloads/month
Distributed task queue with native asyncio support and pluggable brokers.
taskiq-python/github.com/taskiq-python/taskiq / /2,164,630 downloads/month

Messaging 4 projects

Instead of a Python messaging library per broker, an asyncio service can run on FastStream. Otherwise, use confluent-kafka, pika, or paho-mqtt.
Confluent's Python client for Apache Kafka, built on librdkafka.
confluentinc/github.com/confluentinc/confluent-kafka-python / /42,547,647 downloads/month
Pure-Python RabbitMQ/AMQP 0-9-1 client library.
pika/github.com/pika/pika / /11,385,684 downloads/month
The Eclipse Paho MQTT client for Python.
eclipse-paho/github.com/eclipse-paho/paho.mqtt.python / /7,959,182 downloads/month
A framework for building asynchronous services over Apache Kafka, RabbitMQ, NATS, MQTT and Redis.
ag2ai/github.com/ag2ai/faststream / /1,308,459 downloads/month

Job Schedulers 5 projects

A function that runs every hour fits in APScheduler inside your app. A pipeline of tasks outgrows a Python scheduler and needs Airflow, Prefect, or Dagster.
A light but powerful in-process task scheduler that lets you schedule functions.
agronholm/github.com/agronholm/apscheduler / /35,830,696 downloads/month
Python job scheduling for humans.
dbader/github.com/dbader/schedule / /5,874,734 downloads/month
An orchestration platform for the development, production, and observation of data assets.
dagster-io/github.com/dagster-io/dagster / /8,388,912 downloads/month
Airflow is a platform to programmatically author, schedule and monitor workflows.
apache/github.com/apache/airflow / /7,298,317 downloads/month
A modern workflow orchestration framework that makes it easy to build, schedule and monitor robust data pipelines.
PrefectHQ/github.com/PrefectHQ/prefect / /7,289,286 downloads/month

Logging 4 projects

Keep the built-in logging module until you need what a Python logging library adds: structlog's key-value logs in JSON, or Loguru's logger, ready on import.
(Python standard library) Logging facility for Python.
python/docs.python.org/3/library/logging.html /
Structured logging made easy.
hynek/github.com/hynek/structlog / /99,487,005 downloads/month
Library which aims to bring enjoyable logging in Python.
Delgan/github.com/Delgan/loguru / /68,647,796 downloads/month
The observability platform for Python, from the makers of Pydantic.
pydantic/github.com/pydantic/logfire / /13,829,087 downloads/month

Network Virtualization 3 projects

Netmiko is where to start with Python network automation libraries; when many vendors need one API, move to NAPALM. Scapy works a layer down, on raw packets.
A brilliant packet manipulation library.
secdev/github.com/secdev/scapy / /3,378,020 downloads/month
Multi-vendor library to simplify CLI connections to network devices.
ktbyers/github.com/ktbyers/netmiko / /965,848 downloads/month
Cross-vendor API to manipulate network devices.
napalm-automation/github.com/napalm-automation/napalm / /122,802 downloads/month

Contribute

Know a project that belongs here?

Tell us what it does and why it stands out.

Submit a project Star the repository