PySpark

Python API for Apache Spark, a distributed processing framework for big data analysis and machine learning on clusters.

Part of Apache Spark

Directed Acyclic Graph (DAG)

Interlinked with SQL queriers

Tutorial

Can run local.

Similar to Pandas