Spark
1 min read
spark
Written in 2015. Tools, APIs and prices may have changed since.
Lab note What I wrote down while learning this by building with it. Short by design; the code is in pfeilbr/spark-playground.
Project to learn and experiment with the Spark cluster computing system.
Setup
export SPARK_HOME=/path/to/spark-2.2.0-bin-hadoop2.7
export PATH=$SPARK_HOME/bin:$PATH
Running an example
$ spark-submit dataframes-and-sql-examples.py
Docker
$ docker run -i -t -P --name spark -v /path/to/spark-playground:/src sequenceiq/spark:latest bash