Tuning Spark Streaming

Jeroen van Wilgenburg has an excellent blog post on Understanding Spark parameters – A step by step guide to tune your Spark job which provides best practices around dealing with an optimal Spark Streaming setup (receiver, batch size).…

Read more

Setting up a Spark cluster on Google Compute

Ido Green (Developer Advocate at Google) has written about how to set up a Spark cluster on GCE manually already end of last year, however now he has made a provisioning script available via GitHub; read more at: http://greenido.wordpress.com/2014/05/13/spark-cluster-on-google-compute-engine/…

Read more
Proudly published with Ghost