Spark Packages

The folks from Databricks have launched Spark Packages, a community site hosting modules that are not (directly) part of the Apache Spark project.

At time of writing the site contains 16 packages including stuff like launch scripts (GCE, Azure, etc.), integrations (for Kafka, Avro, etc.), utils (testing, RDDs, etc.) and extensions such as Spork (Pig on Spark).

You can contribute to this community site via GitHub: you first sign in via your GitHub account, and then you can register a package by selecting a public GitHub repo under your account.

Happy contributing!

comments powered by Disqus
Proudly published with Ghost

Latest posts