Scala Big Data

23 Big Data packages and projects

• Apache Spark

  10.0 9.9 Scala

  Big data platform

• Kafka

  10.0 9.7 L2 Java

  Kafka is a message broker project and aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds.

• Deeplearning4J

  9.9 9.7 L1 Java

  Deep Learning for Java, Scala & Clojure on Hadoop & Spark - From Skymind

• Flink

  9.9 9.9 L2 Java

  Processing framework with powerful stream- and batch-processing capabilities.

• Scalding

  9.6 7.1 Scala

  A Scala binding for the Cascading abstraction of Hadoop MapReduce.

• Summingbird

  9.3 3.6 Scala

  An implementation of the “lambda architecture” as a software abstraction

• Scio

  9.0 9.5 Scala

  A Scala API for Apache Beam and Google Cloud Dataflow

• Reactive-kafka

  8.9 9.1 Scala

  Reactive Streams API for Apache Kafka.

• BIDMach

  8.3 8.6 Jupyter Notebook

  CPU and GPU machine learning library, using JNI for GPU computation.

• Jupyter Scala

  8.3 9.0 Scala

  Lightweight Scala kernel for Jupyter / IPython 3

• Gearpump

  8.0 7.0 Scala

  Lightweight real-time big data streaming engine

• Sparkta

  7.9 0.0 Scala

  Real Time Aggregation based on Spark Streaming.

• Hail

  7.3 9.8 Scala

  Scalable genetic data analysis

• Vegas

  7.0 3.8 Scala

  The missing MatPlotLib for Scala + Spark

• Scoobi

  6.9 0.0 Scala

  Write type-safe Hadoop programs in idiomatic Scala way

• Scrunch

  4.8 4.3 L3 Java

  A Scala wrapper for Apache Crunch which provides a framework for writing, testing, and running MapReduce pipelines.

• Scoozie

  4.5 0.0 Scala

  Scala DSL on top of Oozie XML

• metorikku

  3.5 8.5 Scala

  A simplified, lightweight ELT Framework based on Apache Spark

• spark-deployer

  3.2 0.0 Scala

  A sbt plugin which helps deploying Apache Spark stand-alone cluster and submitting job on cloud system like AWS EC2.

• raster-frames

  1.7 0.0 Scala

  Spark DataFrames for earth observation data

• GridScale

  1.5 6.1 Java

  A Scala API for computing clusters and grids.

• Shadoop

  1.4 0.0 Scala

  A Scala DSL for Hadoop MapReduce.

• Sparkplug

  1.2 0.0 Scala

  Spark package to "plug" holes in data using SQL based rules

* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest. Visit our partner's website for more details.

Datadog APM: Distributed tracing and real-time app performance metrics to drill down into any service, endpoint, or individual user. More from our partner

Popular Comparisons

• Apache Spark
  vs
  Scalding
• Summingbird
  vs
  Apache Spark
• Gearpump
  vs
  Flink
• Apache Spark
  vs
  Gearpump
• Gearpump
  vs
  Kafka

SaaSHub - The independent software marketplace. Find the best software and alternatives. More from our team