Scala Big Data

26 Big Data packages and projects

• Apache Spark

  10.0 10.0 Scala

  Apache Spark - A unified analytics engine for large-scale data processing
  

• Kafka

  10.0 9.8 L2 Java

  Mirror of Apache Kafka
  

Static code analysis for 29 languages.

Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

Promo www.sonarqube.org



• Flink

  9.9 10.0 L2 Java

  Apache Flink
  

• Deeplearning4J

  9.9 9.9 L1 Java

  Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.
  

• Scalding

  9.6 5.6 Scala

  A Scala API for Cascading
  

• Summingbird

  9.3 0.0 Scala

  Streaming MapReduce with Scalding and Storm
  

• Scio

  9.3 9.5 Scala

  A Scala API for Apache Beam and Google Cloud Dataflow.
  

• Reactive-kafka

  8.9 8.3 Scala

  Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.
  

• Jupyter Scala

  8.7 7.5 Scala

  A Scala kernel for Jupyter
  

• BIDMach

  8.3 0.0 Scala

  CPU and GPU-accelerated Machine Learning Library
  

• Gearpump

  8.1 0.0 Scala

  Lightweight real-time big data streaming engine over Akka
  

• Sparkta

  8.1 0.0 Scala

  Real Time Analytics and Data Pipelines based on Spark Streaming
  

• Hail

  8.1 9.8 Python

  Scalable genomic data analysis.
  

• Vegas

  7.6 0.0 Scala

  The missing MatPlotLib for Scala + Spark
  

• metorikku

  7.0 8.4 Scala

  A simplified, lightweight ETL Framework based on Apache Spark
  

• Scoobi

  6.9 0.0 Scala

  A Scala productivity framework for Hadoop.
  

• Scrunch

  5.1 1.4 L3 Java

  Mirror of Apache Crunch (Incubating)
  

• Scoozie

  4.7 0.0 Scala

  Scala DSL on top of Oozie XML
  

• Schemer

  3.5 0.0 Scala

  Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.
  

• spark-deployer

  3.4 0.0 Scala

  Deploy Spark cluster in an easy way.
  

• raster-frames

  2.1 0.0 Scala

  Spark DataFrames for earth observation data

• GridScale

  2.0 4.9 Scala

  Scala library for accessing various file, batch systems, job schedulers and grid middlewares.
  

• Sparkplug

  1.9 0.0 Scala

  Spark package to "plug" holes in data using SQL based rules ⚡️ 🔌
  

• Spark Utils

  1.9 5.1 Scala

  Basic framework utilities to quickly start writing production ready Apache Spark applications

• Shadoop

  1.3 0.0 Scala

  A wrapper for Hadoop in Scala

• Spark Tools

  1.1 0.0 Scala

  Executable Apache Spark Tools: Format Converter & SQL Processor

* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest.

Less time debugging, more time building sponsored scoutapm.com

Popular Comparisons

• Apache Spark
  vs
  Scalding
• Summingbird
  vs
  Apache Spark
• Flink
  vs
  Gearpump
• Gearpump
  vs
  Apache Spark
• Gearpump
  vs
  Kafka

OPS - Build and Run Open Source Unikernels sponsored github.com/nanovms