GridScale alternatives and similar packages
Based on the "Big Data" category.
Alternatively, view gridscale alternatives based on common mentions on social networks and blogs.
-
Kafka
Kafka is a message broker project and aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds. -
Scrunch
A Scala wrapper for Apache Crunch which provides a framework for writing, testing, and running MapReduce pipelines. -
spark-deployer
A sbt plugin which helps deploying Apache Spark stand-alone cluster and submitting job on cloud system like AWS EC2. -
Schemer
Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API. -
Spark Utils
Basic framework utilities to quickly start writing production ready Apache Spark applications
Get performance insights in less than 4 minutes.
* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest. Visit our partner's website for more details.
Do you think we are missing an alternative of GridScale or a related project?
Popular Comparisons
README
GridScale
GridScale is a scala library for accessing various file and batch systems. For the time being it supports:
- Remote SSH servers,
- PBS clusters,
- SLURM clusters,
- SGE clusters,
- OAR clusters,
- Condor flocks
- HTTP file lists,
- IPFS nodes,
- EGI computing grid via DIRAC pilot jobs system.
Licence
GridScale is licensed under the [GNU Affero GPLv3](LICENSE.md) software license.
Build
GridScale builds with sbt
.
Use the compile
and/or package
task to build all the modules.
Imports
In order to use gridscale you should import the namespace corresponding to the job system you want to use:
import gridscale.pbs._
SBT
GridScale is cross compiled against serveral versions of scala. To use on of its modules add a dependency like:
libraryDependencies += "fr.iscpif.gridscale" %% "pbs" % version
Examples
Up to date examples are available in the [example directory](examples/).
Development
GridScale can be generated locally using:
sbt publish-local
To release in one step, use:
sbt 'release with-defaults'
*Note that all licence references and agreements mentioned in the GridScale README section above
are relevant to that project's source code only.