Scalding v0.12.0 Release Notes
-
- ๐ Fix long compile time for MultiJoin helpers: https://github.com/twitter/scalding/pull/1109
- ๐ Allows reducer estimation to operate on all hfs taps: https://github.com/twitter/scalding/pull/1080
- ๐ Fix bufferedTake: https://github.com/twitter/scalding/pull/1107
- Generate methods for flattening the results of many joins: https://github.com/twitter/scalding/pull/1097
- ๐ง Make TimePathedSource more configurable: https://github.com/twitter/scalding/pull/1105
- โ Adding DailyPrefixSuffixLzoTsv: https://github.com/twitter/scalding/pull/1082
- Option to select the fields for output in templatesource: https://github.com/twitter/scalding/pull/1061
- โ Add a DailySuffixMostRecentLzoProtobuf source: https://github.com/twitter/scalding/pull/1104
- โก๏ธ Updates default scala version to 2.10.4: https://github.com/twitter/scalding/pull/1081
- MultiSourceTap hashcode: https://github.com/twitter/scalding/pull/1101
- ๐ scalding-core: merge flow step strategies to allow reducer estimation combined with other strategies: https://github.com/twitter/scalding/pull/1094
- ๐ Improve command line handling of the execution app: https://github.com/twitter/scalding/pull/1083
- โ More testing around the globifier with new properties: https://github.com/twitter/scalding/pull/1092
- ๐จ Refactor JDBCSource to add compile-time info about type of DB: https://github.com/twitter/scalding/pull/1087
- โ Add a cumulative sum to KeyedList: https://github.com/twitter/scalding/pull/1085
- โ Add in failing test case: https://github.com/twitter/scalding/pull/1090
- โ Adds ability to also get the mode inside the Execution monad.: https://github.com/twitter/scalding/pull/1088
- Enforce invariant: mapGroup iterators all nonempty: https://github.com/twitter/scalding/pull/1072
- ๐ Allow PartitionSource to limit the number of open files: https://github.com/twitter/scalding/pull/1078
- append to Cascading frameworks system property instead of setting it directly: https://github.com/twitter/scalding/pull/1076
- โ Adds some output while assembly is building to keep travis happy: https://github.com/twitter/scalding/pull/1084
- Only request necessary hadoop configs in hraven reducer estimator: https://github.com/twitter/scalding/pull/1067
- โ Add parquet-scrooge sources: https://github.com/twitter/scalding/pull/1064
- Outer join handles case when both are empty: https://github.com/twitter/scalding/pull/1065
- ๐ Fix race in merging: https://github.com/twitter/scalding/pull/1063
- โ Add support for column projection to parquet sources: https://github.com/twitter/scalding/pull/1056
- โ Add typed version of RichPipe 'using': https://github.com/twitter/scalding/pull/1049
- โ Add getExecution/getOrElseExecution: https://github.com/twitter/scalding/pull/1062
- ๐ Change toIteratorExecution to toIterableExecution: https://github.com/twitter/scalding/pull/1058
- Cache Execution evaluations: https://github.com/twitter/scalding/pull/1057
- โ Add support for push down filters in parquet sources: https://github.com/twitter/scalding/pull/1050
- โ Add support for Fold: https://github.com/twitter/scalding/pull/1053
- ๐ move to use JobConf(true) for hadoop crazyness that causes host not foun...: https://github.com/twitter/scalding/pull/1051
- โก๏ธ Disable Cascading update check.: https://github.com/twitter/scalding/pull/1048
- ๐ท Respects -Dmapred.job.name when passed in on the command line: https://github.com/twitter/scalding/pull/1045
- โ Add some instances from Algebird: https://github.com/twitter/scalding/pull/1039
- ๐ Fix join.mapGroup issue: https://github.com/twitter/scalding/pull/1038
- โ Add a defensive .forceToDisk in Sketched: https://github.com/twitter/scalding/pull/1035
- Override toIterator for all Mappable with transformForRead: https://github.com/twitter/scalding/pull/1034
- ๐ Make sinkFields in TypedDelimited final.: https://github.com/twitter/scalding/pull/1032
- ๐ Fixed type of exception thrown by validateTaps: https://github.com/twitter/scalding/pull/1033
- โ Add default local maven repo to the resolver list: https://github.com/twitter/scalding/pull/1024
- โ Add an ExecutionApp trait for objects to skip the Job class: https://github.com/twitter/scalding/pull/1027
- ๐ Make each head pipe have a unique name: https://github.com/twitter/scalding/pull/1025
- โ Run REPL from SBT: https://github.com/twitter/scalding/pull/1021
- โ Add Config to openForRead: https://github.com/twitter/scalding/pull/1023
- ๐ Fix replConfig merging and evaluate values in Config.fromHadoop: https://github.com/twitter/scalding/pull/1015
- REPL Autoload file: https://github.com/twitter/scalding/pull/1009
- ๐ Fix hRaven Reducer Estimator: https://github.com/twitter/scalding/pull/1018
- โก๏ธ Update Cascading JDBC Version.: https://github.com/twitter/scalding/pull/1016
- ๐ Some Execution fixes: https://github.com/twitter/scalding/pull/1007
- ๐จ Refactor InputSizeReducerEstimator to correctly unroll MultiSourceTaps: https://github.com/twitter/scalding/pull/1017
- ๐ Fix issue #1011: Building develop branch fails: https://github.com/twitter/scalding/pull/1012
- hRaven Reducer Estimator: https://github.com/twitter/scalding/pull/996
- JsonLine should handle empty lines: https://github.com/twitter/scalding/pull/966
- โ Add comments for memory-related reduce operations.: https://github.com/twitter/scalding/pull/1006
- โ Add the remaining odds and ends to Execution[T]: https://github.com/twitter/scalding/pull/985
- ๐ Fix up the tests to run forked, and split across lots of travis builds: https://github.com/twitter/scalding/pull/993
- Typedpipe partition: https://github.com/twitter/scalding/pull/987
- ๐ Fix toIterator bug (#988): https://github.com/twitter/scalding/pull/990
- ๐ Basic reducer estimator support: https://github.com/twitter/scalding/pull/973
- ๐ Improve TypedSimilarity algorithm and update test.: https://github.com/twitter/scalding/pull/983
- โ Adds support for Counters inside the Execution Monad.: https://github.com/twitter/scalding/pull/982
- ๐ Make map/flatMap lazy on IterablePipe to address OOM: https://github.com/twitter/scalding/pull/981
- โ JsonLine: enable read transformation in test to get correct fields in sourceTap: https://github.com/twitter/scalding/pull/971
- Read and writable partitioned sources: https://github.com/twitter/scalding/pull/969
- ๐ท Make an Execution[T] type, which is a monad, which makes composing Jobs easy.: https://github.com/twitter/scalding/pull/974
- ๐ Generalize handling of merged TypedPipes: https://github.com/twitter/scalding/pull/975
- Do not inherit from FileSource in LzoTraits: https://github.com/twitter/scalding/pull/976
- ๐ Make TypedPipe immutable: https://github.com/twitter/scalding/pull/968
- โ Adds an optional source: https://github.com/twitter/scalding/pull/963
- โ Add pipe1.join(pipe2) syntax in TypedAPI: https://github.com/twitter/scalding/pull/958
- Extending BddDsl for Typed API: https://github.com/twitter/scalding/pull/956
- VerticaJdbcDriver: https://github.com/twitter/scalding/pull/957
- ๐ fix the example usage in JDBCSource: https://github.com/twitter/scalding/pull/955
- ๐ Push back off ec2 requiring sudo, build failures are a nightmare: https://github.com/twitter/scalding/pull/953
- โ Add ExecutionContextJob to interop execution style with Job style: https://github.com/twitter/scalding/pull/952