All Versions
39
Latest Version
Avg Release Cycle
219 days
Latest Release
1933 days ago

Changelog History
Page 1

  • v0.18.0-RC1

    December 12, 2018
  • v0.17.4 Changes

    December 22, 2017

    ๐Ÿš€ This release is binary compatible with 0.17.3 so it should be safe to use. One behavior change is that skipping null counters is now opt in (which was a default we regretted when shipping 0.17.3). See: #1716

    • โž• add DateRange.prepend: #1748
    • TextLine is now a TypedSink[String]: #1752
    • check for _SUCCESS file in any FileSource based on a config flag: #1758
    • โž• add a setting to skip null counters: #1759
  • v0.17.3 Changes

    September 30, 2017

    ๐Ÿ›  This is a minor bugfix release that works around hadoop giving us a null counter reporter. We work around by ignoring counters. This may not be the best solution, but it unblocks some users. We don't yet know why hadoop sometimes gives this users a null counter reporter.

    ๐Ÿ‘€ See #1726

  • v0.17.2 Changes

    This version is basically the same as 0.17.1 but backward compatible with 0.17.0.

    • โช Revert memory estimator changes on 0.17.x branch: #1704
    • Turn on mima checks on 0.17.x branch: #1706
  • v0.17.1 Changes

    This version breaks backward compatibility with 0.17.0, don't use it, use 0.17.2 instead.

    • ๐Ÿš€ Request for Scalding release 0.17.0: #1641
    • ๐Ÿ‘‰ make ordered serialization stable across compilations: #1664
    • โœ‚ Remove unnecessary semicolon: #1668
    • โž• Add tailrec annotation: #1671
    • Be more paranoid about Kryo registration order: #1673
    • โšก๏ธ Update sbt version to 0.13.15: #1677
    • Register all Boxed classes in Kryo: #1678
    • ๐Ÿ›  Fix serialization of KryoHadoop: #1685
    • Merge pull request #1686 from ttim/cherry_pick_0.17.x_changes
    • ๐Ÿ›  Fix stack overflow in typedPipeMonoid.zero: #1688
    • ๐Ÿ›  A couple of fixes into the 0.17.x branch: #1695
    • Memory estimator changes to 0.17.x branch: #1700
  • v0.17.0 Changes

    ๐Ÿš€ This is the first Scalding release that publishes artifacts for Scala 2.12!

    • โšก๏ธ 2.12 releated updates: #1663, #1646
    • ๐Ÿ‘ท Use reflection over Jobs to find serialized classes: #1654, #1662
    • Simplify match statement and use collection.breakOut: #1661
    • โž• Add explicit types to implicit methods and values: #1660
    • โฌ‡๏ธ Reducer estimation size fixes: #1652, #1650, #1645, #1644
    • ๐Ÿ‘‰ Use Combined*SequenceFile for VKVS, WritableSequenceFileScheme, SequenceFileScheme: #1647
    • ๐Ÿ‘Œ Improve Vertica support in scalding-db: #1655
    • โž• Add andThen to Mappable: #1656
    • Expand libjars globs in ScaldingShell to match the behavior of Tool: #1651
    • ๐Ÿ‘‰ Use Batched in Sketch production: #1648
    • Pick up Algebird 0.13.0: #1640
    • โž• Added API for Execution/Config to work with DistributedCache: #1635
    • โฌ†๏ธ Bump chill version to 0.8.3: #1634
    • ๐Ÿ›  Fixes a bug in how we use this stack: #1632
    • โฌ†๏ธ Upgrade build to sbt 0.13.13: #1629
    • Generate Scalding microsite via sbt-microsites: #1623
    • ๐Ÿ‘ FileSource support for empty directories: #1622, #1618, #1613, #1611, #1591
    • Clean up temporary files created by forceToDiskExecution: #1621
    • Moving the repl in wonderland to a dedicated md file: #1614
    • โšก๏ธ Update Scala and sbt version: #1610
    • ๐Ÿ”จ REFACTOR: Fixed some compilation warnings: #1604
    • ๐Ÿ”จ REFACTOR: Rename parameter to reflect expectation: #1601
    • โž• Add partitioned sources for Parquet thrift / scrooge: #1590
    • โž• Add a test for sortBy: #1594
    • Create COMMITTERS.md: #1589
    • ๐Ÿ‘‰ Use ExecutionContext in Execution.from/fromTry: #1587
    • ๐Ÿ‘Œ Support custom parquet field name strategies: #1580
    • โœ… Deprecate reflection-based JobTest apply method: #1578
    • ๐Ÿ‘‰ Use Caching for FlowDefExecution: #1581
    • ๐Ÿ—„ [parquet tuple macros] listType was deprecated in favor of listOfElements: #1579
    • ๐Ÿ‘‰ Use Batched to speed up CMS summing on mappers: #1575
    • โœ‚ Remove a TypedPipeFactory wrapper which seems unneeded: #1576
    • ๐Ÿ‘‰ Make Writeable sources Mappable to get toIterator: #1573
    • case class implicit children: #1569
  • v0.16.0 Changes

    • โž• Add tests around hashcode collisions : #1299
    • ๐Ÿ›  Fix performance bug in TypedPipeDiff : #1300
    • ๐Ÿ— make serialization modules build on travis : #1301
    • ๐Ÿ‘Œ Improve TypedParquetTuple : #1303
    • โž• Add UnitOrderedSerialization : #1304
    • โช Revert "Add UnitOrderedSerialization" : #1306
    • ๐Ÿ”„ Change groupRandomly & groupAll to use OrderedSerialization : #1307
    • โœ… Make test of Kmeans very very unlikely to fail : #1310
    • ๐Ÿ‘‰ make LongThrift sources TypedSink : #1313
    • ๐Ÿ›  Fix testing VersionedKeyValSource#toIterator for non-Array[Byte] types : #1314
    • ๐Ÿ‘‰ Make SketchJoin ordered serialization aware : #1316
    • โž• Added a sealed trait ordered serializer. When it works its great. Not as reliable as we'd like. But hopefully restrictions on it will do the job : #1320
    • โž• Add secondary sorting using ordered serialization : #1321
    • Bails out from the length calculation if we don't succeed often : #1322
    • increased number of box instances to 250 : #1323
    • ๐Ÿ”€ Apply merge strategy for pom.properties files : #1325
    • ๐Ÿ”€ Apply merge strategy for pom.xml files : #1327
    • โž• Add a OrderedSerialization.viaTransform with no dependencies, and a BijectedOrderedSerialization in scalding core : #1329
    • Precompute int hashes : #1330
    • ๐Ÿ—„ Hide the deprecated string error for getting ASCII bytes. : #1332
    • ๐Ÿ”„ Change defaults for Scalding reducer estimator : #1333
    • Execution id code : #1334
    • โž• Add line numbers at .group and .toPipe boundaries : #1335
    • Ordered Serialization macros for thrift : #1338
    • ๐Ÿ‘‰ make some repl components extensible : #1342
    • โœ‚ Remove the bootstrap section : #1346
    • ๐Ÿ›  Fix the execution test : #1347
    • Implement flatMapValues method : #1348
    • ๐Ÿ’… Consistent style in homepage example : #1349
    • Serialization folding : #1351
    • ๐Ÿ“ฆ Collapses scalding-db packages : #1353
    • ๐Ÿ”€ Merge scalding-macros into scalding-core : #1355
    • Migrate typedtext : #1356
    • โš™ Runtime reducer estimator : #1358
    • โšก๏ธ Update Build.scala : #1361
    • ๐Ÿ‘ Allow overriding of hadoop configuration options for a single source/sink : #1362
    • Missing an extends Serializable, causes issues if capture Config's anywhere : #1365
    • ๐Ÿ›  Fix TypedPipe.limit to be correct, if slightly slower : #1366
    • ๐Ÿ›  Fix scala.Function2 showing up in line numbers : #1367
    • โฌ‡๏ธ Drop with MacroGenerated from Fields macros : #1370
    • ๐Ÿ›  Fix deprecation warnings in TypedDelimited : #1371
    • โช Ianoc/revert changes around making file systems : #1372
    • โช Revert typed tsv behavior : #1373
    • ๐Ÿ‘€ A serialization error we were seeing in repl usage : #1376
    • โž• Add NullSink and test : #1378
    • โž• Add monoid and semigroup for Execution : #1379
    • โฌ†๏ธ Upgrade parquet to 1.8.1 : #1380
    • โฌ†๏ธ Upgrade sbt launcher script (sbt-extras) : #1381
    • ๐Ÿšš Just move whitespace, add comments, simplify a few methods : #1383
    • Don't publish maple when doing 2.11 so we only publish it once -- needed for cross publishing to maven repo's : #1386
    • ๐Ÿ‘Œ Support nesting Options in TypeDescriptor : #1387
    • Enable Scalding-REPL for Scala 2.11 : #1388
    • โšก๏ธ Updates for some upstream fixes/changes : #1390
    • โœ‚ Remove use of hadoop version in estimators : #1391
    • Set hadoop version to dummy value : #1392
    • ๐Ÿ– Handle no history case in RatioBasedEstimator : #1393
    • Inline parquet-scrooge : #1395
    • โœ… RatioBasedEstimator - fix threshold edge case, add tests : #1397
    • ๐Ÿ›  Fixes the scrooge generator tasks not to generate code in the compile target, we were publishing these : #1399
    • ๐Ÿ”ง Ianoc/configure set converter : #1400
    • ๐Ÿ”„ Change hash function in GroupRandomly : #1401
    • ๐Ÿ‘Œ Improve logging in runtime reducer estimators : #1402
    • โž• Add the type in ScroogeReadSupport : #1403
    • โž• Adds a function to test if a sink exists at the version we created : #1404
    • โž• add .groupWith method to TypedPipe : #1406
    • โž• Add some return types : #1407
    • โž• add counter verification logic : #1409
    • ๐Ÿ›  Runtime reducer estimator fixes : #1411
    • ๐Ÿ‘‰ Make sure Execution.zip fails fast : #1412
    • When using WriteExecution and forceToDisk we can share the same flow def closer in construction : #1414
    • Cache the zipped up write executions : #1415
    • ๐Ÿ›  Fix DateOps "match may not be exhaustive" warning : #1416
    • Factor out repeated code into FutureCache : #1417
    • ๐Ÿ›  Fix lack of Externalizer in joins. : #1421
    • โž• Adds much more line number information through the NoStackAndThen class : #1423
    • Requires a DateRange's "end" to be after its "start" : #1425
    • Scalding viz options : #1426
    • ๐Ÿ›  Fixes map-only jobs to accommodate both an lzo source and sink binary converter : #1431
    • ๐Ÿ›  Fix Readme travis link : #1432
    • ๐Ÿ›  Fixes docs wording : #1433
    • ๐Ÿ‘ป Don't squash the exception in history service when there's a failure : #1434
    • ๐ŸŒฒ Log the exception in RatioBasedEstimator when there's a failure : #1435
    • ๐Ÿ‘‰ make getBytesPerReducer support human readable values like 128m and 1g : #1436
    • ๐Ÿ›  Fixes minor KeyedList docs wording : #1437
    • ๐Ÿ›  Fix readPathsFor to use the tz argument : #1439
    • Scalding viz options : #1440
    • โœ… call Job.validate when running tests under JobTest : #1441
    • โœ… opt-in to calling Job.validate in JobTest : #1444
    • ๐Ÿ›  Fix bug with sketch joins and single keys : #1451
    • ๐Ÿ›  Fix incorrect usage of percent. : #1455
    • โž• Add OrderedSerialization2 support in Matrix2. : #1457
    • โž• Add InvalidSourceTap to catch all cases for no good path. : #1458
    • Cluster info and fs shell in repl : #1462
    • โšก๏ธ Update Scala version to 2.10.6 : #1463
    • ๐Ÿ›  Fix median estimation : #1464
    • ๐Ÿ‘‰ Makes the config transient in the KryoHadoop instanciator : #1466
    • ๐Ÿšš Moves the default to 2.11 : #1467
    • โž• Adds Error Message to REPL when Current Directory Not Readable : #1468
    • SuccessFileSource: correctness for multi-dir globs : #1470
    • Limit task history fields consumed from hraven : #1472
    • โœ‚ Remove dependency on dfs-datastores : #1473
    • ScaldingILoop should enable one to pass in in/out : #1475
    • Switch Chat to Gitter : #1477
    • โž• Add two functions that assist in testing a TypedPipe : #1478
    • ๐Ÿ‘‰ Makes permission failures non-fatal when looking for .scalding_repl files : #1479
    • โšก๏ธ Update TypeDescriptor to explain that Option[String] is not supported : #1480
    • โœ‚ Remove a type parameter that doesn't seem to do anything : #1481
    • Utility for expanding libjars : #1483
    • Shouldn't skip hidden files, user can decide such things with their glob : #1485
    • ๐Ÿ›  Fix FileSystem.get issue : #1487
    • โœ‚ Remove dependency on parquet-cascading : #1488
    • โž• Add withConfig api to allow running an execution with a transformed config : #1489
    • Call validateTaps in toIterator codepath : #1490
    • โšก๏ธ Update the build : #1491
    • Arg Descriptions/Help for Execution Apps : #1492
    • ๐Ÿ›  Fix issue #1429 : #1493
    • โšก๏ธ Cache counters for stat updates : #1495
    • โœ… Pulls the core ExecutionTests back into scalding-core : #1498
    • โž• Add a liftToTry function to Execution : #1499
    • Small improvements to the Boxed.scala module : #1500
    • Cache boxed classes : #1501
    • ๐Ÿ›  Fix unnecessary use of .get in Globifier.scala : #1502
    • Replace unintentional use of Unit with () : #1503
    • ๐Ÿ›  Fix unnecessary uses of Option.get : #1506
    • Utility methods for running Executions in parallel : #1507
    • Typed Mapside Reduce : #1508
    • ๐Ÿšš Use wartremover to guard against careless use of _.get : #1509
    • โž• Add in an API around cache isolation : #1511
    • โž• Add implicit Ordering[RichDate] : #1512
    • ๐Ÿ›  Fix MultipleTextLineFiles source in JobTest : #1513
    • โž• Add's support for sealed abstract classes : #1518
    • Update FixedPathSource to strip out '' in paths ending with '/' for writes : #1520
    • ๐Ÿ‘Œ support for more formats to work with RichDate : #1522
    • ๐Ÿšง WIP: Add forceToDisk parameter to hashJoin in TypedPipe : #1529
    • ๐Ÿ›  Fixing comments on partitioned delimited source : #1530
    • โœ‚ Remove weakly typed Source : #1531
    • Maple fix for HBaseTap : #1532
    • โž• Add an enrichment for TypedPipe.inMemoryToList and use it in TypedPipeDiff test : #1533
    • Because, because... fun, the scala compiler has special naming rules it appears when there are leading underscores : #1534
    • ๐Ÿ›  Fix README examples link : #1536
    • ๐Ÿ›  Fixes Config to accommodate spaces in argument values : #1537
    • โž• Add before() and after() to RichDate : #1538
    • โž• Adds late tap validation for cases where race conditions cause it to fail : #1540
    • ๐Ÿ›  Fix Rounding Bug in RatioBasedEstimator : #1542
  • v0.15.0 Changes

    • ๐Ÿšš Move OrderedSerialization into zero-dep scalding-serialization module #1289
    • โฌ†๏ธ bump elephantbird to 4.8 #1292
    • ๐Ÿ›  Fix OrderedSerialization for some forked graphs #1293
    • โž• Add serialization modules to aggregate list #1298
  • v0.14.0 Changes

    • โž• add .unit to Execution object #1189
    • Override hashCode for Args #1190
    • ๐Ÿ‘ป Put a value in a exception message #1191
    • โž• Add an exclusiveUpper method to DateRange #1194
    • Covert LzoTextDelimited to Cascading scheme. #1179
    • โœ‚ Remove Travis IRC notifications #1200
    • โž• add LookupJoin and LookupJoinTest changes from summingbird #1199
    • โž• Add a new ExecutionApp tutorial #1196
    • ๐Ÿšš Move main simple example to be the typed API, and put the .'s at the sta... #1193
    • โž• Add Execution.withArgs #1205
    • โšก๏ธ Config/Cascading updater #1197
    • โœ‚ Remove algebird serializers #1206
    • โœ‚ remove warnings in CumulativeSum #1215
    • Implicit execution context / easier switching between modes #1113
    • โž• add row l1 normalize #1214
    • provide Args as an implicit val #1219
    • call sourceConfInit when reading from taps in local mode #1228
    • โž• Add distinctCount and distinctValues helper methods to KeyedList. #1232
    • ๐Ÿšš import hygiene: remove unused imports and remove JavaConversions use #1239
    • Swap hash and filename for filename-extension-sensitive code #1243
    • โœ‚ Remove more unused imports #1240
    • Provide useHdfsLocalMode for an easy switch to mapreduce local mode #1244
    • โฌ†๏ธ upgrade scalacheck and scalatest #1246
    • โšก๏ธ Optimize string and (hopefully) number comparisons a bit #1241
    • Note the active FlowProcess for Joiners #1235
    • ๐Ÿ‘‰ Make sure Executions are executed at most once #1253
    • ๐Ÿ›  Fix Config.getUniqueIDs #1254
    • โž• Add MustHasReducers trait. #1252
    • ๐Ÿ‘‰ Make sure the EvalCache thread isDaemon #1255
    • ๐Ÿ‘‰ Use non-regex split function #1251
    • ๐Ÿ‘‰ make InputSizeReducerEstimator work for any CompositeTap #1256
    • TimePathedSource helper methods #1257
    • ๐Ÿ›  Fix for reducer estimation not working correctly if withReducers is set to 1 reducer #1263
    • โž• Add make(dest) to TypedPipe #1217
    • ๐Ÿ›  Fix SimpleDateFormat caching by default #1265
    • โฌ†๏ธ upgrade sbt and sbt launcher script #1270
    • โž• Add TypedPipeDiff for comparing typed pipes #1266
    • ๐Ÿ”„ Change separator from \1 to \u0001 #1271
    • Disable reducer estimation for map-only steps #1276
    • ๐Ÿ‘ Local sources support multiple paths #1275
    • ๐Ÿ›  fix the spelling of the cumulativeSumTest file #1281
    • Hydrate both sides of sampledCounts in skewJoinWithSmaller #1278
    • Bijection 0.8.0, algebird 0.10.0, chill 0.6.0, scala 2.10.5 #1287
    • โœ‚ Remove some deprecated items #1288
  • v0.13.1 Changes