Added

• Feed: Monitor.AwaitCompletion enables quasi deterministic waiting for the processing of async reactions within integration tests #170
• Feed: Checkpoint enables committing progress (and obtaining the achieved positions) without stopping the Sink #162
• Feed.SinglePassFeedSource: Coordinates reads of a set of tranches until each reaches its Tail #179
• Streams.Stats.abendThreshold: Abends Sink's processing with a HealthCheckException if a stream continually fails to progress, or continually errors with non-transient exceptions #246
• Streams.Stats.Categorize: Break down ok handler outcomes by stream category #258
• Streams.Stats.Latency.Record(Ok|Outcom|Exn): Gather custom Handler latency metrics #258
• StreamFilter: Generic logic for filtering source events based on Category name, Stream name or Event type #252
• Ingester, Sinks: Expose optional ingesterStateInterval and commitInterval control on Sink factories #154 #239
• Scheduler: Split out stats re rateLimited and timedOut vs exceptions #194
• Scheduler: Added index, eventType to error logging #237
• Scheduler: purgeInterval to control memory usage #97
• Scheduler: wakeForResults option to maximize throughput (without having to drop sleep interval to zero) #161
• Internal.Async.AwaitKeyboardInterruptAsTaskCanceledException: Honored cancellation; moved to Propulsion.Internal #160
• Propulsion.EventStoreDb: Ported EventStore to target Equinox.EventStore >= 4.0.0 (using the gRPC interface) #139
• Propulsion.CosmosStore3: Special cased version of Propulsion.CosmosStore to target Equinox.CosmosStore v [3.0.7-3.99.0] Deprecated; Please migrate to Propulsion.CosmosStore by updating Equinox.CosmosStore dependencies to 4.0.0 #139
• Propulsion.DynamoStore: Equinox.CosmosStore-equivalent functionality for Equinox.DynamoStore. Combines elements of CosmosStore, SqlStreamStore, Feed #140 #140 #177
• Propulsion.MessageDb: FeedSource and CheckpointStore for MessageDb #181 🙏 @nordfjord
• Propulsion.MemoryStore: MemoryStoreSource to align with other sources for integration testing. Includes deterministic AwaitCompletion as per Propulsion.Feed-based Sources #165
• Propulsion.SqlStreamStore: Added startFromTail #173
• Propulsion.Tool: checkpoint commandline option; enables viewing or overriding checkpoints #141
• Propulsion.Tool: sync <kafka|stats> supports from json source option #250
• Propulsion.Tool: Add support for autoscaling throughput of Cosmos containers and databases #142 🙏 @brihadish
• Propulsion.Tool: sync cosmos from <cosmos|json> #252 #263

Changed

• Targeted net6.0 with 6.0.300 SDK, FSharp.Core v 6.0.0, FSharp.Control.TaskSeq v 0.4.0, MathNet.Numerics v 4.15.0
• Changed all Tuple types to struct tuples (System.ValueTuple) #169
• Changed all Async to Task #192
• Dispatcher: Replaced GetConsumingEnumerable with System.Threading.Channels #155
• Ingester,Submitter: Replaced Async.Sleep with Task.WhenAny; Condensed logging #154
• Pipeline: Renamed AwaitShutdown to Await, added Wait #192
• Projector: Replaced Async with task for projector supervision #156
• Scheduler: Replaced Concurrent* with System.Threading.Channels #156
• Scheduler: Struct/voption conversions; buffer reuse #157
• Scheduler: Replaced Thead.Sleep with Task.WhenAny; Added Sleep time logging #161
• Streams: Changed dominant ITimelineEvent EventBody type from byte[] to System.ReadOnlyMemory<byte> (Sinks.EventBody) #169 #208
• Propulsion.CosmosStore: Changed to target Equinox.CosmosStore v 4.0.0 #139
• Propulsion.CosmosStore.CosmosSource: Changed parsing to use System.Text.Json #139
• Propulsion.CosmosStore.CosmosStoreSink+CosmosStorePruner: Exposed *Stats #226
• Propulsion.EventStore: Pinned to target Equinox.EventStore v [3.0.7-3.99.0] Deprecated; Please migrate to Propulsion.EventStoreDb #139
• Propulsion.EventStoreDb.EventStoreSource: Changed API to matchPropulsion.SqlStreamStore API rather thanPropulsion.EventStore #139
• Propulsion.Feed: Moved implementations into main Propulsion library. While this adds a FSharp.Control.TaskSeq dependency, it makes maintenance and navigation easier #265
• Propulsion.Feed,Kafka: Replaced Async with task for supervision #158, #159
• Propulsion.Kafka: Target FsCodec.NewtonsoftJson v 3.0.0 #139
• Propulsion.Prometheus: Extracted Propulsion.Prometheus and Propulsion.Feed.Prometheus in order to remove Prometheus dependency from core package #265
• Propulsion.Tool: project renamed to sync; sources now have a from prefix #252

Removed

• Streams.StreamSpan: Changed from a record to individual arguments of FsCodec.StreamName and Sinks.Event[] #169 #208
• Streams.SpanResult: Replaced with int64 to reflect the updated position #264 #208
• Streams: statsInterval is obtained from the Stats wherever one is supplied #208
• Propulsion.Cosmos: Should not be in general use - users should port to Propulsion.CosmosStore3, then Propulsion.CosmosStore #193
• Destructurama.FSharp dependency #152
• Ingester,Submitter: Removed requirement to specify sleep intervals #154
• Scheduler: Removed need to tune idleDelay except in extreme circumstances #161

Fixed

• Propulsion.CosmosStore.CosmosStoreSource: Propagate correlationId/causationId #260
• Propulsion.SqlStreamStore: Replaced incorrect/meaningless stream name for SqlStreamStoreSource #139
• Synced AwaitTaskCorrect with official version 3c11142
• Projector,Ingester,Submitter, Scheduler: Deterministic shutdown via Cancellation and/or unhandled exceptions #154

New Contributors

• @brihadish made their first contribution in #142
• @nordfjord made their first contribution in #181
• @benjstephenson made their first contribution in #196

Full Changelog: 2.12.2...3.0.0

Versions 2.12.0 and 2.12.1 are unlisted

See ChangeLog

Added

• Prometheus: Propulsion.Streams.Log.LogSink: Support indicating the consumer group name via a Serilog (ForContext) property "group" #137

Changed

• Kafka: Target FsCodec.NewtonsoftJson v 2.3.2 #138
• Prometheus: Propulsion.Streams.Log.LogSink: Replace mandatory group argument with optional defaultGroup to emphasize primacy of "group" Log property #137

Targets (granularly) Equinox.CosmosStore 3.0.6, Equinox.Cosmos 2.6.0, Equinox.EventStore 3.0.6, SqlStreamStore 1.2.0-beta.8, FsCodec 2.3.2, FsKafka 1.7.x, Microsoft.Azure.Cosmos v 3.22.0, Microsoft.Azure.DocumentDB.ChangeFeedProcessor v 2.2.8, prometheus-net v 3.6.0

See ChangeLog

Added

• Kafka.Producers: optional constructor parameters #135 🙏 @avsaditya19

Changed

• CosmosStore: Target Equinox.CosmosStore v 3.0.6#136
• EventStore: Target Equinox.EventStore v 3.0.6 #134

Targets (granularly) Equinox.CosmosStore 3.0.6, Equinox.Cosmos 2.6.0, Equinox.EventStore 3.0.6, SqlStreamStore 1.2.0-beta.8, FsCodec 2.0.0, FsKafka 1.7.x, Microsoft.Azure.Cosmos v 3.22.0, Microsoft.Azure.DocumentDB.ChangeFeedProcessor v 2.2.8, prometheus-net v 3.6.0

See ChangeLog

Changed

• CosmosStoreSource: Generalize Run with integrated Ctrl-C handling to Start yielding a Pipeline #133

Targets (granularly) Equinox.CosmosStore 3.0.5, Equinox.Cosmos 2.6.0, Equinox.EventStore 2.6.0, SqlStreamStore 1.2.0-beta.8, FsCodec 2.0.0, FsKafka 1.7.x, Microsoft.Azure.Cosmos v 3.22.0, Microsoft.Azure.DocumentDB.ChangeFeedProcessor v 2.2.8, prometheus-net v 3.6.0

See ChangeLog

Added

• Streams: Added propulsion_scheduler_busy metrics: count and seconds #126

Changed

• Feed.PeriodicSource: Add readTranches parameter to Pump #130
• Feed.PeriodicSource: Add TrancheId parameter to crawl #130
• CosmosStore: Target Equinox.CosmosStore v 3.0.5
• SqlStreamStore: Rebase on Propulsion.Feed.FeedSource #131

Removed

• net461 support #123

Fixed

• StreamsProjector: Fix failure to pump batches when using maxReadAhead of Int32.MaxValue #124

Targets (granularly) Equinox.CosmosStore 3.0.5, Equinox.Cosmos 2.6.0, Equinox.EventStore 2.6.0, SqlStreamStore 1.2.0-beta.8, FsCodec 2.0.0, FsKafka 1.7.x, Microsoft.Azure.Cosmos v 3.22.0, Microsoft.Azure.DocumentDB.ChangeFeedProcessor v 2.2.8, prometheus-net v 3.6.0

See ChangeLog

Changed

• CosmosStore: Target Microsoft.Azure.Cosmos (V3 CFP) 3.0.22

Targets (granularly) Equinox.CosmosStore 3.0.4, Equinox.Cosmos 2.6.0, Equinox.EventStore 2.6.0, SqlStreamStore 1.2.0-beta.8, FsCodec 2.0.0, FsKafka 1.7.x, Microsoft.Azure.Cosmos v 3.22.0, Microsoft.Azure.DocumentDB.ChangeFeedProcessor v 2.2.8, prometheus-net v 3.6.0

See ChangeLog

Added

• Feed.PeriodicSource: Enables periodic traversal of a source where the source does not present an incrementally readable change feed #117
• Feed.Prometheus: Prometheus integration providing metrics not dissimilar to those available from .CosmosStore (no lag metrics though) #117
• Pipeline: Added AwaitWithStopOnCancellation #118
• CosmosStore: Added handling of With(Error|LeaseAcquire|LeaseRevoke)Notification APIs introduced in Microsoft.Azure.Cosmos v 3.22.0-preview #120

Changed

• CosmosStore: Target Microsoft.Azure.Cosmos (V3 CFP) 3.0.22-preview
• CosmosStore: Target Equinox.CosmosStore v 3.0.4
• CosmosStore: rename maxDocuments to maxItems
• Pipeline: Renamed AwaitCompletion to AwaitShutdown #118
• Kafka: Target FsKafka [1.7.0, 1.9.99)

Removed

• Kafka0: Triggered by removal of underlying FsKafka0 #119
• Kafka: Removed support for net461, triggered by removal of same in underlying FsKafka #119

Fixed

• Fix CI to stop emitting builds with incorrect AssemblyVersion 1.0.0.0 (updated MinVer to 2.5.0)
• Update global.json to use SDK version 5.0.200

Targets (granularly) Equinox.CosmosStore 3.0.4, Equinox.Cosmos 2.6.0, Equinox.EventStore 2.6.0, SqlStreamStore 1.2.0-beta.8, FsCodec 2.0.0, FsKafka 1.7.x, Microsoft.Azure.Cosmos v 3.22.0-preview, Microsoft.Azure.DocumentDB.ChangeFeedProcessor v 2.2.8, prometheus-net v 3.6.0

See ChangeLog

Changed

• CosmosStore: Target Microsoft.Azure.Cosmos v 3.0.20-preview (breaking change vs 3.0.19-preview1) #115

Targets (granularly) Equinox.CosmosStore 3.0.0, Equinox.Cosmos 2.6.0, Equinox.EventStore 2.6.0, SqlStreamStore 1.2.0-beta.8, FsCodec 2.0.0, FsKafka 1.5.x, Microsoft.Azure.Cosmos v 3.20.0-preview, Microsoft.Azure.DocumentDB.ChangeFeedProcessor v 2.2.8, prometheus-net v 3.6.0

See ChangeLog

Changed

• CosmosStore: Port to Equinox.CosmosStore v 3.0.1, Microsoft.Azure.Cosmos v 3.0.19-preview1 #113 #114

Targets Equinox.Cosmos 2.6.0, Equinox.CosmosStore 3.0.0,Equinox.EventStore 2.6.0, SqlStreamStore 1.2.0-beta.8, FsCodec 2.0.0, FsKafka 1.5.x, Microsoft.Azure.Cosmos v 3.19-preview1, Microsoft.Azure.DocumentDB.ChangeFeedProcessor v 2.2.8, prometheus-net v 3.6.0

See ChangeLog

Added

• Prometheus integration
• Propulsion.Feed: Initial implementation of generic feed consumer with support for checkpointing in an Equinox stream (HT @scrwtp who provided the baseline impl in Propulsion.SqlStreamStore) #106
• Propulsion.CosmosStore.CosmosStoreSource: NOTE subject to breaking changes in 2.11.0 as it will be rebased on Microsoft.Azure.Cosmos #103
• Propulsion.CosmosStore.CosmosStoreSink+CosmosStorePruner - equivalents of features in Propulsion.Cosmos #89
• StreamSpan.Version, SpanResult.NoneProcessed #102
• Streams Scheduler: Tune memory consumption re write positions of inactive streams #94
• StreamsProjector: Tuned submission policy to forward 80% of batches read ahead (except in Kafka case where there's limited benefit so default remains to submit 5 per partition) #105

Changed

• CosmosStore: Target Equinox.CosmosStore v 3.0.0
• Cosmos: Target Equinox.Cosmos v [2.6.0,2.99.0]
• EventStore: Target Equinox.EventStore v [2.6.0,2.99.0]
• lots of finessing of log message text
• Unified Kafka.StreamsConsumerStats + Streams.Projector.Stats as Propulsion.Streams.Stats
• Remove wart from createObserver signature introduced in #91 #101
• Add Major for Propulsion.Tool #92
• Replaced numeric field names with strings in latency percentiles message #93
• Kafka: Replace defaulting of Producer's acks defaulting to Leader with mandatory argument #89

Targets Equinox.Cosmos 2.6.0, Equinox.CosmosStore 3.0.0, Equinox.EventStore 2.6.0, SqlStreamStore 1.2.0-beta.8, FsCodec 2.0.0, FsKafka 1.5.x, Microsoft.Azure.DocumentDB.ChangeFeedProcessor v 2.2.8, prometheus-net v 3.6.0