Providers

Reference

Providers

The market-data surface: the data provider a venue exposes, its capability feeds (backfill, live ticks, depth, summary, fundamentals, open interest), the history sources and cache seam, and the activity/progress records that report backfill state.

On this page

Providers: SabrTrader.Pipeline.Providers

An IDataProvider is the market-data port of one connected venue: it advertises a ProviderCapabilities flag set and exposes the feeds it supports as nullable properties. A plugin does not construct one directly; the host builds it through the venue seam (IVenuePlugin → IVenueDescriptor → IVenueSession.Data); see the Venue seam reference and the Venue plugins chapter. Optional facets OF the data plane are probed with AsCapability<T>() rather than added as properties here.

Type Members
IDataProvider : IDisposable Key, DisplayName, Capabilities, Status; nullable feeds Backfill, LiveTicks, Instruments, Depth, Mbo, OpenInterest, MarketSummary, Fundamentals; bool SupportsBarSpec(BarSpecification); event Action<ProviderConnectionStatus>? ConnectionStatusChanged, event Action? MarketDataFeedsChanged; Task ConnectAsync(ct), Task DisconnectAsync().
ProviderCredentialField record (string Key, string DisplayName, ProviderCredentialKind Kind, string? Placeholder = null, bool Required = true, IReadOnlyList<string>? Choices = null, string? DefaultValue = null, string? HelpText = null, string? WarningText = null, bool Multiline = false, bool DynamicChoices = false, bool ReadOnly = false, string? StorageName = null, bool IsIdentity = false), plus init ProviderFieldPresetHint? PresetHint: one row of a venue's SettingsSchema. StorageName (dotted paths allowed) is the key the value is persisted under when it differs from Key. IsIdentity marks a login field that names the person at the venue; only such a field leaves the machine, hashed, for trial-abuse detection. PresetHint ties the field's empty value to a preset picked in another Choice field (see Preset-driven defaults).
ProviderFieldPresetHint record (string ChoiceFieldKey, IReadOnlyDictionary<string, string> HintByChoice). ChoiceFieldKey is the Key of the Choice field that picks the preset; HintByChoice maps each option to the value an empty field takes under it. When the user picks another option, the host empties the field and shows that option's entry greyed in it. An option with no entry shows no hint. The venue resolves the empty value itself; the hint only shows the user which value applies.
ProviderCredentialKind enum: Plain, Secret, Choice, Toggle, Url.
ProviderCapabilities [Flags] enum: None, Backfill, LiveTicks, InstrumentMetadata, OrderRouting, OrderbookDepth, Replay. What IDataProvider.Capabilities advertises; set the bits that match the non-null feeds you expose. The venue manifest's VenueCapabilities is the finer-grained declaration the host uses to route.
BarSpecKind [Flags] enum: None, Time, Tick, Volume, Range, Renko, HeikinAshi. A coarse bar-type mask; the host asks IDataProvider.SupportsBarSpec(spec) instead, so a provider has no member to return it from.
DataConnectionStatus enum: NotApplicable, Disconnected, Connecting, Connected, ConnectionLost, Reconnecting, Failed. The live data connection as an indicator or strategy sees it, read through IIndicatorContext.DataConnection and delivered to OnDataConnectionChanged. It follows the chart's data venue, so it keeps tracking the right connection when the platform replaces a session after a reconnect. NotApplicable means the host has no live connection to report (a backtest, an optimizer pass, historical analysis). See Data connection.
ProviderConnectionStatus enum: Disconnected, Connecting, Connected, ConnectionLost, Failed. The status a venue reports on IDataProvider.Status and IVenueSession.Status. Indicators and strategies see it mapped to DataConnectionStatus.
IDataProviderDecorator IDataProvider Inner { get; }: implemented by a provider that wraps another, so capability probes can reach the inner one.
DataProviderCapabilities static: T? AsCapability<T>(this IDataProvider?): returns the provider, or the first provider down its IDataProviderDecorator.Inner chain, that implements T; null when none does.
IContinuousFuturesSource string? ContinuousBackfillSymbol(string root), Task<string?> ContinuousSeriesContractAsync(string root, CancellationToken ct), bool ContinuousSeriesServesTickData, IRawTickRangeProvider? ContinuousTickContracts, all default null / false, so a venue implements only what it offers. One capability covers the whole continuous-contract story: a server-stitched symbol, the dated contract that series is on right now (the venue then decides the product's roll; a venue that cannot say it offers no series, and the platform stitches its dated contracts on its own roll dates), and the cached per-contract tick source for continuous tick replay. Set ContinuousSeriesServesTickData only once proven: it declares that the server-stitched series ALSO serves tick-level history, so a continuous chart of any bar type (tick, volume, range, Renko) and its order flow load from that series exactly as the candles do, reading the same months the candles show. Default false: some server-stitched series serve time bars only.
IInstrumentDirectory IReadOnlyList<string> ListInstruments().
IRemoteSymbolSearch Task<RemoteSymbolHit?> ResolveAsync(string symbol, ct), Task<IReadOnlyList<RemoteSymbolHit>> SearchAsync(string query, ct): server-side symbol lookup for venues with no enumerable catalog.
RemoteSymbolHit record (string Symbol, string? Exchange = null, string? Description = null, InstrumentCategory? Category = null, QuantityUnit? QuantityUnit = null, double? TickSize = null, double? PointValue = null, double? LotSize = null, double? QuantityStep = null, double? MinQuantity = null, string? QuoteCurrency = null).
ISearchOnlyUniverse bool UniverseIsSearchOnly { get; }: true when the venue's universe is reachable only through search, so discovery routes there.
IPermanentConnectionFailure Empty marker interface on an exception that signals a connection failure a retry cannot fix (bad credentials, as opposed to a dropped link).
IDataProviderFactory string Key, string DisplayName, IReadOnlyList<ProviderCredentialField> RequiredCredentials, IDataProvider Create(IReadOnlyDictionary<string, string> credentials). A building block, not a discovery contract: the plugin loader discovers IVenuePlugin, and a data source ships as a venue plugin whose credential fields travel as VenueManifest.SettingsSchema. A venue family that shares one credential shape can compose a factory inside its session.
SeedingFeedLifecycleLog static Action<string>? Write: a host diagnostic sink the application sets at startup. Host plumbing; plugins leave it alone.
IProviderClock DateTime? NowUtc { get; }: the provider's own notion of "now", in the convention its bars and ticks are stamped. It keeps a chart agnostic to live versus replay: the chart asks the provider what time it is instead of assuming wall-clock. Null means no session clock (a replay provider with nothing loaded); wall-clock is never reported as provider time. Read live: a replay clock advances as playback runs, so callers re-read rather than cache. Providers that do not implement it are treated as real-time.

Capability feeds

Each optional feed follows the same subscribe-and-dispose shape: Subscribe(instrumentId, onUpdate) returns an IDisposable, and a SubscriptionReset event tells consumers to drop state and re-subscribe (after a reconnect, for example).

Type Members
IMarketSummaryFeed : IDisposable IDisposable Subscribe(string instrumentId, Action<MarketSummaryUpdate> onUpdate); event Action? SubscriptionReset.
MarketSummaryUpdate record struct (string InstrumentId, DateTime ExchangeTimestampUtc, double? SettlementPrice = null, double? PreviousClose = null, double? OpeningPrice = null, double? DailyHigh = null, double? DailyLow = null, long? DailyVolume = null, double? LastPrice = null). Merge per field, latest non-null wins; null means "no statement", never "reset". LastPrice is the venue's stated last price, which consumers use only when the venue has no trade stream.
IFundamentalsFeed : IDisposable IDisposable Subscribe(string instrumentId, Action<FundamentalUpdate> onUpdate); event Action? SubscriptionReset.
FundamentalUpdate record struct (string InstrumentId, FundamentalDataType Type, DateTime ExchangeTimestampUtc, double? Value = null, DateTime? DateValue = null).
FundamentalDataType enum: Beta, CurrentRatio, DividendAmount, DividendPayDate, DividendYield, EarningsPerShare, FiveYearsGrowthPercentage, MarketCapitalization, NextYearEarningsPerShare, PercentHeldByInstitutions, PriceEarningsRatio, RevenuePerShare, SharesOutstanding, ShortInterest, ShortInterestRatio, AverageDailyVolume, CalendarYearHigh, CalendarYearHighDate, CalendarYearLow, CalendarYearLowDate, High52Weeks, High52WeeksDate, Low52Weeks, Low52WeeksDate, HistoricalVolatility.
IOpenInterestFeed : IDisposable IDisposable Subscribe(string instrumentId, Action<OpenInterestUpdate> onUpdate); event Action? SubscriptionReset.
OpenInterestUpdate record struct (string InstrumentId, double OpenInterest, DateTime ExchangeTimestampUtc).
MarketSummaryReplayCache Thread-safe "latest non-null wins" cache a venue uses to replay the merged summary to a late subscriber: void Observe(in MarketSummaryUpdate), MarketSummaryUpdate? Replay(string instrumentId), Forget(string instrumentId), Clear().
Depth feeds. IMarketDepthFeed (level-2) and IMarketByOrderFeed (MBO / level-3) are documented under the Orderflow & depth reference; IDataProvider exposes them as the Depth and Mbo properties.

Backfill & ticks: SabrTrader.Pipeline.Runtime

IBackfillProvider and its refinements are the consumer-facing history surface. A venue plugin normally gets them from the backfill composer, which builds the cached chain from the history sources in the storage seam; implement a refinement directly only when your venue serves it natively. The host probes for each refinement. ITickSource delivers live ticks.

Type Members
IBackfillProvider Task<IReadOnlyList<Bar>> Load(BarSpecification spec, CancellationToken ct = default).
ILookbackAwareBackfillProvider adds Load(spec, TimeSpan lookback, ct).
IRangeAwareBackfillProvider adds Load(spec, DateTime fromUtc, DateTime toUtcExclusive, ct).
IProgressReportingBackfillProvider adds LoadWithProgress(spec, IProgress<BackfillProgress>, ct).
IProgressReportingRangeBackfillProvider adds LoadWithProgress(spec, fromUtc, toUtcExclusive, IProgress<BackfillProgress>, ct).
IOrderFlowAwareBackfillProvider adds Task<BackfillResultWithOrderFlow> LoadWithOrderFlow(spec, OrderFlowBackfillRequest, ct).
IOpenInterestAwareBackfillProvider adds Task<BackfillResultWithOpenInterest> LoadWithOpenInterest(spec, OpenInterestBackfillRequest, ct).
ILiveHistoryProvider bool ProvidesLiveHistory { get; }, Task<LiveHistory> LoadLive(BarSpecification spec, LiveHistoryRequest request, ct). Optional capability of a backfill provider: load a series' history for a consumer that goes live afterwards, as one answer that carries the still-forming bar and exactly how far its data reaches. Probe ProvidesLiveHistory before calling; LoadLive throws InvalidOperationException when it is false. The chain the backfill composer builds implements it. A decorator serves live history only when the provider it wraps does, transforms the bars exactly as its Load does, and keeps the reach only while the forming bar stays the last bar.
LiveHistoryRequest record (DateTime AsOfUtc) with init DateTime? FromUtc, DateTime? ToUtcExclusive, TimeSpan? Lookback, IProgress<BackfillProgress>? Progress, OrderFlowResolution? OrderFlow (null for bars only), string? TickSizeInstrumentId. AsOfUtc is the one instant the history reaches and live ticks take over from; every "is this bar still forming" decision is made against it. The window is [FromUtc, ToUtcExclusive): the end defaults to AsOfUtc and is never after it, the start defaults to the end minus Lookback, and Lookback to the provider's default depth. A window that ends before AsOfUtc must name its start or its lookback. Methods: (DateTime FromUtc, DateTime ToUtcExclusive) ResolveWindow(TimeSpan defaultLookback) gives the concrete window given your default depth; bool TryGetExplicitWindow(out DateTime fromUtc, out DateTime toUtcExclusive) is true when the request names absolute instants (a start, or an end plus a lookback) and false when it reaches AsOfUtc from a depth; OrderFlowBackfillRequest ToOrderFlowRequest() builds the equivalent order-flow load (explicit window when there is one, else the lookback) and throws when OrderFlow is null; Validate() throws when the instants are not UTC or the window cannot be described.
LiveHistory LiveHistory(IReadOnlyList<Bar> bars, IReadOnlyList<OrderFlowBar>? orderFlow, LiveHandoffPoint? formingReach): Bars (oldest first), OrderFlow (index-aligned with Bars, or null when the load carried none), FormingReach. The reach is set only on a still-forming last bar and says how far its data goes: live ticks at or before it are already in the bar, every later tick extends it. Null means the provider cannot say, and the bar is taken as a snapshot through AsOfUtc. The constructor throws when the footprint list is not index-aligned or the reach lies outside the last bar.
OrderFlowBackfillRequest record (DateTime? FromUtc = null, DateTime? ToUtcExclusive = null, TimeSpan? Lookback = null, IProgress<BackfillProgress>? Progress = null, OrderFlowResolution Resolution = OrderFlowResolution.Auto); init OrderFlowResolution? AutoTier: the concrete tier an Auto request resolves to when the CALLER has already decided it (a window profile sizes Auto by the window's span, which the chart period cannot know); the request stays an Auto request, so a venue's exact per-price ladders still serve it where it has them. Only valid on an Auto request and never Auto itself. string? TickSizeInstrumentId: the instrument whose tick size prices this load when it is not the requested one; the continuous-futures stitch names the front month here for each older month's slice, because an expired month has no reference data left to read its own tick size from. HasRange, HasLookback, Validate().
OpenInterestBackfillRequest record (DateTime? FromUtc = null, DateTime? ToUtcExclusive = null, TimeSpan? Lookback = null, IProgress<BackfillProgress>? Progress = null): HasRange, HasLookback, Validate().
ITickSource : IDisposable IDisposable Subscribe(string instrumentId, Action<Tick> onTick); plus a default-implemented routed overload Subscribe(string instrumentId, string? exchange, Action<Tick> onTick) that delegates to the plain one. Exchange travels on the REQUEST: a venue that serves the same instrument from several exchanges overrides the routed overload, a single-feed venue inherits the default. A DECORATOR must override and forward it, or the picked exchange is dropped and the chart mixes feeds.
ITimeBarPeriodSupport bool SupportsNatively(TimeSpan period), TimeSpan? LargestNativeDivisor(TimeSpan period); extension TimeBarPeriodSupportExtensions.CanServe(period) refuses timeframes the venue cannot serve. Two implementations ship: LadderTimeBarPeriodSupport, for a venue with a fixed ladder of native periods (see the venue seam), and AnyWholeSecondPeriodSupport, for a venue that publishes no period ladder. AnyWholeSecondPeriodSupport.Instance is its one shared instance (the class is sealed and has no public constructor): SupportsNatively is true for every positive whole-second period, and LargestNativeDivisor always returns null, because a whole second divides only whole-second periods and those are already native. The host uses it when a venue states no period support of its own, so a venue only passes it to say so explicitly.
BackfillProgress record (int BarsReceived, DateTime? LatestBarTimeUtc, double PercentComplete, string StageMessage).
BackfillResultWithOrderFlow Bars, OrderFlow, FirstBarStartUtc, LastBarEndUtc, Count; static Empty.
BackfillResultWithOpenInterest Bars, OpenInterest, FirstBarStartUtc, LastBarEndUtc, Count; static Empty.

Storage seam: SabrTrader.Pipeline.Storage

Two halves. The history sources (IHistoricalTimeBarSource, IHistoricalTickSource, IHistoricalOrderFlowBarSource, INativeBarSource) are what a venue implements and hands to the backfill composer. The other half is the host's local tick/bar cache: the host reads what it already holds and asks your sources only for the DateRanges still missing.

Type Members
HistoryRequest record (string InstrumentId, string? Exchange, DateTime FromUtc, DateTime ToUtcExclusive, IProgress<BackfillProgress>? Progress = null): one history request. Exchange is a picked feed or null for default routing; Progress receives per-page progress or is null.
HistoryServed<T> Items, FromUtc, ToUtcExclusive, ServedFromUtc, ServedToUtcExclusive, IsComplete; factories Complete(items, request), Through(items, request, servedToUtcExclusive), Span(items, request, servedFromUtc, servedToUtcExclusive), each also with explicit (fromUtc, toUtcExclusive) instead of the request, and None(request) for a window the venue answered but holds nothing it can vouch for. The span the venue vouches it served completely; the cache seals only whole days inside it (see Saying what you served).
IHistoricalTimeBarSource Task<HistoryServed<Bar>> LoadTimeBars(HistoryRequest request, TimeSpan period, ct).
IHistoricalTickSource Task<HistoryServed<Tick>> LoadTicks(HistoryRequest request, ct), bool ProvidesHistoricalTicks { get; }.
NoHistoricalTicks IHistoricalTickSource for a venue with no historical trades API: Instance, ProvidesHistoricalTicks is false, and LoadTicks throws BackfillSourceUnavailableException.
IHistoricalOrderFlowBarSource bool ProvidesProfiledMinuteBars { get; }, Task<HistoryServed<OrderFlowBar>> LoadProfiledMinuteBars(HistoryRequest request, ct).
INativeBarSource bool SupportsNativeBars(spec), Task<HistoryServed<Bar>> LoadNativeBars(spec, fromUtc, toUtcExclusive, IProgress<BackfillProgress>? progress, ct).
ISeamPriorityHistoricalTickSource : IHistoricalTickSource adds Task<HistoryServed<Tick>> LoadTicksUncachedForSeam(HistoryRequest request, ct): serves the history-to-live seam's uncached fetch on a dedicated lane, so it is not starved behind a bulk day backfill. Same contract as LoadTicks; the served span is exactly where the seam hands over to live ticks.
IOpenInterestAwareTimeBarSource : IHistoricalTimeBarSource adds Task<BackfillResultWithOpenInterest> LoadTimeBarsWithOpenInterest(HistoryRequest request, TimeSpan period, ct): native time bars with per-bar open interest (double.NaN for a bar with none), aligned 1:1. Not cached, so it carries no served span: a venue that serves nothing throws BackfillSourceUnavailableException.
IRawTickRangeProvider bool ProvidesHistoricalTicks (default true), Task<IReadOnlyList<Tick>> LoadTicksAsync(string instrumentId, DateTime fromUtc, DateTime toUtcExclusive, IProgress<BackfillProgress>? progress = null, ct), LoadTicksUncachedAsync(string instrumentId, DateTime fromUtc, DateTime toUtcExclusive, ct) and its routed overload with string? exchange. The cached, range-aware raw tick surface a backfill provider offers (store first, gap fill on a miss), used by backtest tick replay and continuous tick replay. LoadTicksUncachedAsync reads straight from the venue for the small in-progress window at the seam. A decorator forwards ProvidesHistoricalTicks and the routed overload to its inner provider.
LiveHandoffPoint record struct (DateTime At, int HistoricalTradesAtInstant): where a series' history ends and live ticks take over. Every tick stamped before At is history, and so are the first HistoricalTradesAtInstant trades stamped exactly At; every other tick is live. The count matters because several trades often share one exchange timestamp, and history and the live feed deliver them in the same order. Factories: Before(at) (everything before, nothing at it, as at a closed time bar's end), Through(at) (everything at or before, as for a snapshot taken at that instant), AfterLastOf(IReadOnlyList<Tick> ticks) (exactly those ticks; throws on an empty list). The count is never negative.
DateRange record (DateTime FromUtc, DateTime ToUtcExclusive): IsEmpty, Length, Contains(utc), Overlaps(other), TouchesOrOverlaps(other).
CacheProviderKey CacheProviderKey.Normalize(providerKey): the provider namespace the host's caches key under. DepthStoreKey normalises its provider key through it.
BackfillSourceUnavailableException Exception a history source throws when the venue could not be asked: not connected, unreachable, 5xx or 429, timed out. The cache serves what it has, leaves the window missing and stops asking the venue for the rest of that load.

Historical data service: SabrTrader.Pipeline.Services

The host-side facade your plugins call to load history by venue without touching the store/provider seams directly.

Type Members
IHistoricalDataService Task<LoadResult> LoadAsync(string venueKey, BarSpecification spec, DateTime fromUtc, DateTime toUtcExclusive, IProgress<LoadProgress>? progress = null, ct); Task<IReadOnlyList<Tick>> LoadTicksAsync(string venueKey, string instrumentId, DateTime fromUtc, DateTime toUtcExclusive, IProgress<LoadProgress>? progress = null, ct); Task<BackfillResultWithOrderFlow?> LoadWithOrderFlowAsync(string venueKey, BarSpecification spec, DateTime fromUtc, DateTime toUtcExclusive, IProgress<LoadProgress>? progress = null, ct), bars with their order flow for backtesting order-flow strategies, or null when the active provider chain cannot serve order flow (fall back to LoadAsync).
LoadProgress record (int BarsReceived, DateTime LatestTimestampUtc, double PercentComplete).
LoadResult record (IReadOnlyList<Bar> Bars, LoadSummary Summary).
LoadSummary record (string VenueKey, string InstrumentId, BarSpecification Spec, DateTime FromUtc, DateTime ToUtcExclusive, int BarCount, DateTime? FirstBarTimeUtc, DateTime? LastBarTimeUtc).

Host-side types

These types sit in the same namespaces as the provider surface. The host and first-party venues use them; a plugin reads them at most.

Type What it is
IInstrumentAdjustmentResolver Task EnsureAdjustmentVersionAsync(string instrumentId, ct): an optional companion to IInstrumentMetadata for a venue whose corporate-action version needs an async lookup. The cache awaits it before building a time-bar cache key, so a new split invalidates cached bars on the first request.
AsyncSerialRequestQueue · RequestPriority SabrTrader.Pipeline.Async: a single-slot async queue. ExecuteAsync(work, ct) and ExecuteAsync(work, RequestPriority priority, ct) (plain and Task<T> forms) run at most one body at a time across callers; RequestPriority.High cuts in front of every queued Normal request. A venue uses it to serialise requests to an API that allows one in flight.
IPerfRecorder · PerfEvent · PerfStats · PerfStatsKey · PerfStatsSnapshot SabrTrader.Pipeline.Perf: the host's timing recorder behind the Task Manager. BeginScope(category, detail?, chartId?) returns a PerfScope (a disabled recorder returns PerfScope.Disabled, the no-op scope whose Dispose does nothing, so an instrumented hot path costs one branch while recording is off); Snapshot(lookback) returns per-key PerfStats (Count, TotalMs, MinMs, MaxMs, MeanMs, P50Ms, P95Ms) and the raw PerfEvents (Category, Detail, ChartId, StartTimestamp, ElapsedTicks, ManagedThreadId). An indicator times its rendering through IIndicatorRenderContext.BeginScope instead.
← BackDrawing