Data Store

A DataStoreConfig reads from a stored dataset. Filters narrow the rows that are imported.

DataStoreConfig — data source backed by a managed data store.

class DataStoreConfig(data_store_key, tables)

Data source that reads rows from a stored dataset.

Apply a DataFilter via set_model_filter() to restrict which rows are imported.

Parameters:

• data_store_key (str) – Key of the data store to read from.

• tables (dict[str, str]) – Mapping from local model table name to the source table name.

set_model_filter(model_filter)

Apply a row-level DataFilter to narrow the rows pulled in.

Return type:

DataStoreConfig

set_debug_file(debug_file)

Emit a debug copy of the imported data.

Return type:

DataStoreConfig

set_using_interface(using_interface)

Read through the data store’s interface layer rather than directly.

Return type:

DataStoreConfig

set_direct_data_pull(direct_data_pull)

Bypass any caching layer and pull rows directly each import.

Return type:

DataStoreConfig

property type: DataSourceType

The DataSourceType discriminator for this configuration.

Filters

Abstract DataFilter base class for row-level DataStoreConfig filters.

Concrete subclasses declare their @type discriminator via the @json_type_info decorator, and implement type.

class DataFilter

Abstract base for typed data-store row filters.

property type: DataFilterType

The DataFilterType discriminator for this filter.

DataFilterType discriminator enum for DataFilter.

class DataFilterType(*values)

Identifies the kind of row filter applied by DataFilter subclasses.

Values:

EQUALITY: Match rows whose value equals a source key. INEQUALITY: Match rows whose value falls within a numeric range. SET: Match rows whose value is in a set of source keys. WILDCARD: Match rows whose value matches a glob pattern. REGEX: Match rows whose value matches a regular expression.

EQUALITY = 'equality'

INEQUALITY = 'inequality'

SET = 'set'

WILDCARD = 'wildcard'

REGEX = 'regex'

EqualityDataFilter — a DataFilter matching exact-equality rows.

class EqualityDataFilter(path, source_key, value=None)

Match rows whose value at path equals the value resolved from source_key.

Parameters:

• path (list[str]) – Field path within each source row (e.g. ["address", "country"]).

• source_key (Parameter | Calculation) – Model named value supplying the comparison value.

• value (str | None) – Optional literal value emitted alongside the resolved key.

property type: DataFilterType

The DataFilterType discriminator for this filter.

build()

Serialise to a JSON-compatible dict.

Return type:

dict[str, Any]

InequalityDataFilter — a DataFilter matching numeric range rows.

class InequalityDataFilter(path, lower_key, upper_key, lower=None, upper=None)

Match rows whose numeric value at path falls within a range.

Parameters:

• path (list[str]) – Field path within each source row.

• lower_key (float | Parameter | Calculation) – Lower bound source — a literal or model named value.

• upper_key (float | Parameter | Calculation) – Upper bound source — a literal or model named value.

• lower (float | None) – Optional literal lower-bound value emitted alongside the key.

• upper (float | None) – Optional literal upper-bound value emitted alongside the key.

property type: DataFilterType

The DataFilterType discriminator for this filter.

build()

Serialise to a JSON-compatible dict.

Return type:

dict[str, Any]

RegexDataFilter — a DataFilter matching rows by regex.

class RegexDataFilter(path, source_key, value=None)

Match rows whose value at path matches the regex resolved from source_key.

Parameters:

• path (list[str]) – Field path within each source row.

• source_key (Parameter | Calculation) – Model named value supplying the regex pattern.

• value (str | None) – Optional literal regex string emitted alongside the key.

property type: DataFilterType

The DataFilterType discriminator for this filter.

build()

Serialise to a JSON-compatible dict.

Return type:

dict[str, Any]

SetDataFilter — a DataFilter matching rows by set membership.

class SetDataFilter(path, source_keys, values=None)

Match rows whose value at path is in a set of values resolved from source_keys.

Parameters:

• path (list[str]) – Field path within each source row.

• source_keys (list[Parameter | Calculation]) – Model named values supplying the allowed values.

• values (set[str] | None) – Optional literal value set emitted alongside the keys.

property type: DataFilterType

The DataFilterType discriminator for this filter.

build()

Serialise to a JSON-compatible dict.

Return type:

dict[str, Any]

WildcardDataFilter — a DataFilter matching rows by glob pattern.

class WildcardDataFilter(path, source_key, value=None, case_sensitive=False)

Match rows whose value at path matches the glob pattern resolved from source_key.

Parameters:

• path (list[str]) – Field path within each source row.

• source_key (Parameter | Calculation) – Model named value supplying the glob pattern.

• value (str | None) – Optional literal pattern emitted alongside the key.

• case_sensitive (bool) – Whether matching is case sensitive.

property type: DataFilterType

The DataFilterType discriminator for this filter.

build()

Serialise to a JSON-compatible dict.

Return type:

dict[str, Any]