Skip to content

Datasets

Registry operations are explicitly network-capable. Dataset loading validates the generic measurement manifest and returns the same model and measurement objects used elsewhere in the API. See the dataset guide for the on-disk contract and provenance policy.

DatasetRegistryEntry

Metadata and local-cache state for one registered dataset version.

id property

id: str

Return the stable registry identifier.

version property

version: str

Return the source-controlled dataset version.

title property

title: str

Return the human-readable dataset title.

description property

description: str

Return the registry description of the data and acquisition.

format_version property

format_version: int

Return the measurement-manifest format version.

archive_url property

archive_url: str

Return the URL of the downloadable archive.

archive_sha256 property

archive_sha256: str

Return the expected lowercase SHA-256 archive digest.

archive_size_bytes property

archive_size_bytes: int

Return the expected archive size in bytes.

license_spdx property

license_spdx: str

Return the dataset license as an SPDX identifier.

license_url property

license_url: str

Return the URL of the dataset license text.

citation_doi property

citation_doi: str

Return the dataset citation DOI recorded by the registry.

citation_text property

citation_text: str

Return the preferred human-readable citation.

source_url property

source_url: str

Return the authoritative source or landing-page URL.

source_description property

source_description: str

Return provenance details supplied by the dataset publisher.

tags property

tags: list[str]

Return searchable registry tags.

cached property

cached: bool

Return whether a verified local copy is available.

cache_path property

cache_path: Path | None

Return the verified local dataset directory, if cached.

DatasetRegistry

DatasetRegistry(
    *,
    registry_url: str | None = None,
    cache_dir: Path | None = None,
)

Discover, verify, cache, and open datasets from a JSON registry.

Parameters:

Name Type Description Default
registry_url str | None

Registry JSON URL. None selects the packaged default registry.

None
cache_dir Path | None

Managed cache directory. None selects the platform-specific default.

None
Notes

Listing may fetch registry metadata. Download and open may access the network; an already cached dataset is verified before use.

registry_url property

registry_url: str

Return the configured registry URL.

cache_dir property

cache_dir: Path

Return the managed local cache directory.

list

Return registry entries annotated with their local cache state.

download

download(id: str) -> Path

Download, checksum-verify, extract, and return one dataset directory.

download_all

download_all() -> list[Path]

Download and verify every registry entry, returning local directories.

open

open(id: str) -> Dataset

Open a cached dataset, downloading it first when necessary.

clean

clean(id: str) -> bool

Remove one cached dataset and return whether it existed.

clean_all

clean_all() -> int

Remove all managed datasets and return the number removed.

open_spectral

open_spectral(id: str) -> SpectralDataset

Open an explicit version-two registry ID through verified downloads/cache.

Parameters:

Name Type Description Default
id str

Registered spectral dataset ID. This explicit operation may access the network and releases the GIL. Version-one profiles require open instead.

required

Returns:

Type Description
SpectralDataset

Validated resident channels and scalar frames; failures raise DatasetError.

Dataset

Loaded measurements, models, optional truth, and provenance.

Dataset loading validates the manifest and array shapes before returning this object. Arrays are exposed using the same conventions as the core API.

path property

path: Path | None

Return the local dataset directory, if it originated on disk.

measurements property

measurements: MeasurementStack

Return the resident measured-intensity stack.

true_model property

true_model: ImagePlaneModel

Return the acquisition model, including known simulated errors.

reconstruction_model property

reconstruction_model: ImagePlaneModel

Return the model intended for reconstruction.

ground_truth_object property

ground_truth_object: ComplexArray | None

Return the complex ground-truth field when supplied by the dataset.

valid_object_mask property

valid_object_mask: MaskArray | None

Return the optional valid-pixel mask for object-space evaluation.

provenance property

provenance: Mapping[str, str]

Return immutable acquisition and source provenance fields.

measurement_units property

measurement_units: str | None

Return the declared units of measured intensities, if present.

reconstruction_problem

reconstruction_problem() -> ReconstructionProblem

Build a validated problem from the dataset measurements and model.

subset

subset(
    *,
    frames: list[int] | None = None,
    every_nth_frame: int | None = None,
    crop: tuple[int, int, int, int] | None = None,
) -> DatasetSubset

Resolve an owned frame selection and detector crop, releasing the GIL.

Parameters:

Name Type Description Default
frames list[int] | None

Unique zero-based acquisition indices in the desired order; an empty or out-of-range selection raises DatasetError.

None
every_nth_frame int | None

Positive step selecting 0, step, 2*step, ...; mutually exclusive with frames. Omit both to retain every frame.

None
crop tuple[int, int, int, int] | None

Zero-based (row, column, height, width) detector rectangle. Sizes must be positive, contained in the original image, and map to integer reconstruction pixels; otherwise DatasetError. Omit to resolve the full original image rectangle.

None

Returns:

Type Description
DatasetSubset

Owned measurements, updated compiled models, cropped truth/mask, and unchanged original dataset provenance and intensity units.

DatasetSubset

Resolved ordinary dataset selection returned by Dataset.subset.

Measurements and models retain original acquisition identities for benchmarking. NumPy array getters return writable C-contiguous copies: truth is complex128 and its mask is uint8, both in (row, column) order.

spatial_crop property

spatial_crop: tuple[int, int, int, int]

Resolved (row, column, height, width) in original detector pixels.

measurements property

measurements: MeasurementStack

Owned selected stack with original frame and illumination metadata.

true_model property

true_model: ImagePlaneModel

Acquisition model recompiled for the selection and cropped grids.

reconstruction_model property

reconstruction_model: ImagePlaneModel

Assumed model recompiled for the selection and cropped grids.

ground_truth_object property

ground_truth_object: ComplexArray | None

Writable copy of cropped complex truth, or None if unavailable.

valid_object_mask property

valid_object_mask: MaskArray | None

Writable copy of the cropped binary truth mask, when supplied.

provenance property

provenance: Mapping[str, str]

New mapping of the original dataset's source provenance.

measurement_units property

measurement_units: str | None

Unchanged declared intensity units, if supplied.

reconstruction_problem

reconstruction_problem() -> ReconstructionProblem

Clone the selected stack and assumed model into a validated problem.

open_dataset

open_dataset(
    id: str,
    *,
    registry_url: str | None = None,
    cache_dir: Path | None = None,
) -> Dataset

Open a registered dataset using a temporary DatasetRegistry.

Parameters are equivalent to DatasetRegistry; id is the exact registry identifier. The dataset is downloaded if no verified cached copy is available.

SpectralDataset

Resident version-two spectral dataset with explicit external conversion metadata.

Scalar detector frames remain grayscale. Channel IDs, vacuum wavelengths, response provenance and sparse exposure ordering are explicit and validated. No registration, resampling or RGB demixing is inferred. Arrays are writable copies.

path instance-attribute

path: Path

Local converted bundle directory from which data were loaded.

measurements instance-attribute

measurements: MeasurementStack

Copied resident scalar detector stack including masks and weights.

model instance-attribute

Owned ordered compiled kernels sharing a validated common grid.

response_provenance instance-attribute

response_provenance: dict[str, str]

Required response/weight/calibration origins keyed by stable channel ID.

provenance instance-attribute

provenance: dict[str, str]

Converter-supplied dataset provenance key/value metadata.

measurement_units instance-attribute

measurement_units: str | None

Optional semantic detector units such as camera counts.

ground_truth_objects instance-attribute

ground_truth_objects: list[ComplexArray | None]

Optional copied complex128 common-grid truth in stable channel order.

valid_object_masks instance-attribute

valid_object_masks: list[MaskArray | None]

Optional copied binary uint8 truth-validity masks in channel order.

reconstruction_problem

reconstruction_problem() -> SpectralReconstructionProblem

Copy resident data and compiled channels into an owned spectral problem.

load_spectral_dataset

load_spectral_dataset(*, path: Path) -> SpectralDataset

Load an explicitly converted version-two profile entirely offline.

Parameters:

Name Type Description Default
path Path

Local directory containing dataset.json, compiled kernels and scalar frames. Validates safe paths including symlink containment, explicit wavelengths, stable channel order, sparse references and response provenance.

required

Returns:

Type Description
SpectralDataset

Resident data, model and optional per-channel truth. File work releases the GIL; invalid profiles and file failures raise DatasetError. Ordinary version-one data requires the existing single-wavelength loader.