Datasets¶
Registry operations are explicitly network-capable. Dataset loading validates the generic measurement manifest and returns the same model and measurement objects used elsewhere in the API. See the dataset guide for the on-disk contract and provenance policy.
DatasetRegistryEntry
¶
DatasetRegistry
¶
DatasetRegistry(
*,
registry_url: str | None = None,
cache_dir: Path | None = None,
)
Discover, verify, cache, and open datasets from a JSON registry.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
registry_url
|
str | None
|
Registry JSON URL. |
None
|
cache_dir
|
Path | None
|
Managed cache directory. |
None
|
Notes
Listing may fetch registry metadata. Download and open may access the
network; an already cached dataset is verified before use.
list
¶
list() -> list[DatasetRegistryEntry]
Return registry entries annotated with their local cache state.
download
¶
download(id: str) -> Path
Download, checksum-verify, extract, and return one dataset directory.
download_all
¶
Download and verify every registry entry, returning local directories.
open_spectral
¶
open_spectral(id: str) -> SpectralDataset
Open an explicit version-two registry ID through verified downloads/cache.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Registered spectral dataset ID. This explicit operation may access the network and releases the GIL. Version-one profiles require open instead. |
required |
Returns:
| Type | Description |
|---|---|
SpectralDataset
|
Validated resident channels and scalar frames; failures raise DatasetError. |
Dataset
¶
Loaded measurements, models, optional truth, and provenance.
Dataset loading validates the manifest and array shapes before returning this object. Arrays are exposed using the same conventions as the core API.
measurements
property
¶
measurements: MeasurementStack
Return the resident measured-intensity stack.
true_model
property
¶
true_model: ImagePlaneModel
Return the acquisition model, including known simulated errors.
reconstruction_model
property
¶
reconstruction_model: ImagePlaneModel
Return the model intended for reconstruction.
ground_truth_object
property
¶
ground_truth_object: ComplexArray | None
Return the complex ground-truth field when supplied by the dataset.
valid_object_mask
property
¶
valid_object_mask: MaskArray | None
Return the optional valid-pixel mask for object-space evaluation.
provenance
property
¶
provenance: Mapping[str, str]
Return immutable acquisition and source provenance fields.
measurement_units
property
¶
measurement_units: str | None
Return the declared units of measured intensities, if present.
reconstruction_problem
¶
reconstruction_problem() -> ReconstructionProblem
Build a validated problem from the dataset measurements and model.
subset
¶
subset(
*,
frames: list[int] | None = None,
every_nth_frame: int | None = None,
crop: tuple[int, int, int, int] | None = None,
) -> DatasetSubset
Resolve an owned frame selection and detector crop, releasing the GIL.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
frames
|
list[int] | None
|
Unique zero-based acquisition indices in the desired order; an
empty or out-of-range selection raises |
None
|
every_nth_frame
|
int | None
|
Positive step selecting 0, step, 2*step, ...; mutually exclusive
with |
None
|
crop
|
tuple[int, int, int, int] | None
|
Zero-based |
None
|
Returns:
| Type | Description |
|---|---|
DatasetSubset
|
Owned measurements, updated compiled models, cropped truth/mask, and unchanged original dataset provenance and intensity units. |
DatasetSubset
¶
Resolved ordinary dataset selection returned by Dataset.subset.
Measurements and models retain original acquisition identities for benchmarking. NumPy array getters return writable C-contiguous copies: truth is complex128 and its mask is uint8, both in (row, column) order.
spatial_crop
property
¶
spatial_crop: tuple[int, int, int, int]
Resolved (row, column, height, width) in original detector pixels.
measurements
property
¶
measurements: MeasurementStack
Owned selected stack with original frame and illumination metadata.
true_model
property
¶
true_model: ImagePlaneModel
Acquisition model recompiled for the selection and cropped grids.
reconstruction_model
property
¶
reconstruction_model: ImagePlaneModel
Assumed model recompiled for the selection and cropped grids.
ground_truth_object
property
¶
ground_truth_object: ComplexArray | None
Writable copy of cropped complex truth, or None if unavailable.
valid_object_mask
property
¶
valid_object_mask: MaskArray | None
Writable copy of the cropped binary truth mask, when supplied.
provenance
property
¶
provenance: Mapping[str, str]
New mapping of the original dataset's source provenance.
measurement_units
property
¶
measurement_units: str | None
Unchanged declared intensity units, if supplied.
reconstruction_problem
¶
reconstruction_problem() -> ReconstructionProblem
Clone the selected stack and assumed model into a validated problem.
open_dataset
¶
open_dataset(
id: str,
*,
registry_url: str | None = None,
cache_dir: Path | None = None,
) -> Dataset
Open a registered dataset using a temporary
DatasetRegistry.
Parameters are equivalent to DatasetRegistry;
id is the exact
registry identifier. The dataset is downloaded if no verified cached copy
is available.
SpectralDataset
¶
Resident version-two spectral dataset with explicit external conversion metadata.
Scalar detector frames remain grayscale. Channel IDs, vacuum wavelengths, response provenance and sparse exposure ordering are explicit and validated. No registration, resampling or RGB demixing is inferred. Arrays are writable copies.
measurements
instance-attribute
¶
measurements: MeasurementStack
Copied resident scalar detector stack including masks and weights.
model
instance-attribute
¶
model: SpectralImagePlaneModel
Owned ordered compiled kernels sharing a validated common grid.
response_provenance
instance-attribute
¶
response_provenance: dict[str, str]
Required response/weight/calibration origins keyed by stable channel ID.
provenance
instance-attribute
¶
provenance: dict[str, str]
Converter-supplied dataset provenance key/value metadata.
measurement_units
instance-attribute
¶
measurement_units: str | None
Optional semantic detector units such as camera counts.
ground_truth_objects
instance-attribute
¶
ground_truth_objects: list[ComplexArray | None]
Optional copied complex128 common-grid truth in stable channel order.
valid_object_masks
instance-attribute
¶
valid_object_masks: list[MaskArray | None]
Optional copied binary uint8 truth-validity masks in channel order.
reconstruction_problem
¶
reconstruction_problem() -> SpectralReconstructionProblem
Copy resident data and compiled channels into an owned spectral problem.
load_spectral_dataset
¶
load_spectral_dataset(*, path: Path) -> SpectralDataset
Load an explicitly converted version-two profile entirely offline.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
Path
|
Local directory containing dataset.json, compiled kernels and scalar frames. Validates safe paths including symlink containment, explicit wavelengths, stable channel order, sparse references and response provenance. |
required |
Returns:
| Type | Description |
|---|---|
SpectralDataset
|
Resident data, model and optional per-channel truth. File work releases the GIL; invalid profiles and file failures raise DatasetError. Ordinary version-one data requires the existing single-wavelength loader. |