Optimization

Parameter optimization for synthetic microscope image generation.

This module optimizes the selected parameters to match real microscope images using a weighted combination of:

  • Color/Intensity Distribution (histogram L1 distance)

  • SSIM (Structural Similarity Index)

  • PSNR (Peak Signal-to-Noise Ratio)

Uses scipy.optimize.differential_evolution with multi-threading for efficient global optimization across multiple CPU cores.

class DECheckpointManager(checkpoint_dir: Path, run_start_time: str = None)

Bases: object

Manages checkpointing for differential evolution optimization.

Saves population state after each iteration to allow resuming if the process is killed.

save_checkpoint(iteration: int, population: ndarray, population_energies: ndarray, best_x: ndarray, best_fun: float, convergence: float, config: dict) → Path

Save current optimization state to a checkpoint file.

Parameters:

  • iteration (int) – Current iteration number.

  • population (np.ndarray) – Current population array.

  • population_energies (np.ndarray) – Energy (loss) values for each member.

  • best_x (np.ndarray) – Best parameter vector found so far.

  • best_fun (float) – Best loss value found so far.

  • convergence (float) – Current convergence metric.

  • config (dict) – Optimization configuration for validation on resume.

Returns:

Path to the saved checkpoint file.

Return type:

Path

find_latest_checkpoint() → Path

Find the most recent checkpoint file by modification time.

Returns:

Path to the latest checkpoint, or None if no checkpoints exist.

Return type:

Path

load_checkpoint(checkpoint_path: Path = None) → dict

Load optimization state from a checkpoint file.

Parameters:

checkpoint_path (Path) – Path to checkpoint file. If None, loads the latest checkpoint.

Returns:

Checkpoint data containing population, energies, best solution,

and config. Returns None if no checkpoint found or loading fails.

Return type:

dict

validate_checkpoint(checkpoint: dict, current_config: dict) → bool

Validate that a checkpoint is compatible with the current configuration.

Parameters:

  • checkpoint (dict) – Loaded checkpoint data.

  • current_config (dict) – Current optimization configuration.

Returns:

True if the checkpoint is compatible, False otherwise.

Return type:

bool

class ObjectiveFunction(image_pairs, weights, temp_base_dir, n_vertices, region_weights)

Bases: object

Picklable objective function for differential evolution.

Wraps the synthetic image generation and metric computation into a callable that can be serialized for multiprocessing.

find_real_images(input_dir: Path, limit: int = None) → List[Tuple[Path, Path]]

Find all real microscope images and their corresponding masks.

Searches recursively for .tif files and matches them with their corresponding _masks.tif files. Skips files prefixed with syn_.

Parameters:

  • input_dir (Path) – Directory to search for image pairs.

  • limit (int) – Maximum number of pairs to return. If None, returns all.

Returns:

List of (image_path, mask_path) tuples.

Return type:

List[Tuple[Path, Path]]

compute_weighted_loss(metrics: Dict, weights: Dict, metrics_bg: Dict | None = None, metrics_fg: Dict | None = None, region_weights: Dict | None = None) → float

Compute weighted loss from metrics dictionary.

Combines histogram distance, SSIM, and PSNR into a single scalar loss. Optionally computes region-specific losses for background and foreground.

Parameters:

  • metrics (Dict) – Full image metrics from compute_all_metrics().

  • weights (Dict) – Weights for each metric (‘histogram_distance’, ‘ssim’, ‘psnr’).

  • metrics_bg (Dict) – Background region metrics. If None, uses full image only.

  • metrics_fg (Dict) – Foreground region metrics. If None, uses full image only.

  • region_weights (Dict) – Weights for ‘background’ and ‘foreground’ regions.

Returns:

Weighted loss value (lower is better).

Return type:

float

extract_masked_region(image: ndarray, mask: ndarray, region: str) → ndarray

Extract foreground or background region from image using mask.

Parameters:

  • image (np.ndarray) – Input image (float [0,1]).

  • mask (np.ndarray) – Segmentation mask (2D or 3D).

  • region (str) – Region to extract: ‘foreground’ or ‘background’.

Returns:

Image with the non-selected region zeroed out.

Return type:

np.ndarray

optimize_parameters(image_pairs: List[Tuple[Path, Path]], bounds: List[Tuple[float, float]], weights: Dict, region_weights: Dict, maxiter: int = 50, popsize: int = 15, workers: int = -1, seed: int = 42, n_vertices: int = 8, resume: bool = False, no_checkpoint: bool = False) → Dict

Optimize synthetic image parameters using differential evolution.

Minimizes a weighted loss combining histogram distance, SSIM, and PSNR across all image pairs. Supports checkpointing and resuming.

Parameters:

  • image_pairs (List[Tuple[Path, Path]]) – List of (image_path, mask_path) tuples.

  • bounds (List[Tuple[float, float]]) – Parameter bounds for each dimension.

  • weights (Dict) – Metric weights for loss computation.

  • region_weights (Dict) – Weights for background and foreground regions.

  • maxiter (int) – Maximum number of iterations.

  • popsize (int) – Population size multiplier (population = popsize * num_params).

  • workers (int) – Number of worker processes (-1 = all CPUs, 1 = sequential).

  • seed (int) – Random seed for reproducibility.

  • n_vertices (int) – Number of vertices for cell shape extraction.

  • resume (bool) – If True, resume from latest checkpoint.

  • no_checkpoint (bool) – If True, disable checkpointing.

Returns:

Optimization results including ‘parameters’, ‘optimization_info’,

’bounds’, and ‘weights’.

Return type:

Dict

compute_final_metrics(image_pairs: List[Tuple[Path, Path]], params: Dict, output_dir: Path, n_vertices: int) → Tuple[Dict, List[Dict]]

Compute final metrics for all images with optimized parameters.

Generates synthetic images for each pair using the optimized parameters and computes full, foreground, and background metrics.

Parameters:

  • image_pairs (List[Tuple[Path, Path]]) – List of (image_path, mask_path) tuples.

  • params (Dict) – Optimized parameter dictionary.

  • output_dir (Path) – Output directory for temporary files.

  • n_vertices (int) – Number of vertices for cell shape extraction.

Returns:

(average_metrics, per_image_metrics) where

average_metrics contains mean values and per_image_metrics is a list of per-image metric dictionaries.

Return type:

Tuple[Dict, List[Dict]]

save_results(results: Dict, avg_metrics: Dict, per_image_metrics: List[Dict], output_dir: Path)

Save optimization results to JSON and CSV files.

Parameters:

  • results (Dict) – Optimization results from optimize_parameters().

  • avg_metrics (Dict) – Average metrics from compute_final_metrics().

  • per_image_metrics (List[Dict]) – Per-image metrics list.

  • output_dir (Path) – Output directory for saving files.

generate_all_synthetics(image_pairs: List[Tuple[Path, Path]], params: Dict, output_dir: Path, n_vertices: int)

Generate synthetic images for all image pairs with optimized parameters.

Parameters:

  • image_pairs (List[Tuple[Path, Path]]) – List of (image_path, mask_path) tuples.

  • params (Dict) – Optimized parameter dictionary.

  • output_dir (Path) – Output directory for saving synthetic images.

  • n_vertices (int) – Number of vertices for cell shape extraction.