tidy3d.web.api.container.Batch

tidy3d.web.api.container.Batch#

class Batch[source]#

Bases: WebContainer

Interface for submitting several Simulation objects to sever.

Parameters:

Notes

Commonly one needs to submit a batch of Simulation. The built-in Batch object is the best way to upload, start, monitor, and load a series of tasks. The batch object is like a Job, but stores task metadata for a series of simulations.

See also

tidy3d.web.api.webapi.run_async()

Submits a set of Simulation objects to server, starts running, monitors progress, downloads, and loads results as a BatchData object.

Job:

Interface for managing the running of a Simulation on server.

Notebooks

Attributes

jobs

Create a series of tasks in the Batch and upload them to server.

num_jobs

Number of jobs in the batch.

attrs

Methods

delete()

Delete server-side data associated with each task in the batch.

download([path_dir, replace_existing])

Download results of each task.

estimate_cost([verbose])

Compute the maximum FlexCredit charge for a given Batch.

get_info()

Get information about each task in the Batch.

get_run_info()

get information about a each of the tasks in the Batch.

load([path_dir, replace_existing, skip_download])

Download results and load them into BatchData object.

monitor(*[, download_on_success, path_dir, ...])

Monitor progress of each running task.

postprocess_start([worker_group, verbose])

Start the postprocess phase for all applicable jobs in the batch.

real_cost([verbose])

Get the sum of billed costs for each task associated with this batch.

run([path_dir, priority])

Upload and run each simulation in Batch.

start([priority])

Start running all tasks in the Batch.

to_file(fname)

Exports Tidy3dBaseModel instance to .yaml, .json, or .hdf5 file

upload()

Upload a series of tasks associated with this Batch using multi-threading.

Inherited Common Usage

simulations#
folder_name#
verbose#
solver_version#
callback_url#
simulation_type#
parent_tasks#
num_workers#
reduce_simulation#
pay_type#
jobs_cached#
lazy#
run(path_dir='.', priority=None)[source]#

Upload and run each simulation in Batch.

Parameters:

  • path_dir (str) – Base directory where data will be downloaded, by default current working directory.

  • priority (int = None) – Priority of the simulation in the Virtual GPU (vGPU) queue (1 = lowest, 10 = highest). It affects only simulations from vGPU licenses and does not impact simulations using FlexCredits.

Returns:

Contains Union[SimulationData, HeatSimulationData, EMESimulationData] for each Union[:class:.Simulation`, HeatSimulation, EMESimulation] in Batch.

Return type:

BatchData

Note

A typical usage might look like:

>>> from tidy3d.web.api.container import Batch
>>> custom_batch = Batch()
>>> batch_data = custom_batch.run() 
>>> for task_name, sim_data in batch_data.items(): 
...     # do something with data.

batch_data does not store all of the data objects in memory, rather it iterates over the task names and loads the corresponding data from file one by one. If no file exists for that task, it downloads it.

property jobs#

Create a series of tasks in the Batch and upload them to server.

Note

To start the simulations running, must call Batch.start() after uploaded.

to_file(fname)[source]#

Exports Tidy3dBaseModel instance to .yaml, .json, or .hdf5 file

Parameters:

fname (str) – Full path to the .yaml or .json file to save the Tidy3dBaseModel to.

Example

>>> simulation.to_file(fname='folder/sim.json')
property num_jobs#

Number of jobs in the batch.

upload()[source]#

Upload a series of tasks associated with this Batch using multi-threading.

get_info()[source]#

Get information about each task in the Batch.

Returns:

Mapping of task name to data about task associated with each task.

Return type:

Dict[str, TaskInfo]

start(priority=None)[source]#

Start running all tasks in the Batch.

Parameters:

priority (int = None) – Priority of the simulation in the Virtual GPU (vGPU) queue (1 = lowest, 10 = highest). It affects only simulations from vGPU licenses and does not impact simulations using FlexCredits.

Note

To monitor the running simulations, can call Batch.monitor().

get_run_info()[source]#

get information about a each of the tasks in the Batch.

Returns:

Dict[str – Maps task names to run info for each task in the Batch.

Return type:

RunInfo]

postprocess_start(worker_group=None, verbose=True)[source]#

Start the postprocess phase for all applicable jobs in the batch.

This simply forwards to each Job’s postprocess_start(…). The Job decides whether it’s a Component Modeler task and whether it can/should start now. This method does not wait for postprocessing to finish.

monitor(*, download_on_success=False, path_dir='.', replace_existing=False, postprocess_worker_group=None)[source]#

Monitor progress of each running task.

  • For Component Modeler jobs, automatically triggers postprocessing once run finishes.

  • Optionally downloads results as soon as a job reaches final success.

  • Rich progress bars in verbose mode; quiet polling otherwise.

Parameters:

  • download_on_success (bool = False) – If True, automatically start downloading the results for a job as soon as it reaches success.

  • path_dir (str = './') – Base directory where data will be downloaded, by default the current working directory. Only used when download_on_success is True.

  • replace_existing (bool = False) – Downloads the data even if path exists (overwriting the existing). Only used when download_on_success is True.

download(path_dir='.', replace_existing=False)[source]#

Download results of each task.

Parameters:

  • path_dir (str = './') – Base directory where data will be downloaded, by default the current working directory.

  • replace_existing (bool = False) – Downloads the data even if path exists (overwriting the existing).

Note

To load and iterate through the data, use Batch.items().

The data for each task will be named as {path_dir}/{task_id}.hdf5. The Batch hdf5 file will be automatically saved as {path_dir}/batch.hdf5, allowing one to load this Batch later using batch = Batch.from_file().

load(path_dir='.', replace_existing=False, skip_download=False)[source]#

Download results and load them into BatchData object.

Parameters:

  • path_dir (str = './') – Base directory where data will be downloaded, by default current working directory.

  • replace_existing (bool = False) – Downloads the data even if path exists (overwriting the existing).

  • skip_download (bool = False) – Does not trigger download. Should be True if already downloaded.

Returns:

delete()[source]#

Delete server-side data associated with each task in the batch.

real_cost(verbose=True)[source]#

Get the sum of billed costs for each task associated with this batch.

Parameters:

verbose (bool = True) – Whether to log the cost and helpful messages.

Returns:

Billed cost for the entire Batch.

Return type:

float

estimate_cost(verbose=True)[source]#

Compute the maximum FlexCredit charge for a given Batch.

Parameters:

verbose (bool = True) – Whether to log the cost and helpful messages.

Note

Cost is calculated assuming the simulation runs for the full run_time. If early shut-off is triggered, the cost is adjusted proportionately.

Returns:

Estimated total cost of the tasks in FlexCredits.

Return type:

float

__hash__()#

Hash method.