# Web API#

Run this notebook in your browser using Binder.

This notebook is a tutorial of the API used for submitting simulations to our servers.

[1]:

import tidy3d as td


## Setup#

Let’s set up a simple simulation to get started.

[2]:

# set up parameters of simulation
dl = 0.05
pml = td.PML()
sim_size = [4, 4, 4]
freq0 = 3e14
fwidth = 1e13
run_time = 1/fwidth

# create structure
dielectric = td.Medium.from_nk(n=2, k=0, freq=freq0)
square = td.Structure(
geometry=td.Box(center=[0, 0, 0], size=[1.5, 1.5, 1.5]),
medium=dielectric)

# create source
source = td.UniformCurrentSource(
center=(-1.5, 0, 0),
size=(0, 0.4, 0.4),
source_time = td.GaussianPulse(
freq0=freq0,
fwidth=fwidth),
polarization='Ex')

# create monitor
monitor = td.FieldMonitor(
fields=['Ex', 'Ey', 'Ez'],
center=(0, 0, 0),
size=(td.inf, td.inf, 0),
freqs=[freq0],
name='field')

# Initialize simulation
sim = td.Simulation(
size=sim_size,
grid_spec=td.GridSpec.uniform(dl),
structures=[square],
sources=[source],
monitors=[monitor],
run_time=run_time,
pml_layers=(pml, pml, pml),
)


## Running simulation manually#

For the most control, you can run the simulation through the Tidy3D web API. Each simulation running on the server is identified by a task_id, which must be specified in the web API. Let’s walk through submitting a simulation this way.

[3]:

import tidy3d.web as web

# upload the simulation to our servers

# start the simulation running

# monitor the simulation, dont move on to next line until completed.


[16:17:13] INFO     Using Tidy3D credentials from stored file                      auth.py:74

[16:17:16] INFO     Uploaded task 'webAPI' with task_id                         webapi.py:120
'407edea5-e177-4e19-b427-989994d4238b'.

[16:17:20] INFO     Maximum flex unit cost: 0.20                                webapi.py:253

           INFO     status = queued                                             webapi.py:262

[16:17:28] INFO     status = preprocess                                         webapi.py:274

[16:17:43] INFO     starting up solver                                          webapi.py:278

[16:18:09] INFO     running solver                                              webapi.py:284

           INFO     status = postprocess                                        webapi.py:301

[16:18:23] INFO     status = success                                            webapi.py:307

[16:18:24] INFO     downloading file "monitor_data.hdf5" to "data/sim.hdf5"     webapi.py:537

[16:18:25] INFO     loading SimulationData from data/sim.hdf5                   webapi.py:369

           WARNING  Simulation final field decay value of 0.336 is greater than webapi.py:375
the simulation shutoff threshold of 1e-05. Consider
simulation again with large run_time duration for more
accurate results.


While we broke down each of the individual steps above, one can also perform the entire process in one line by calling the web.run() function as follows.

sim_data = web.run(sim, task_name='webAPI', path='data/sim.hdf5')


(We won’t run it again in this notebook to save time).

Sometimes this is more convenient, but other times it can be helpful to have the steps broken down one by one, for example if the simulation is long, you may want to web.start and then exit the session and load the results at a later time using web.load.

## Job Container#

The web.Job interface provides a more convenient way to manage single simuations, mainly because it eliminates the need for keeping track of the task_id and original Simulation.

While Job has methods with names identical to the web API functions above, which give some more granular control, it is often most convenient to call [.run()] so we will do that now.

[4]:

# set the logging level to warning to reduce downloading messages.
td.set_logging_level('warning')

# initializes job, puts task on server (but doesnt run it)

# start job, monitor, and load results
sim_data = job.run(path='data/sim.hdf5')

[16:19:37] WARNING  Simulation final field decay value of 0.336 is greater than webapi.py:375
the simulation shutoff threshold of 1e-05. Consider
simulation again with large run_time duration for more
accurate results.


Another convenient thing about Job objects is that they can be saved and loaded just like other Tidy3d components.

This is convenient if you want to save and load up the results of a job that has already finished, without needing to know the task_id.

[5]:

# saves the job metadata to a single file
job.to_file('data/job.json')

# can exit session, break here, or continue in new session.


[16:19:39] WARNING  Simulation final field decay value of 0.336 is greater than webapi.py:375
the simulation shutoff threshold of 1e-05. Consider
simulation again with large run_time duration for more
accurate results.


## Batch Processing#

Commonly one needs to submit a batch of Simulations. One way to approach this using the web API is to upload, start, monitor, and load a series of tasks one by one, but this is clumsy and you can lose your data if the session gets interrupted.

A better way is to use the built-in Batch object. The batch object is like a Job, but stores task metadata for a series of simulations.

[6]:

# make a dictionary of  {task name : simulation} for demonstration
sims = {f'sim_{i}': sim for i in range(3)}

# initialize a batch and run them all
batch = web.Batch(simulations=sims)

# run the batch and store all of the data in the data/ dir.
batch_results = batch.run(path_dir='data')

[16:19:51] Started working on Batch.                                         container.py:361

[16:21:54] Batch complete.                                                   container.py:382


When the batch is completed, the output is not a SimulationData but rather a BatchData. The data within this BatchData object can either be indexed directly batch_results[task_name] or can be looped through batch_results.items() to get the SimulationData for each task.

This was chosen to reduce the memory strain from loading all SimulationData objects at once.

Alternatively, the batch can be looped through (several times) using the .items() method, similar to a dictionary.

[7]:

# grab the sum of intensities in the simulation one by one (to save memory)
intensities = {}
field_data = sim_data.at_centers('field').sel(f=freq0)
intensity = abs(field_data.Ex)**2 + abs(field_data.Ey)**2 + abs(field_data.Ez)**2
sum_intensity = float(intensity.sum(('x', 'y')).values)

print(intensities)

[16:21:57] WARNING  Simulation final field decay value of 0.336 is greater than webapi.py:375
the simulation shutoff threshold of 1e-05. Consider
simulation again with large run_time duration for more
accurate results.

[16:21:59] WARNING  Simulation final field decay value of 0.336 is greater than webapi.py:375
the simulation shutoff threshold of 1e-05. Consider
simulation again with large run_time duration for more
accurate results.

[16:22:00] WARNING  Simulation final field decay value of 0.336 is greater than webapi.py:375
the simulation shutoff threshold of 1e-05. Consider
simulation again with large run_time duration for more
accurate results.

{'sim_0': 12626395.288917286, 'sim_1': 12626391.910953935, 'sim_2': 12626395.288917286}