Design Space Exploration Plugin#
Basics#
The design plugin tidy3d.plugins.design is a wrapper that makes it simple and convenient for tidy3d users to define experiments to systematically explore a design space.
In short, users define the dimensions of their design problem, as well as the method they wish to explore the design space. These specifications are combined in a DesignSpace object.
Then, the user passes a function that defines the input / output relationship they wish to explore. The function arguments correspond to the dimensions defined in the DesignSpace and the function outputs can be anything.
The resulting data is stored as a Result object, which has its own methods but can be easily converted to a pandas.DataFrame for analysis, post processing, or visualization. The columns in this DataFrame correspond to the function inputs and outputs and each data point corresponds to a row in this DataFrame.
For reference, here is a diagram of the entire process.
For another example of the design plugin used in practice, see about halfway down our parameter scan notebook.
Note: this feature is in pre-release stage. if you have questions or suggestions for the API, please feel free to make an issue on our github page.
The plugin is imported from tidy3d.plugins.design so it’s convenient to import this namespace first along with the other packages we need.
[1]:
import numpy as np
import matplotlib.pylab as plt
import typing
import tidy3d as td
import tidy3d.web as web
import tidy3d.plugins.design as tdd
Quickstart#
While the design plugin is built on Tidy3D and has special features for handling FDTD simulations, it can also be run with any general design problem. Here is a quick, complete example before diving into the use of the plugin with Tidy3D simulations.
Here we want to sample a design space with two dimensions x and y with Monte Carlo. Our function simply returns a Gaussian centered at (0,0) and we’ll evaluate the result for Gaussians with 3 different variances.
[2]:
# define your design space (parameters (x,y) within certain ranges)
param_x = tdd.ParameterFloat(name="x", span=(-15,15))
param_y = tdd.ParameterFloat(name="y", span=(-15,15))
# define your sampling method, Monte Carlo method with 10,000 points
method = tdd.MethodMonteCarlo(num_points=10000)
# put everything together in a `DesignSpace` container
design_space = tdd.DesignSpace(method=method, parameters=[param_x, param_y])
# define your figure of merit function. Here we compute a gaussian as a function of x and y with different values for the width.
sigmas = {f"sigma={s}": s for s in [5, 10, 20]}
def f(x:float, y:float) -> typing.Dict[str, float]:
"""gaussian distribution as a function of x and y."""
r2 = x**2 + y**2
gaussian = lambda sigma: np.exp(-r2 / sigma ** 2)
# return a dictionary, where the key is used to label the output names
return {key: gaussian(sigma) for key, sigma in sigmas.items()}
# note, we return a dictionary here, they keys correspond to our output data labels
# in the future, this function will involve running tidy3d simulations
# .run the design on our function and convert the results to a pandas.DataFrame
df = design_space.run(f).to_dataframe()
# plot the results using pandas.DataFrame builtins
f, axes = plt.subplots(1, 3, figsize=(10, 3), tight_layout=True)
for ax, C in zip(axes, sigmas.keys()):
_ = df.plot.hexbin(x='x', y='y', gridsize=20, C=C, cmap="magma", ax=ax)
ax.set_title(C)
[3]:
# look at the first 5 outputs
df.head()
[3]:
| x | y | sigma=5 | sigma=10 | sigma=20 | |
|---|---|---|---|---|---|
| 0 | -6.370582 | -4.134946 | 0.099531 | 0.561681 | 0.865710 |
| 1 | 5.074127 | -11.570364 | 0.001687 | 0.202665 | 0.670957 |
| 2 | -12.986654 | -6.476491 | 0.000220 | 0.121726 | 0.590671 |
| 3 | -6.148295 | -5.722131 | 0.059500 | 0.493890 | 0.838315 |
| 4 | 7.761197 | -14.456069 | 0.000021 | 0.067736 | 0.510158 |
Design Space Exploration in Tidy3D#
This is essentially the same process that we will use to perform a parameter sweep of a photonic device. The only difference is that our function evaluation each point will involve a tidy3d simulation.
For a concrete example, let’s analyze the transmission of a 1D multilayer slab. Our system will have num layers with refractive index alternating between two values, each with thickness t. We write a function to compute the transmitted flux through this system at frequency freq0 as a function of num and t.
[4]:
lambda0 = 0.63
freq0 = td.C_0 / lambda0
mnt_name = "flux"
buffer = 1.5 * lambda0
refractive_indices = [1.5, 1.2]
Let’s write our measurement function, for convenience, we’ll break it into pre-processing and post-processing functions. The pre-processing function returns a tidy3d.Simulation as a function of the input parameters and the post-processing function returns the transmitted flux as a function of the tidy3d.SimulationData associated with the simulation.
[5]:
def pre(num: int, t: float) -> td.Simulation:
"""Pre-processing function, which creates a tidy3d simulation given the function inputs."""
layers = []
z = 0
for i in range(num):
n = refractive_indices[i % len(refractive_indices)]
thickness = t * lambda0 / n
medium = td.Medium(permittivity = n ** 2)
geometry = td.Box(center=(0, 0, z + thickness / 2), size=(td.inf, td.inf, thickness))
layers.append(td.Structure(geometry=geometry, medium=medium))
z += thickness
Lz = z + 2 * buffer
sim = td.Simulation(
size=(0, 0, Lz),
center=(0, 0, z / 2),
structures=layers,
sources=[td.PlaneWave(
size=(td.inf, td.inf, 0),
center=(0, 0, -buffer * 0.75),
source_time=td.GaussianPulse(freq0=freq0, fwidth=freq0/10),
direction="+",
)],
monitors=[td.FluxMonitor(
size=(td.inf, td.inf, 0),
center=(0, 0, z + buffer * 0.75),
freqs=[freq0],
name=mnt_name,
)],
boundary_spec=td.BoundarySpec.pml(x=False, y=False, z=True),
run_time=100 / freq0,
)
return sim
def post(data: td.SimulationData) -> dict:
"""Post-processing function, which processes the tidy3d simulation data to return the function output."""
flux = np.sum(data['flux'].flux.values)
return {"flux": flux}
Note: if we were to write the full transmission function using pre and post it would look like the function below. However, the sweep plugin will handle running the simulation for us, as we’ll see later.
def transmission(num: int, t: float) -> float:
"""Transmission as a function of number of layers and their thicknesses.
Note: not used, just a demonstration of how the pre and post functions are related.
"""
sim = pre(num=num, t=t)
data = web.run(sim, task_name=f"num={num}_t={t}")
return post(data=data)
Note: we choose to have our post-processing function return a dictionary
{"flux": flux}instead of just afloat. If a dictionary is returned, thedesignplugin will use the keys to label the dataset outputs, otherwise it will use defaults, so this approach is slightly cleaner.
Let’s visualize the simulation for some example parameters to make sure it looks ok.
[6]:
sim = pre(num=5, t=0.4)
ax = sim.plot_eps(x=0, hlim=(-1, 1))
Parameters#
If we want, we could query these functions directly to perform our own parameter scan, but the utility of the design plugin is that it lets us simply define our design problem as a specification and then takes care of all of the accounting for us.
The first step is to define the design “parameters” (or dimensions), which also serve as inputs to our function defined earlier.
In this case, we have a parameter num, which is a non-negative integer and a parameter t, which is a positive float.
We can construct a named tdd.Parameter for each of these and define some spans as well.
[7]:
param_num = tdd.ParameterInt(name='num', span=(1, 5))
param_t = tdd.ParameterFloat(name='t', span=(0.1, 0.5), num_points=5)
Note that the name fields should correspond to the argument names defined in the function.
The float parameter takes an optional num_points option, which tells the design tool how to discretize the continuous dimension only when doing a grid sweep. For random or Monte Carlo sweeps, it is not used.
Also, it is possible to define parameters that are simply sets of quantities that we might want to select using allowed_values, such as:
[8]:
param_str = tdd.ParameterAny(name="some_string", allowed_values=("these", "are", "values"))
but we will ignore this case as it’s not needed here and the internal logic is similar to that for integer parameters.
Note: to do more complex sets of integer parameters (like skipping every ‘n’ integers). We recommend simply using
ParameterAnyand just passing your integers toallowed_values.
By defining our design parameters like this, we can more concretely define what types and allowed values can be passed to each argument in our function for the parameter sweep.
Method#
Now that we’ve defined our parameters, we also need to define the procedure to use to sample the parameter space we’ve defined. One approach is to independently sample points within the parameter spans, for example using a Monte Carlo method, another is to perform a grid search to uniformly scan between the bounds of each dimension. There are also more complex methods, such as Bayesian optimization and gradient-based optimization, which will be introduced in future releases.
In the design plugin, we define the specifications for the parameter sweeps using Method objects.
Here’s one example, which defines a grid search.
[9]:
method = tdd.MethodGrid()
For this example, let’s instead do a random (Monte Carlo) sampling with a set number of points.
Note: the default Monte Carlo uses the Latin hypercube sampling method to sample the space. One can also pass an object with a
.random()method to thesamplerkeyword argument ofMethodRandomCustomfor more customization. Other Monte Carlo methods from thescipyquasi-Monte Carlo engines work well for this and provide more flexibility.
[10]:
method = tdd.MethodMonteCarlo(num_points=40)
Design Space#
With the design parameters and our method defined, we can combine everything into a DesignSpace, which is mainly a container that provides some higher level methods for interacting with these objects.
[11]:
design_space = tdd.DesignSpace(parameters=[param_num, param_t], method=method)
Now we need to pass our transmission function to the DesignSpace object to get our results.
To start the parameter scan, the DesignSpace has two methods .run() and .run_batch(), which accept our function(s) defined earlier in the following ways.
DesignSpace.run(transmission)samples the design parameters according to themethodand then evaluates each point one by one. This is the most general approach becausetransmissionis not restricted to any relationships regardingSimulationandSimulationData. Because of this, we can’t make any assumptions about whethertransmissioncan be parallelized and will need to call it sequentially for each point in the parameter scan.On the other hand,
DesignSpace.run_batch(pre, post)accepts our pre and post processing functions. Because it can assume that one or many simulations are created and run in between, this method can safely construct aweb.Batchunder the hood and run all of the the tasks in parallel over the parameter sweep points. The final results are then stitched together in the end.
Note: the pre processing function can return a single
Simulation, alistortupleofSimulations or adictcontainingSimulations. The user just needs to be sure that the post processsing function accepts the correspondingSimulationData,*argsofSimulationDataor**kwargsofSimulationData, respectively. For example ifdict(a=sim1, b=sim2)is returned by the pre-processing function, the post-processing function must have a signature ofdef fn_post(a, b).
Results#
The DesignSpace.run() and DesignSpace.run_batch() functions described before both return a Result object, which is basically a dataset containing the function inputs, outputs, source code, and any task ID information corresponding to each data point.
Note that task ID information can only be gathered when using
run_batchbecause in the more generalrun()function,tidy3dcan’t know which tasks were involved in each data point.
[12]:
results = design_space.run_batch(pre, post)
12:48:17 EST Created task '{'num': 4, 't': 0.39104830719293704}' with task_id 'fdve-36528fd2-0075-4718-a331-88975d1e1236' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-36528fd2-007 5-4718-a331-88975d1e1236'.
12:48:18 EST Created task '{'num': 2, 't': 0.2641436911175711}' with task_id 'fdve-902d2ebd-347b-45d1-a5c2-14652cca24df' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-902d2ebd-347 b-45d1-a5c2-14652cca24df'.
12:48:19 EST Created task '{'num': 2, 't': 0.28300943918384247}' with task_id 'fdve-77a2c431-41c3-412e-8833-85d91d8acc4b' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-77a2c431-41c 3-412e-8833-85d91d8acc4b'.
12:48:20 EST Created task '{'num': 3, 't': 0.42091835112436937}' with task_id 'fdve-590b44cb-fb45-4fdc-b84c-2f7bf74c2a37' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-590b44cb-fb4 5-4fdc-b84c-2f7bf74c2a37'.
Created task '{'num': 3, 't': 0.32989630122210556}' with task_id 'fdve-fe18afee-82b4-4663-a379-8c6aeaa187a1' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-fe18afee-82b 4-4663-a379-8c6aeaa187a1'.
12:48:21 EST Created task '{'num': 4, 't': 0.1565956235754127}' with task_id 'fdve-7e588a7d-0d1a-4a49-bbf3-767fb70b462e' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-7e588a7d-0d1 a-4a49-bbf3-767fb70b462e'.
12:48:22 EST Created task '{'num': 1, 't': 0.3834121450295175}' with task_id 'fdve-b423aa46-d4e6-497e-a13a-298c112322c1' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-b423aa46-d4e 6-497e-a13a-298c112322c1'.
12:48:23 EST Created task '{'num': 2, 't': 0.449283495270945}' with task_id 'fdve-da2dcba9-42c5-4ff1-86c6-df84389aa9b7' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-da2dcba9-42c 5-4ff1-86c6-df84389aa9b7'.
12:48:24 EST Created task '{'num': 1, 't': 0.24663579395284363}' with task_id 'fdve-98dd7052-b13f-413b-be35-aa8cb66cc552' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-98dd7052-b13 f-413b-be35-aa8cb66cc552'.
Created task '{'num': 1, 't': 0.2986926434541891}' with task_id 'fdve-fbff1900-d37b-4b68-a93f-5c5889470d39' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-fbff1900-d37 b-4b68-a93f-5c5889470d39'.
12:48:25 EST Created task '{'num': 3, 't': 0.23230515715946173}' with task_id 'fdve-06bf928c-53cf-411b-a535-320cf7c717c9' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-06bf928c-53c f-411b-a535-320cf7c717c9'.
12:48:26 EST Created task '{'num': 4, 't': 0.1779530107434239}' with task_id 'fdve-ff5ccea4-ed6a-4ad6-9742-9902c8136472' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-ff5ccea4-ed6 a-4ad6-9742-9902c8136472'.
12:48:27 EST Created task '{'num': 4, 't': 0.21474008401151304}' with task_id 'fdve-4d2cef62-4e17-4307-b22b-cb4f8f3842f8' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-4d2cef62-4e1 7-4307-b22b-cb4f8f3842f8'.
12:48:28 EST Created task '{'num': 2, 't': 0.4191613126582051}' with task_id 'fdve-bcfd1069-1313-436d-8cfb-b3618ebb0fd2' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-bcfd1069-131 3-436d-8cfb-b3618ebb0fd2'.
Created task '{'num': 4, 't': 0.3583715116305205}' with task_id 'fdve-94b9bdf4-9fa0-42c7-b25c-30c6e108e76c' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-94b9bdf4-9fa 0-42c7-b25c-30c6e108e76c'.
12:48:29 EST Created task '{'num': 2, 't': 0.4072394131242616}' with task_id 'fdve-d8013a43-f962-4135-a653-8031b1ef8b54' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-d8013a43-f96 2-4135-a653-8031b1ef8b54'.
12:48:30 EST Created task '{'num': 4, 't': 0.12783721349799737}' with task_id 'fdve-3d56cc29-80d7-4177-ae9c-3ff75bb34bc2' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-3d56cc29-80d 7-4177-ae9c-3ff75bb34bc2'.
12:48:31 EST Created task '{'num': 2, 't': 0.34126462547186254}' with task_id 'fdve-38e10e4b-880b-46ed-bdf9-9bb9837e6805' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-38e10e4b-880 b-46ed-bdf9-9bb9837e6805'.
12:48:32 EST Created task '{'num': 3, 't': 0.2796131712420378}' with task_id 'fdve-27aa8684-5ac5-4bf0-9ea8-360baeb7f7d0' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-27aa8684-5ac 5-4bf0-9ea8-360baeb7f7d0'.
12:48:33 EST Created task '{'num': 4, 't': 0.49713178281968096}' with task_id 'fdve-b50b4d41-8779-4466-9448-29809c62dee3' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-b50b4d41-877 9-4466-9448-29809c62dee3'.
12:48:35 EST Created task '{'num': 1, 't': 0.4643611514479381}' with task_id 'fdve-6a59543a-2267-4f2d-950c-f29f5927be51' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-6a59543a-226 7-4f2d-950c-f29f5927be51'.
12:48:36 EST Created task '{'num': 1, 't': 0.10237910641300418}' with task_id 'fdve-9ddeab96-9a75-4109-804b-b4cc8b033194' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-9ddeab96-9a7 5-4109-804b-b4cc8b033194'.
12:48:37 EST Created task '{'num': 3, 't': 0.3024051055098004}' with task_id 'fdve-ba81f72a-ef87-4fdc-a105-0e321d46b52f' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-ba81f72a-ef8 7-4fdc-a105-0e321d46b52f'.
Created task '{'num': 2, 't': 0.3329137431318657}' with task_id 'fdve-4f818161-6916-47ce-8796-cd62df61c599' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-4f818161-691 6-47ce-8796-cd62df61c599'.
12:48:39 EST Created task '{'num': 1, 't': 0.47446357100268965}' with task_id 'fdve-1de45d53-4010-47f4-a2e4-c5061e0c0ae3' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-1de45d53-401 0-47f4-a2e4-c5061e0c0ae3'.
12:48:40 EST Created task '{'num': 3, 't': 0.36954685228475126}' with task_id 'fdve-27588b73-95a6-47fc-8fe2-cdfd8a12d77c' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-27588b73-95a 6-47fc-8fe2-cdfd8a12d77c'.
Created task '{'num': 2, 't': 0.22687967276438825}' with task_id 'fdve-2263e341-912b-4cf2-8789-a990c729ec73' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-2263e341-912 b-4cf2-8789-a990c729ec73'.
12:48:41 EST Created task '{'num': 4, 't': 0.14033182398699115}' with task_id 'fdve-7805c345-b0d0-444c-9b76-aa645c8a01cd' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-7805c345-b0d 0-444c-9b76-aa645c8a01cd'.
12:48:42 EST Created task '{'num': 1, 't': 0.18601118292150817}' with task_id 'fdve-699a8ddb-c159-43da-8843-f11b102473cb' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-699a8ddb-c15 9-43da-8843-f11b102473cb'.
12:48:43 EST Created task '{'num': 3, 't': 0.4340066298259795}' with task_id 'fdve-807b50e7-9698-43b2-bbea-96efda133efa' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-807b50e7-969 8-43b2-bbea-96efda133efa'.
12:48:44 EST Created task '{'num': 3, 't': 0.2585030227639742}' with task_id 'fdve-b476ead6-2af6-44e1-aecf-277650079408' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-b476ead6-2af 6-44e1-aecf-277650079408'.
Created task '{'num': 4, 't': 0.1396454514302624}' with task_id 'fdve-4bd76985-a0f8-4e1d-a636-3a7af3b81520' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-4bd76985-a0f 8-4e1d-a636-3a7af3b81520'.
12:48:45 EST Created task '{'num': 1, 't': 0.48646658222029504}' with task_id 'fdve-62d5dc15-593b-484c-9070-97981fd6583e' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-62d5dc15-593 b-484c-9070-97981fd6583e'.
12:48:46 EST Created task '{'num': 1, 't': 0.4563471473510632}' with task_id 'fdve-6169a1e8-f153-4244-9b3d-987b0cc7659e' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-6169a1e8-f15 3-4244-9b3d-987b0cc7659e'.
12:48:47 EST Created task '{'num': 4, 't': 0.19650335371370198}' with task_id 'fdve-6c7a75e2-7378-4c80-a8e5-3cf4ad41c46f' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-6c7a75e2-737 8-4c80-a8e5-3cf4ad41c46f'.
Created task '{'num': 3, 't': 0.31309264988618085}' with task_id 'fdve-5902056b-13cd-480c-a6de-515e05429905' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-5902056b-13c d-480c-a6de-515e05429905'.
12:48:48 EST Created task '{'num': 2, 't': 0.37251139356098995}' with task_id 'fdve-8135f926-aa6d-4485-bfae-151754e2e8b6' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-8135f926-aa6 d-4485-bfae-151754e2e8b6'.
12:48:49 EST Created task '{'num': 1, 't': 0.11814596132082872}' with task_id 'fdve-00945ea8-3c8e-43f9-8209-29fc92a3130b' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-00945ea8-3c8 e-43f9-8209-29fc92a3130b'.
12:48:50 EST Created task '{'num': 2, 't': 0.1604121526299734}' with task_id 'fdve-18e65b94-5354-4025-bb60-c4cff65711cd' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-18e65b94-535 4-4025-bb60-c4cff65711cd'.
12:48:51 EST Created task '{'num': 3, 't': 0.20907493741523975}' with task_id 'fdve-876834d0-fb1a-4a70-bd49-39ba68433011' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-876834d0-fb1 a-4a70-bd49-39ba68433011'.
12:49:18 EST Started working on Batch.
12:49:39 EST Maximum FlexCredit cost: 1.000 for the whole batch.
Use 'Batch.real_cost()' to get the billed FlexCredit cost after the Batch has completed.
12:50:12 EST Batch complete.
12:50:17 EST loading simulation from ./fdve-36528fd2-0075-4718-a331-88975d1e1236.hdf5
12:50:18 EST loading simulation from ./fdve-902d2ebd-347b-45d1-a5c2-14652cca24df.hdf5
12:50:19 EST loading simulation from ./fdve-77a2c431-41c3-412e-8833-85d91d8acc4b.hdf5
12:50:20 EST loading simulation from ./fdve-590b44cb-fb45-4fdc-b84c-2f7bf74c2a37.hdf5
12:50:21 EST loading simulation from ./fdve-fe18afee-82b4-4663-a379-8c6aeaa187a1.hdf5
loading simulation from ./fdve-7e588a7d-0d1a-4a49-bbf3-767fb70b462e.hdf5
12:50:22 EST loading simulation from ./fdve-b423aa46-d4e6-497e-a13a-298c112322c1.hdf5
12:50:23 EST loading simulation from ./fdve-da2dcba9-42c5-4ff1-86c6-df84389aa9b7.hdf5
12:50:24 EST loading simulation from ./fdve-98dd7052-b13f-413b-be35-aa8cb66cc552.hdf5
12:50:25 EST loading simulation from ./fdve-fbff1900-d37b-4b68-a93f-5c5889470d39.hdf5
12:50:26 EST loading simulation from ./fdve-06bf928c-53cf-411b-a535-320cf7c717c9.hdf5
12:50:27 EST loading simulation from ./fdve-ff5ccea4-ed6a-4ad6-9742-9902c8136472.hdf5
12:50:28 EST loading simulation from ./fdve-4d2cef62-4e17-4307-b22b-cb4f8f3842f8.hdf5
12:50:29 EST loading simulation from ./fdve-bcfd1069-1313-436d-8cfb-b3618ebb0fd2.hdf5
12:50:30 EST loading simulation from ./fdve-94b9bdf4-9fa0-42c7-b25c-30c6e108e76c.hdf5
12:50:31 EST loading simulation from ./fdve-d8013a43-f962-4135-a653-8031b1ef8b54.hdf5
loading simulation from ./fdve-3d56cc29-80d7-4177-ae9c-3ff75bb34bc2.hdf5
12:50:32 EST loading simulation from ./fdve-38e10e4b-880b-46ed-bdf9-9bb9837e6805.hdf5
12:50:33 EST loading simulation from ./fdve-27aa8684-5ac5-4bf0-9ea8-360baeb7f7d0.hdf5
12:50:34 EST loading simulation from ./fdve-b50b4d41-8779-4466-9448-29809c62dee3.hdf5
12:50:35 EST loading simulation from ./fdve-6a59543a-2267-4f2d-950c-f29f5927be51.hdf5
12:50:36 EST loading simulation from ./fdve-9ddeab96-9a75-4109-804b-b4cc8b033194.hdf5
12:50:37 EST loading simulation from ./fdve-ba81f72a-ef87-4fdc-a105-0e321d46b52f.hdf5
12:50:38 EST loading simulation from ./fdve-4f818161-6916-47ce-8796-cd62df61c599.hdf5
12:50:39 EST loading simulation from ./fdve-1de45d53-4010-47f4-a2e4-c5061e0c0ae3.hdf5
loading simulation from ./fdve-27588b73-95a6-47fc-8fe2-cdfd8a12d77c.hdf5
12:50:40 EST loading simulation from ./fdve-2263e341-912b-4cf2-8789-a990c729ec73.hdf5
12:50:41 EST loading simulation from ./fdve-7805c345-b0d0-444c-9b76-aa645c8a01cd.hdf5
12:50:42 EST loading simulation from ./fdve-699a8ddb-c159-43da-8843-f11b102473cb.hdf5
12:50:43 EST loading simulation from ./fdve-807b50e7-9698-43b2-bbea-96efda133efa.hdf5
12:50:44 EST loading simulation from ./fdve-b476ead6-2af6-44e1-aecf-277650079408.hdf5
12:50:45 EST loading simulation from ./fdve-4bd76985-a0f8-4e1d-a636-3a7af3b81520.hdf5
12:50:46 EST loading simulation from ./fdve-62d5dc15-593b-484c-9070-97981fd6583e.hdf5
loading simulation from ./fdve-6169a1e8-f153-4244-9b3d-987b0cc7659e.hdf5
12:50:47 EST loading simulation from ./fdve-6c7a75e2-7378-4c80-a8e5-3cf4ad41c46f.hdf5
12:50:48 EST loading simulation from ./fdve-5902056b-13cd-480c-a6de-515e05429905.hdf5
12:50:49 EST loading simulation from ./fdve-8135f926-aa6d-4485-bfae-151754e2e8b6.hdf5
12:50:50 EST loading simulation from ./fdve-00945ea8-3c8e-43f9-8209-29fc92a3130b.hdf5
12:50:51 EST loading simulation from ./fdve-18e65b94-5354-4025-bb60-c4cff65711cd.hdf5
12:50:52 EST loading simulation from ./fdve-876834d0-fb1a-4a70-bd49-39ba68433011.hdf5
Note: we can pass
verbose=Falseto therun_batch()function to turn off outputs for large batches.
Results contains three main related datastructures.
dims, which correpsond to thekwargsof the pre-processing function,('num', 't')here.coords, which is a tuple containing the values passed in for each of the dims.coords[i]is a tuple ofn, andtvalues for theith function call.values, which is a tuple containing the outputs of the postprocessing functions. In this casevalues[i]stores the transmission of theith function call.
The Results can be converted to a pandas.DataFrame where each row is a separate data point and each column is either an input or output for a function. It also contains various methods for plotting and managing the data.
Note: if the measurement function returns a dictionary, the keys will be used as labels for the columns in the
DataFrame. If it returns atupleorlist, each element will get a unique key assigned to them in theDataFramedepending on the index.
[13]:
# The first 5 data points
df = results.to_dataframe()
df.head()
[13]:
| num | t | flux | |
|---|---|---|---|
| 0 | 4 | 0.391048 | 0.981639 |
| 1 | 2 | 0.264144 | 0.951866 |
| 2 | 2 | 0.283009 | 0.950883 |
| 3 | 3 | 0.420918 | 0.934276 |
| 4 | 3 | 0.329896 | 0.961000 |
The pandas documentation provides excellent explanations of the various postprocessing and visualization functions that can be used.
[14]:
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 3.5))
# plot a hexagonal binning of the results
im = df.plot.hexbin(x="t", y="num", gridsize=10, C="flux", cmap="magma", ax=ax1)
ax1.set_title('SCS (hexagonal binning)')
# scatterplot the raw results with flux on the y axis
im = df.plot.scatter(x="t", y="flux", s=50, c="num", cmap="magma", ax=ax2)
ax2.set_title('SCS (data scatter plot)')
plt.show()
Note, these functions just call pandas.DataFrame plotting under the hood, so it is worth referring to the docs page on DataFrame visuazation for more tips on plotting design problem results.
Modifying Results#
After the design problems is run, one might want to modify the results through adding or removing elements or combining different results. Here we will show how to perform some of these advanced features in the design plugin.
Combining Results#
We can combine the results of two separate design problems assuming they were created with the same functions.
This can be done either with result.combine(other_result) or with a shorthand of result + other_result.
In this example, let’s say we want to explore more the region where the transmission seems to be lowest, for t between 0.2 and 0.3. We create a new parameter with the desired span for t and use it to create an updated copy of the previous design, still using the same method.
[15]:
param_t2 = tdd.ParameterFloat(name='t', span=(0.2, 0.3), num_points=5)
design_space2 = design_space.updated_copy(parameters=[param_num, param_t2])
results2 = design_space2.run_batch(pre, post)
Created task '{'num': 1, 't': 0.2961044830724139}' with task_id 'fdve-547d370b-971d-4aa7-9d88-64c50bde4c5a' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-547d370b-971 d-4aa7-9d88-64c50bde4c5a'.
12:50:53 EST Created task '{'num': 4, 't': 0.2976875881343715}' with task_id 'fdve-7d556de3-1018-4189-8416-12f7c4b4ecae' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-7d556de3-101 8-4189-8416-12f7c4b4ecae'.
12:50:54 EST Created task '{'num': 3, 't': 0.23521392150815235}' with task_id 'fdve-7a6cfb5b-5049-4a00-a371-b0ecfb38308d' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-7a6cfb5b-504 9-4a00-a371-b0ecfb38308d'.
12:50:55 EST Created task '{'num': 2, 't': 0.2876479565149917}' with task_id 'fdve-11c96bcf-eb37-424f-947c-b3a0d33dd0ba' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-11c96bcf-eb3 7-424f-947c-b3a0d33dd0ba'.
Created task '{'num': 1, 't': 0.2562312271076913}' with task_id 'fdve-52915d8d-2a6c-4fc5-9ee8-35fc31f503a8' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-52915d8d-2a6 c-4fc5-9ee8-35fc31f503a8'.
12:50:56 EST Created task '{'num': 2, 't': 0.278862707887489}' with task_id 'fdve-1da01fdf-09d4-4706-bf33-6367dd0523d7' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-1da01fdf-09d 4-4706-bf33-6367dd0523d7'.
12:50:57 EST Created task '{'num': 1, 't': 0.2501454363434439}' with task_id 'fdve-ab776dd3-2b30-4433-b54b-ae5083f06c63' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-ab776dd3-2b3 0-4433-b54b-ae5083f06c63'.
12:50:58 EST Created task '{'num': 4, 't': 0.215080915154303}' with task_id 'fdve-0d34d558-8c8e-483e-ac7f-a0f4a8341970' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-0d34d558-8c8 e-483e-ac7f-a0f4a8341970'.
12:50:59 EST Created task '{'num': 2, 't': 0.23991874334291075}' with task_id 'fdve-9e276c5b-906b-47dd-b7ab-eeecba1d278c' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-9e276c5b-906 b-47dd-b7ab-eeecba1d278c'.
12:51:00 EST Created task '{'num': 1, 't': 0.22288696641932507}' with task_id 'fdve-02cb81c8-96d7-43e1-81e2-b02c3bc2f695' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-02cb81c8-96d 7-43e1-81e2-b02c3bc2f695'.
12:51:01 EST Created task '{'num': 1, 't': 0.23039999688828675}' with task_id 'fdve-8eb9d3d2-38a9-4563-9767-146f27bc19f2' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-8eb9d3d2-38a 9-4563-9767-146f27bc19f2'.
Created task '{'num': 1, 't': 0.27034347843469464}' with task_id 'fdve-90a6b7eb-be3f-4e42-9622-e9fffb5b4738' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-90a6b7eb-be3 f-4e42-9622-e9fffb5b4738'.
12:51:02 EST Created task '{'num': 2, 't': 0.2602623908803701}' with task_id 'fdve-5fe38108-cb9a-43ef-b705-31424efba77a' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-5fe38108-cb9 a-43ef-b705-31424efba77a'.
12:51:03 EST Created task '{'num': 3, 't': 0.24786386561423437}' with task_id 'fdve-74bf2a09-616d-43cf-9c5b-23ee522e85f6' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-74bf2a09-616 d-43cf-9c5b-23ee522e85f6'.
12:51:04 EST Created task '{'num': 2, 't': 0.26737201530223764}' with task_id 'fdve-d2b5edc7-3aff-49eb-9ec1-a5c1e125e147' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-d2b5edc7-3af f-49eb-9ec1-a5c1e125e147'.
Created task '{'num': 2, 't': 0.27352426393716034}' with task_id 'fdve-85eac023-ae2a-465c-b860-219965c77674' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-85eac023-ae2 a-465c-b860-219965c77674'.
12:51:05 EST Created task '{'num': 4, 't': 0.2907874414792439}' with task_id 'fdve-3001525f-3ff7-45e3-b524-e2077dd91b01' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-3001525f-3ff 7-45e3-b524-e2077dd91b01'.
12:51:06 EST Created task '{'num': 4, 't': 0.21902315810952375}' with task_id 'fdve-9b85f6d3-e086-46ea-a481-5dd7d769294f' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-9b85f6d3-e08 6-46ea-a481-5dd7d769294f'.
12:51:07 EST Created task '{'num': 3, 't': 0.24250911234814854}' with task_id 'fdve-b168f7cd-1985-47b6-bdbb-4efb7a6cfcec' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-b168f7cd-198 5-47b6-bdbb-4efb7a6cfcec'.
12:51:08 EST Created task '{'num': 1, 't': 0.2758767207774615}' with task_id 'fdve-83de9e87-7738-4608-bab2-e977939103a1' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-83de9e87-773 8-4608-bab2-e977939103a1'.
Created task '{'num': 3, 't': 0.2948969383636933}' with task_id 'fdve-fd9d73bb-5c1d-44e1-b784-d95bd687efde' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-fd9d73bb-5c1 d-44e1-b784-d95bd687efde'.
12:51:09 EST Created task '{'num': 2, 't': 0.22504462873601763}' with task_id 'fdve-cf835cc7-7cc9-4497-89d7-16d85af4588f' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-cf835cc7-7cc 9-4497-89d7-16d85af4588f'.
12:51:10 EST Created task '{'num': 4, 't': 0.2202286813649967}' with task_id 'fdve-5229576b-4295-4b66-be56-166946c0673d' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-5229576b-429 5-4b66-be56-166946c0673d'.
12:51:11 EST Created task '{'num': 3, 't': 0.20643038566059427}' with task_id 'fdve-8369a36e-a6f3-473b-9542-9c24cd576a72' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-8369a36e-a6f 3-473b-9542-9c24cd576a72'.
Created task '{'num': 1, 't': 0.2628785872524213}' with task_id 'fdve-8e92f1d0-d589-4d33-adff-1f7d8afe7e2b' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-8e92f1d0-d58 9-4d33-adff-1f7d8afe7e2b'.
12:51:12 EST Created task '{'num': 1, 't': 0.21435142577868263}' with task_id 'fdve-4080e362-2ebc-41b3-9957-e389b90537f2' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-4080e362-2eb c-41b3-9957-e389b90537f2'.
12:51:13 EST Created task '{'num': 1, 't': 0.20158771484069032}' with task_id 'fdve-58b25f94-bb14-4b1b-b361-0e53468f2f79' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-58b25f94-bb1 4-4b1b-b361-0e53468f2f79'.
12:51:14 EST Created task '{'num': 3, 't': 0.2822017238141206}' with task_id 'fdve-fb39eb18-4ac4-4cd4-afab-4a6855b70537' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-fb39eb18-4ac 4-4cd4-afab-4a6855b70537'.
Created task '{'num': 3, 't': 0.2461628079904204}' with task_id 'fdve-94bbd596-2b31-47d6-b3c4-a73d290639c2' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-94bbd596-2b3 1-47d6-b3c4-a73d290639c2'.
12:51:15 EST Created task '{'num': 3, 't': 0.20950456498374312}' with task_id 'fdve-44e223eb-fdc9-4d02-80ee-dd2d3b36d8ad' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-44e223eb-fdc 9-4d02-80ee-dd2d3b36d8ad'.
12:51:16 EST Created task '{'num': 2, 't': 0.2840465470707318}' with task_id 'fdve-e5d97975-4622-48b5-bc74-fff9f2afb1df' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-e5d97975-462 2-48b5-bc74-fff9f2afb1df'.
12:51:17 EST Created task '{'num': 4, 't': 0.2325000428626322}' with task_id 'fdve-148508f7-2d1c-4702-9e3a-beb895439110' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-148508f7-2d1 c-4702-9e3a-beb895439110'.
12:51:18 EST Created task '{'num': 4, 't': 0.2683174992952657}' with task_id 'fdve-b0c6053d-6264-4bfc-855b-aad8abe24e94' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-b0c6053d-626 4-4bfc-855b-aad8abe24e94'.
Created task '{'num': 2, 't': 0.28652386216518}' with task_id 'fdve-44adb914-1ec5-493f-9800-9770fa743947' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-44adb914-1ec 5-493f-9800-9770fa743947'.
12:51:20 EST Created task '{'num': 2, 't': 0.25401454216762864}' with task_id 'fdve-55ce2814-aa74-44da-8a30-415cce79dd20' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-55ce2814-aa7 4-44da-8a30-415cce79dd20'.
12:51:21 EST Created task '{'num': 4, 't': 0.25983203363082746}' with task_id 'fdve-2320d0f1-ffde-4deb-9dce-8d50395a0d4d' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-2320d0f1-ffd e-4deb-9dce-8d50395a0d4d'.
Created task '{'num': 4, 't': 0.2406192367698622}' with task_id 'fdve-53fca372-9881-471b-8b46-2d63e7ca925b' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-53fca372-988 1-471b-8b46-2d63e7ca925b'.
12:51:22 EST Created task '{'num': 3, 't': 0.22760564533213493}' with task_id 'fdve-30d6476a-52ad-434a-a207-37abe5553bbb' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-30d6476a-52a d-434a-a207-37abe5553bbb'.
12:51:23 EST Created task '{'num': 4, 't': 0.20398476226421267}' with task_id 'fdve-0fb92c7e-f648-49f1-b0b1-48aa7cc12ee4' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-0fb92c7e-f64 8-49f1-b0b1-48aa7cc12ee4'.
12:51:24 EST Created task '{'num': 3, 't': 0.21030752569767663}' with task_id 'fdve-4f2e74cb-d11f-43b0-a1bf-1498f7092198' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-4f2e74cb-d11 f-43b0-a1bf-1498f7092198'.
12:51:44 EST Started working on Batch.
12:52:04 EST Maximum FlexCredit cost: 1.000 for the whole batch.
Use 'Batch.real_cost()' to get the billed FlexCredit cost after the Batch has completed.
12:53:03 EST Batch complete.
12:53:09 EST loading simulation from ./fdve-547d370b-971d-4aa7-9d88-64c50bde4c5a.hdf5
12:53:10 EST loading simulation from ./fdve-7d556de3-1018-4189-8416-12f7c4b4ecae.hdf5
12:53:11 EST loading simulation from ./fdve-7a6cfb5b-5049-4a00-a371-b0ecfb38308d.hdf5
loading simulation from ./fdve-11c96bcf-eb37-424f-947c-b3a0d33dd0ba.hdf5
12:53:12 EST loading simulation from ./fdve-52915d8d-2a6c-4fc5-9ee8-35fc31f503a8.hdf5
12:53:13 EST loading simulation from ./fdve-1da01fdf-09d4-4706-bf33-6367dd0523d7.hdf5
12:53:14 EST loading simulation from ./fdve-ab776dd3-2b30-4433-b54b-ae5083f06c63.hdf5
12:53:15 EST loading simulation from ./fdve-0d34d558-8c8e-483e-ac7f-a0f4a8341970.hdf5
12:53:16 EST loading simulation from ./fdve-9e276c5b-906b-47dd-b7ab-eeecba1d278c.hdf5
12:53:17 EST loading simulation from ./fdve-02cb81c8-96d7-43e1-81e2-b02c3bc2f695.hdf5
12:53:18 EST loading simulation from ./fdve-8eb9d3d2-38a9-4563-9767-146f27bc19f2.hdf5
12:53:19 EST loading simulation from ./fdve-90a6b7eb-be3f-4e42-9622-e9fffb5b4738.hdf5
loading simulation from ./fdve-5fe38108-cb9a-43ef-b705-31424efba77a.hdf5
12:53:20 EST loading simulation from ./fdve-74bf2a09-616d-43cf-9c5b-23ee522e85f6.hdf5
12:53:21 EST loading simulation from ./fdve-d2b5edc7-3aff-49eb-9ec1-a5c1e125e147.hdf5
12:53:22 EST loading simulation from ./fdve-85eac023-ae2a-465c-b860-219965c77674.hdf5
12:53:23 EST loading simulation from ./fdve-3001525f-3ff7-45e3-b524-e2077dd91b01.hdf5
12:53:24 EST loading simulation from ./fdve-9b85f6d3-e086-46ea-a481-5dd7d769294f.hdf5
12:53:25 EST loading simulation from ./fdve-b168f7cd-1985-47b6-bdbb-4efb7a6cfcec.hdf5
12:53:26 EST loading simulation from ./fdve-83de9e87-7738-4608-bab2-e977939103a1.hdf5
12:53:27 EST loading simulation from ./fdve-fd9d73bb-5c1d-44e1-b784-d95bd687efde.hdf5
12:53:28 EST loading simulation from ./fdve-cf835cc7-7cc9-4497-89d7-16d85af4588f.hdf5
loading simulation from ./fdve-5229576b-4295-4b66-be56-166946c0673d.hdf5
12:53:29 EST loading simulation from ./fdve-8369a36e-a6f3-473b-9542-9c24cd576a72.hdf5
12:53:30 EST loading simulation from ./fdve-8e92f1d0-d589-4d33-adff-1f7d8afe7e2b.hdf5
12:53:31 EST loading simulation from ./fdve-4080e362-2ebc-41b3-9957-e389b90537f2.hdf5
12:53:32 EST loading simulation from ./fdve-58b25f94-bb14-4b1b-b361-0e53468f2f79.hdf5
12:53:33 EST loading simulation from ./fdve-fb39eb18-4ac4-4cd4-afab-4a6855b70537.hdf5
12:53:34 EST loading simulation from ./fdve-94bbd596-2b31-47d6-b3c4-a73d290639c2.hdf5
12:53:35 EST loading simulation from ./fdve-44e223eb-fdc9-4d02-80ee-dd2d3b36d8ad.hdf5
12:53:36 EST loading simulation from ./fdve-e5d97975-4622-48b5-bc74-fff9f2afb1df.hdf5
12:53:37 EST loading simulation from ./fdve-148508f7-2d1c-4702-9e3a-beb895439110.hdf5
12:53:38 EST loading simulation from ./fdve-b0c6053d-6264-4bfc-855b-aad8abe24e94.hdf5
loading simulation from ./fdve-44adb914-1ec5-493f-9800-9770fa743947.hdf5
12:53:39 EST loading simulation from ./fdve-55ce2814-aa74-44da-8a30-415cce79dd20.hdf5
12:53:40 EST loading simulation from ./fdve-2320d0f1-ffde-4deb-9dce-8d50395a0d4d.hdf5
12:53:41 EST loading simulation from ./fdve-53fca372-9881-471b-8b46-2d63e7ca925b.hdf5
12:53:42 EST loading simulation from ./fdve-30d6476a-52ad-434a-a207-37abe5553bbb.hdf5
12:53:43 EST loading simulation from ./fdve-0fb92c7e-f648-49f1-b0b1-48aa7cc12ee4.hdf5
12:53:44 EST loading simulation from ./fdve-4f2e74cb-d11f-43b0-a1bf-1498f7092198.hdf5
Now we can inspect the new results and the combined data as we did before:
[16]:
results_combined = results + results2
df2 = results2.to_dataframe()
df_combined = results_combined.to_dataframe()
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 3.5))
im = df2.plot.scatter(x="t", y="flux", s=50, c="num", cmap="magma", ax=ax1)
ax1.set_title('Second run')
im = df_combined.plot.scatter(x="t", y="flux", s=50, c="num", cmap="magma", ax=ax2)
ax1.set_title('Combined results')
plt.show()
We can also change the method if we want…
[17]:
# make another set of results using the same function and design but different method
design_space3 = design_space.updated_copy(method=tdd.MethodRandom(num_points=2, monte_carlo_warning=False))
results3 = design_space3.run_batch(pre, post)
Created task '{'num': 1, 't': 0.3070917025327629}' with task_id 'fdve-7d1d6023-7055-44ae-92f7-dfc2e470a08c' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-7d1d6023-705 5-44ae-92f7-dfc2e470a08c'.
12:53:45 EST Created task '{'num': 3, 't': 0.40569222338714184}' with task_id 'fdve-89704ef4-3277-44c7-b198-7c5098447bec' and task_type 'FDTD'.
View task using web UI at 'https://tidy3d.simulation.cloud/workbench?taskId=fdve-89704ef4-327 7-44c7-b198-7c5098447bec'.
12:53:47 EST Started working on Batch.
12:53:48 EST Maximum FlexCredit cost: 0.050 for the whole batch.
Use 'Batch.real_cost()' to get the billed FlexCredit cost after the Batch has completed.
12:54:05 EST Batch complete.
12:54:07 EST loading simulation from ./fdve-7d1d6023-7055-44ae-92f7-dfc2e470a08c.hdf5
loading simulation from ./fdve-89704ef4-3277-44c7-b198-7c5098447bec.hdf5
… and combine the third run to the previous (now using the combine method, which is equivalent to adding both):
[18]:
all_results = results_combined.combine(results3)
print(all_results == results + results2 + results3)
True
Adding and removing results#
We can also add and remove individual entries from the Results with the add and delete methods.
Let’s add a new data point to our results and then delete it.
[19]:
# define the function inputs and outputs
fn_args = {"t": 1.2, "num": 5}
value = 1.9
To add a data element, we pass in the fn_args (inputs) and the value (outputs).
[20]:
# add it to the results and view the last 5 entries (data is present)
results = results.add(fn_args=fn_args, value=value)
results.to_dataframe().tail()
[20]:
| num | t | flux | |
|---|---|---|---|
| 36 | 2 | 0.372511 | 0.927329 |
| 37 | 1 | 0.118146 | 0.922039 |
| 38 | 2 | 0.160412 | 0.918622 |
| 39 | 3 | 0.209075 | 0.764402 |
| 40 | 5 | 1.200000 | 1.900000 |
We can select a specific data point by passing the function inputs as keyword arguments to the Results.sel method.
[21]:
print(results.sel(**fn_args))
1.9
Similarly, we can remove a data point by passing the fn_args (inputs) to the Results.delete method.
[22]:
# remove this data from the results and view the last 5 entries (data is gone)
results = results.delete(fn_args=fn_args)
results.to_dataframe().tail()
[22]:
| num | t | flux | |
|---|---|---|---|
| 35 | 3 | 0.313093 | 0.895099 |
| 36 | 2 | 0.372511 | 0.927329 |
| 37 | 1 | 0.118146 | 0.922039 |
| 38 | 2 | 0.160412 | 0.918622 |
| 39 | 3 | 0.209075 | 0.764402 |
Leveraging DataFrame functions#
Since we have the ability to convert the Results to and from pandas.DataFrame objects, we are able to perform various manipulations (including the ones above) on the data using pandas. For example, we can create a new DataFrame object that multiplies all of the original flux values by 1/2 and convert this back to a Result if desired.
[23]:
df = results.to_dataframe()
df.head()
[23]:
| num | t | flux | |
|---|---|---|---|
| 0 | 4 | 0.391048 | 0.981639 |
| 1 | 2 | 0.264144 | 0.951866 |
| 2 | 2 | 0.283009 | 0.950883 |
| 3 | 3 | 0.420918 | 0.934276 |
| 4 | 3 | 0.329896 | 0.961000 |
[24]:
# get a dataframe where everything is squared
df_half_flux = df.copy()
df_half_flux["flux"] = df_half_flux["flux"].map(lambda x: x * 0.5)
df_half_flux.head()
[24]:
| num | t | flux | |
|---|---|---|---|
| 0 | 4 | 0.391048 | 0.490820 |
| 1 | 2 | 0.264144 | 0.475933 |
| 2 | 2 | 0.283009 | 0.475442 |
| 3 | 3 | 0.420918 | 0.467138 |
| 4 | 3 | 0.329896 | 0.480500 |
We generally recommend you make use of the many features provided by pandas.DataFrame to do further postprocessing and analysis of your sweep results, but to convert a DataFrame back to a Result, one can simply pass it to Results.from_dataframe along with the dimensions of the data (if the DataFrame has been modified).
[25]:
# load it into a Result, note that we need to pass dims because this is an entirely new DataFrame without metadata
results_squared = tdd.Result.from_dataframe(df_half_flux, dims=("num", "t"))
[ ]: