Event-Triggered Dual Heuristic Programming (ET-DHP)¶

ET-DHP is an adaptive optimal controller from the Dual Heuristic Programming family extended with an event-triggered sampling scheme. Between triggers the actuator simply holds the last control and no actor/critic updates run, so the controller's computational rate is decoupled from the sensor/simulation rate — the computational saving can reach an order of magnitude on stabilisation tasks while preserving closed-loop regulation quality. See also the nonlinear F-16 model: NonlinearLongitudinalF16.

Key ideas¶

Event-triggered supervisor: a Lipschitz-style rule compares the measured state against the state captured at the last trigger; updates fire only when the deviation exceeds a growing, saturating threshold
Neural plant model: a pre-trained two-layer MLP predicts \(x_{k+1} = f(x_k, u_k)\); autograd through it yields the Jacobians \(F = \partial f/\partial x\) and \(G = \partial f/\partial u\) used by the actor and critic targets
Bounded actor: deterministic policy \(u = u_b \cdot \tanh(D(x))\) respects a per-channel actuator bound even during the early random-search phase; \(u(0)=0\) by construction (no bias layers), so the regulator fixed point is exact
Costate critic: the critic directly regresses \(\lambda(x) = \partial J/\partial x\) (DHP form), enabling a clean matrix-vector actor update without a scalar \(J\)-head
Abu-Khalaf–Lewis bounded-control cost: integral term \(Y(u)\) added to the running cost makes \(u = u_b \cdot \tanh(D)\) the exact optimum of the underlying quadratic regulator

The diagram traces one control step: the measured state \(x_k\) is compared against the last triggered state \(x_{\mathrm{et}}\); on a trigger the plant model provides \(F\) and \(G\) via autograd, the critic computes \(\lambda(x_{k+1})\), a closed-form \(u^{*}\) is assembled, and the actor/critic take SGD steps. Between triggers the last command \(u_{k-1}\) is simply held and no gradients are evaluated.

Aspect	HDP	DHP	ET-DHP
Critic output	\(J(x)\)	\(\lambda(x) = \partial J/\partial x\)	\(\lambda(x)\)
Plant model	Known / none	Analytical or NN	Pre-trained NN
Sampling	Time-triggered	Time-triggered	Event-triggered
Actor bounds	Often unbounded	Often unbounded	\(u_b \cdot \tanh(D)\)
Cost function	Quadratic	Quadratic	Quadratic + bounded-control integral

ET-DHP components¶

Component	Role	Implementation
PlantModelNN	One-step predictor \(x_{k+1} = f(x_k, u_k)\); source of \(F\), \(G\) Jacobians	`tensoraerospace.agent.et_dhp.PlantModelNN`
ETDHPActor	Bounded deterministic policy \(u_b \cdot \tanh(D(x))\)	`tensoraerospace.agent.et_dhp.ETDHPActor`
ETDHPCritic	Costate network \(\lambda(x) = \partial J/\partial x\)	`tensoraerospace.agent.et_dhp.ETDHPCritic`
EventTrigger	Lipschitz rule deciding when updates fire	`tensoraerospace.agent.et_dhp.EventTrigger`
ETDHPAgent	Orchestrates all components, predict/learn interface	`tensoraerospace.agent.et_dhp.ETDHPAgent`

Algorithm¶

At every discrete step \(k\) with measurement \(x_k\):

Event check. Compare \(\|x_k - x_{\mathrm{et}}\|\) against the Lipschitz threshold

\[ \rho \, \|x_{\mathrm{et}}\| \, \frac{1 - (2\rho)^{k - k_{\mathrm{trig}}}}{1 - 2\rho} \]

where \(x_{\mathrm{et}}\) and \(k_{\mathrm{trig}}\) are the state and step captured at the most recent trigger, and \(\rho \in (0, 0.5)\). If the bound is exceeded — trigger; otherwise hold the last control and skip training.

Plant Jacobians. Forward pass \((x, u)\) through the pre-trained plant network and row-wise autograd to extract \(F = \partial f/\partial x\), \(G = \partial f/\partial u\).
Closed-form optimal control (Modares–Lewis bounded-action form):

\[ u^{*} = u_b \cdot \tanh\!\left(-\frac{\gamma}{2 u_b} R^{-1} G^{\top} \lambda(x_{k+1})\right) \]

Costate target. Using the running cost \(r = x^{\top} Q x + Y(u)\) with the bounded-control integral cost

\[ Y(u) = 2 u_b^2 \, \mathrm{diag}(R) \cdot \bigl[\tanh(D)\cdot D + \tfrac{1}{2}\log(1 - \tanh^2 D)\bigr], \]

the costate target is \(\lambda_{\mathrm{target}} = \gamma F^{\top} \lambda(x_{k+1}) + \partial r/\partial x\).

Gradient steps. SGD on the actor against \(\mathrm{MSE}(u, u^{*})\) and on the critic against \(\mathrm{MSE}(\lambda(x), \lambda_{\mathrm{target}})\).

Quick start¶

import numpy as np
from tensoraerospace.agent.et_dhp import ETDHPAgent, ETDHPConfig

# Regulation-state transform: convert raw env observation into x_tilde
# that drives to zero at the desired operating point.
def state_transform(obs, reference_signal, time_step):
    return np.degrees(np.asarray(obs).reshape(-1))  # example: deg units

cfg = ETDHPConfig(
    actor_hidden=(24, 24),
    critic_hidden=(24, 24),
    model_hidden=(24, 24),
    actor_lr=5e-3,
    critic_lr=5e-3,
    model_lr=5e-3,
    model_epochs=300,
    Q=[10.0, 0.2, 0.0, 0.0],
    R=[0.5],
    gamma=0.95,
    num_epochs_per_trigger=5,
    u_bound=5.0,
    rho=0.15,
    trigger_floor=0.05,
    weight_init_scale=0.3,
    seed=0,
)

agent = ETDHPAgent(
    n_state=4,
    n_control=1,
    state_transform=state_transform,
    config=cfg,
)

# 1. Pre-train the plant model on an offline PE roll-out.
agent.fit_plant_model(states_arr, actions_arr, next_states_arr,
                      batch_size=128, verbose=True)

# 2. Online event-triggered closed-loop control.
obs, _ = env.reset()
agent.reset()
for k in range(number_time_steps - 2):
    agent.predict(obs, reference_signal, k)
    u_cmd = agent.last_action()
    obs_next, _, done, _, _ = env.step(u_cmd)
    metrics = agent.learn(obs_next, reference_signal, k, dt=dt)
    obs = obs_next
    if done:
        break

Tip

The actor's fixed point is \(u(0) = 0\). For tracking tasks, design state_transform so that perfect tracking corresponds to the zero regulation state (e.g. subtract the reference signal from the measured state).

Hyperparameters¶

General¶

Parameter	Default	Description
`gamma`	1.0	Discount factor
`num_epochs_per_trigger`	10	Inner SGD sweeps per triggered step
`weight_init_scale`	0.5	Uniform-init bound for all network weights
`seed`	None	RNG seed for reproducibility
`device`	`"cpu"`	PyTorch device

Actor¶

Parameter	Default	Description
`actor_hidden`	(10, 10)	Hidden layer sizes
`actor_lr`	1e-3	SGD learning rate
`u_bound`	1.0	Per-channel absolute actuator bound

Critic¶

Parameter	Default	Description
`critic_hidden`	(10, 10)	Hidden layer sizes
`critic_lr`	1e-3	SGD learning rate

Plant model¶

Parameter	Default	Description
`model_hidden`	(10, 10)	Hidden layer sizes
`model_lr`	1e-3	Adam learning rate for offline fit
`model_epochs`	200	Epochs of offline pre-training
`online_model_fit`	False	Keep adapting the plant model after the offline phase

Cost function¶

Parameter	Default	Description
`Q`	(1.0,)	Diagonal weights of state cost \(x^{\top} Q x\); length must equal `n_state`
`R`	(1.0,)	Diagonal weights of control cost; length must equal `n_control`

Event trigger¶

Parameter	Default	Description
`rho`	0.1	Lipschitz constant \(\in (0, 0.5)\); smaller ⇒ more triggers, tighter tracking
`trigger_floor`	1e-3	Minimum threshold (state units) to suppress noise-induced firings

Exploration¶

Parameter	Default	Description
`exploration_fn`	None	Optional callable `(time_sec) -> array` injecting PE into the actor target

Supported environments¶

NonlinearLongitudinalF16-v0
LinearLongitudinalF16-v0

API reference¶

`ETDHPAgent(n_state, n_control, state_transform=None, config=None, log_dir=None, wandb_project=None, wandb_entity=None, wandb_run_name=None, wandb_tags=None, wandb_config=None)` ¶

Event-triggered DHP agent with bounded actor and costate critic.

The agent operates on a regulation state x̃ — typically the tracking error y − y_ref possibly augmented with unmeasured channels regulated to zero (elevator rate, etc.). The caller is responsible for turning the raw environment observation into that regulation state via :meth:obs_to_state, overridable by passing a custom state_transform on construction.

Parameters:

Name	Type	Description	Default
`n_state`	`int`	Length of the regulation state vector `x̃`.	required
`n_control`	`int`	Number of control channels.	required
`state_transform`	`Callable[[ndarray, ndarray \| None, int], ndarray] \| None`	Callable `(obs, ref, time_step) -> x̃` that builds the regulation state from the raw Gym observation and reference. If `None` (default), the state is assumed to already be in regulation form.	`None`
`config`	`ETDHPConfig \| None`	Optional :class:`ETDHPConfig` instance.	`None`

`obs_to_state(obs, reference_signal, time_step)` ¶

Turn a raw env observation into the regulation state x̃.

When no custom transform is supplied this is just the identity, which makes sense for pure stabilisation tasks. For tracking tasks the user can pass a state_transform that returns the tracking error (possibly padded with zero-regulated channels).

`reset()` ¶

Re-arm the event trigger for a new episode.

Does not touch learned weights so training accumulates across episodes, consistent with the rest of the tensoraerospace agents.

`fit_plant_model(states, actions, next_states, *, batch_size=256, epochs=None, verbose=False)` ¶

Train :attr:plant_model on offline transitions (x, u, x').

The agent expects the state tuples in regulation form (i.e. already transformed by :meth:obs_to_state). Typically a PE roll-out is collected on the real env, converted via obs_to_state and then handed here.

Parameters:

Name	Type	Description	Default
`states`	`ndarray`	Array of shape `(N, n_state)`.	required
`actions`	`ndarray`	Array of shape `(N, n_control)`.	required
`next_states`	`ndarray`	Array of shape `(N, n_state)`.	required
`batch_size`	`int`	Mini-batch size for Adam.	`256`
`epochs`	`int \| None`	Overrides `cfg.model_epochs` when set.	`None`
`verbose`	`bool`	Print per-epoch MSE when True.	`False`

Returns:

Type	Description
`list[float]`	Per-epoch MSE loss trace.

`predict(obs, reference_signal=None, time_step=0, *, deterministic=True)` ¶

Compute the control action for the current measurement.

deterministic is accepted for API consistency with other tensoraerospace agents but is ignored — exploration is handled inside :meth:learn via cfg.exploration_fn, so the control that reaches the plant is always the actor's current best.

`learn(next_obs, reference_signal=None, time_step=0, *, dt=1.0)` ¶

Run the event-triggered update step.

Must be called after :meth:predict and the environment step. When the event-trigger threshold is not crossed this method is essentially free — only the Lipschitz test runs, no autograd and no optimiser touch.

Parameters:

Name	Type	Description	Default
`next_obs`	`ndarray`	Environment measurement `y_{k+1}`.	required
`reference_signal`	`ndarray \| None`	Optional reference trajectory, forwarded to the state transform.	`None`
`time_step`	`int`	Discrete step index `k` of the measurement (same index that was passed to :meth:`predict`).	`0`
`dt`	`float`	Simulation step (s). Used only to advance the internal wall-clock that `cfg.exploration_fn` receives.	`1.0`

Returns:

Type	Description
`dict[str, float]`	Dictionary of scalar metrics: `triggered` (1/0),
`dict[str, float]`	`norm_diff`, `condition`, `actor_loss`,
`dict[str, float]`	`critic_loss`.

`last_action()` ¶

Return the control that the env should actually apply.

Between triggers this is the same action that was last produced by :meth:predict; on triggered steps it reflects the freshly updated actor.

`train_episode(env, *, max_steps=None, reference_signal=None)` ¶

Run one episode of ET-DHP training against a Gym env.

This is a thin wrapper intended for scripts/notebooks; the more flexible path is to call :meth:predict / :meth:learn directly and forward last_action to the environment.

Parameters:

Name	Type	Description	Default
`env`	`Any`	Gymnasium env exposing `reset`/`step`/`dt`.	required
`max_steps`	`int \| None`	Optional cap on episode length.	`None`
`reference_signal`	`ndarray \| None`	Override the env's reference (defaults to `env.reference_signal` when present).	`None`

Returns:

Type	Description
`dict[str, float]`	Summary dict: `steps`, `triggers`, `ep_return`.

`train(env, *, num_episodes=1, max_steps=None, reference_signal=None)` ¶

Train ET-DHP for num_episodes episodes (unified interface).

Thin wrapper around :meth:train_episode that loops the requested number of episodes, flushes the metrics writer, and asserts the unified-metrics contract once the session has ended.

Parameters:

Name	Type	Description	Default
`env`	`Any`	Gymnasium env.	required
`num_episodes`	`int`	Episode budget.	`1`
`max_steps`	`int \| None`	Optional per-episode step cap.	`None`
`reference_signal`	`ndarray \| None`	Optional reference signal forwarded to :meth:`train_episode`.	`None`

Returns:

Type	Description
`dict[str, float]`	Aggregated session-level summary.

`get_param_env()` ¶

Build a JSON-serialisable config for :meth:save.

The state_transform callable is never serialised — on load the caller must supply it again via :meth:from_pretrained / :meth:_load_from_dir. This matches how Gymnasium treats other non-picklable env-side components.

`save(path=None, *, save_gradients=False)` ¶

Write the agent to a directory.

Files produced

config.json — constructor kwargs + serialised :class:ETDHPConfig.
actor.pth / critic.pth / plant_model.pth — PyTorch state dicts.
event_trigger.json — supervisor counters and the state captured at the last trigger.
actor_optim.pth / critic_optim.pth / model_optim.pth — optimiser state dicts (only when save_gradients=True).

Parameters:

Name	Type	Description	Default
`path`	`Union[str, Path, None]`	Base directory (`None` → CWD).	`None`
`save_gradients`	`bool`	Persist optimiser states for resume.	`False`

Returns:

Type	Description
`str`	Absolute path to the created run directory.

`from_pretrained(repo_name, access_token=None, version=None, state_transform=None, load_gradients=False)` `classmethod` ¶

Load an agent from a local directory or the Hugging Face Hub.

Parameters:

Name	Type	Description	Default
`repo_name`	`str`	Local folder path, or `namespace/repo_name` on the Hugging Face Hub.	required
`access_token`	`Optional[str]`	Hub access token for private repos.	`None`
`version`	`Optional[str]`	Hub revision / branch / tag.	`None`
`state_transform`	`Callable[[ndarray, ndarray \| None, int], ndarray] \| None`	Re-attach the observation-to-regulation transform used during training (optional but required for tracking tasks).	`None`
`load_gradients`	`bool`	Also restore optimiser state dicts.	`False`

Returns:

Name	Type	Description
`ETDHPAgent`	`'ETDHPAgent'`	Reconstructed agent.

`publish_to_hub(repo_name, folder_path, access_token=None)` ¶

Upload a :meth:save directory to the Hugging Face Hub.

Parameters:

Name	Type	Description	Default
`repo_name`	`str`	Target repository id, e.g. `"me/my-etdhp"`.	required
`folder_path`	`Union[str, Path]`	Local folder produced by :meth:`save`.	required
`access_token`	`Optional[str]`	Hub access token.	`None`

`ETDHPConfig(actor_hidden=(10, 10), critic_hidden=(10, 10), model_hidden=(10, 10), actor_lr=0.001, critic_lr=0.001, model_lr=0.001, model_epochs=200, online_model_fit=False, Q=(1.0,), R=(1.0,), gamma=1.0, num_epochs_per_trigger=10, u_bound=1.0, rho=0.1, trigger_floor=0.001, exploration_fn=None, weight_init_scale=0.5, device='cpu', seed=None, history=dict())` `dataclass` ¶

Hyper-parameters for :class:ETDHPAgent.

Parameters:

Name	Type	Description	Default
`actor_hidden`	`Sequence[int]`	Tanh hidden-layer sizes for the actor MLP.	`(10, 10)`
`critic_hidden`	`Sequence[int]`	Tanh hidden-layer sizes for the critic MLP.	`(10, 10)`
`model_hidden`	`Sequence[int]`	Tanh hidden-layer sizes for the plant-model MLP.	`(10, 10)`
`actor_lr`	`float`	SGD learning rate for the actor.	`0.001`
`critic_lr`	`float`	SGD learning rate for the critic.	`0.001`
`model_lr`	`float`	Adam learning rate for the plant-model pre-training.	`0.001`
`model_epochs`	`int`	Number of epochs for the offline plant-model fit (`fit_plant_model`). Online fine-tuning uses one epoch per triggered step when enabled via `online_model_fit`.	`200`
`online_model_fit`	`bool`	Whether to keep adapting the plant model on every triggered step. Disabled by default — the reference paper freezes `f_θ` after the offline phase.	`False`
`Q`	`Sequence[float]`	Diagonal weights of the quadratic state cost `xᵀ Q x`. Length must equal `n_state`.	`(1.0,)`
`R`	`Sequence[float]`	Diagonal weights of the quadratic control cost. Length must equal `n_control`.	`(1.0,)`
`gamma`	`float`	Discount factor. Default `1.0` (finite-horizon / regulation-style, as in the reference).	`1.0`
`num_epochs_per_trigger`	`int`	Inner-loop actor/critic iterations run each time the event trigger fires. The reference uses 10 on the nonlinear F-16-like plant.	`10`
`u_bound`	`float`	Per-channel actuator bound fed to the bounding layer of the actor. Interpreted in the same units as the action the env expects.	`1.0`
`rho`	`float`	Lipschitz constant for the event trigger (see :class:`EventTrigger`). Must lie in `(0, 0.5)`.	`0.1`
`trigger_floor`	`float`	Minimum trigger threshold (state units). Small positive values suppress noise-induced retriggers when `x ≈ 0`.	`0.001`
`exploration_fn`	`Callable[[float], ndarray] \| None`	Optional callable `(time_sec) -> np.ndarray` of length `n_control`. If provided, the returned vector is added to the actor's target control during training, which injects persistent excitation into the NN update without perturbing the control actually applied to the plant. `None` disables exploration.	`None`
`weight_init_scale`	`float`	Uniform-init bound for each NN weight.	`0.5`
`device`	`str`	Torch device.	`'cpu'`
`seed`	`int \| None`	Optional seed for numpy and torch RNGs.	`None`

`ETDHPActor(n_state, n_control, hidden_sizes=(10, 10), u_bound=1.0, init_scale=0.5)` ¶

Bases: Module

Bounded deterministic policy u = u_b · tanh(D_nn(x)).

The D_nn pre-bound output is returned alongside the bounded control because the ET-DHP reward contains a "bounding" term that depends on the raw logits — see :func:bounded_integral_cost in :mod:tensoraerospace.agent.et_dhp.model.

Parameters:

Name	Type	Description	Default
`n_state`	`int`	Length of the state vector (actor input).	required
`n_control`	`int`	Number of actuators.	required
`hidden_sizes`	`Sequence[int]`	Tanh hidden-layer sizes.	`(10, 10)`
`u_bound`	`float`	Per-channel absolute bound on the output. The bounding layer rescales `tanh(D_nn)` by this value.	`1.0`
`init_scale`	`float`	Uniform-init bound for each weight.	`0.5`

`forward(state)` ¶

Return (u_bounded, D_nn).

D_nn is the pre-saturation output of the last linear layer. The bounded control is u_bound · tanh(D_nn).

`ETDHPCritic(n_state, hidden_sizes=(10, 10), init_scale=0.5)` ¶

Bases: Module

Costate critic λ(x) = ∂J/∂x for DHP.

Directly regresses the partial derivative of the cost-to-go with respect to the state (same dimensionality as x) so the actor update can take a plain matrix-vector form and skip the scalar J-head altogether — this is the original DHP formulation from Werbos/Prokhorov.

Parameters:

Name	Type	Description	Default
`n_state`	`int`	Length of the state vector `x`.	required
`hidden_sizes`	`Sequence[int]`	Tanh hidden-layer sizes.	`(10, 10)`
`init_scale`	`float`	Uniform-init bound for each weight.	`0.5`

`forward(state)` ¶

Return λ(x) of shape (n_state,) (or batched).

`PlantModelNN(n_state, n_control, hidden_sizes=(10, 10), init_scale=0.5)` ¶

Bases: Module

One-step predictor x_{k+1} = f(x_k, u_k) for ET-DHP.

The network realises a discrete-time approximation of the plant dynamics. It is queried twice during each learning sweep:

Forward — predict the next state.
Backward — extract the Jacobians F = ∂f/∂x and G = ∂f/∂u via autograd; these feed directly into the critic target and the optimal-control formula used by the actor.

Parameters:

Name	Type	Description	Default
`n_state`	`int`	Length of the state vector `x`.	required
`n_control`	`int`	Length of the control vector `u`.	required
`hidden_sizes`	`Sequence[int]`	Sizes of the tanh hidden layers. Defaults to `(10, 10)` to match the reference implementation.	`(10, 10)`
`init_scale`	`float`	Uniform-init bound for each weight (no bias). The reference uses `0.5`.	`0.5`

`forward(xu)` ¶

Return the predicted next state from a concatenated [x; u].

`EventTrigger(rho=0.1, trigger_first_step=True, min_floor=0.0)` ¶

Stateful Lipschitz-based event-trigger for ET-DHP.

Keeps track of the reference state captured at the last trigger and compares its distance to the incoming measured state against a growing threshold that saturates once (2ρ)^Δk → 0.

Parameters:

Name	Type	Description	Default
`rho`	`float`	Lipschitz constant `ρ`. Paper uses 0.1 — lower values give tighter tracking at the cost of more triggers.	`0.1`
`trigger_first_step`	`bool`	If True (default), the very first call to :meth:`should_trigger` always returns True so the networks and control run once at `k = 0`.	`True`
`min_floor`	`float`	Lower bound on the threshold. Adds a small absolute slack so the trigger does not fire on pure measurement noise when `x ≈ 0` (the multiplicative `‖x_et‖` term collapses to zero at the origin otherwise).	`0.0`

`reset()` ¶

Re-arm the trigger for a new episode.

`threshold(step)` ¶

Compute the current Lipschitz threshold at step.

Returns the floor when the trigger has never fired yet (so the first evaluation still runs).

`should_trigger(state_measured, step)` ¶

Return True when the networks must be re-evaluated.

Parameters:

Name	Type	Description	Default
`state_measured`	`ndarray`	Current (possibly noisy) state measurement.	required
`step`	`int`	Current discrete time index.	required

Sources¶

Sun, B., Liu, C., Dally, K., van Kampen, E.-J. (2022). Intelligent Aircraft Stabilization Control with Event-Triggered Scheme. CEAS EuroGNC 2022.
Abu-Khalaf, M., Lewis, F. L. (2005). Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica, 41(5), 779–791.
Modares, H., Lewis, F. L. (2014). Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning. Automatica, 50(7), 1780–1792.

Event-Triggered Dual Heuristic Programming (ET-DHP)¶

Key ideas¶

Differences from related agents¶

ET-DHP components¶

Algorithm¶

Quick start¶

Hyperparameters¶

General¶

Actor¶

Critic¶

Plant model¶

Cost function¶

Event trigger¶

Exploration¶

Supported environments¶

API reference¶

ETDHPAgent(n_state, n_control, state_transform=None, config=None, log_dir=None, wandb_project=None, wandb_entity=None, wandb_run_name=None, wandb_tags=None, wandb_config=None) ¶

obs_to_state(obs, reference_signal, time_step) ¶

reset() ¶

fit_plant_model(states, actions, next_states, *, batch_size=256, epochs=None, verbose=False) ¶

predict(obs, reference_signal=None, time_step=0, *, deterministic=True) ¶

learn(next_obs, reference_signal=None, time_step=0, *, dt=1.0) ¶

last_action() ¶

train_episode(env, *, max_steps=None, reference_signal=None) ¶

train(env, *, num_episodes=1, max_steps=None, reference_signal=None) ¶

get_param_env() ¶

save(path=None, *, save_gradients=False) ¶

from_pretrained(repo_name, access_token=None, version=None, state_transform=None, load_gradients=False) classmethod ¶

publish_to_hub(repo_name, folder_path, access_token=None) ¶

ETDHPActor(n_state, n_control, hidden_sizes=(10, 10), u_bound=1.0, init_scale=0.5) ¶

forward(state) ¶

ETDHPCritic(n_state, hidden_sizes=(10, 10), init_scale=0.5) ¶

forward(state) ¶

PlantModelNN(n_state, n_control, hidden_sizes=(10, 10), init_scale=0.5) ¶

forward(xu) ¶

EventTrigger(rho=0.1, trigger_first_step=True, min_floor=0.0) ¶

reset() ¶

threshold(step) ¶

should_trigger(state_measured, step) ¶

Sources¶

`ETDHPAgent(n_state, n_control, state_transform=None, config=None, log_dir=None, wandb_project=None, wandb_entity=None, wandb_run_name=None, wandb_tags=None, wandb_config=None)` ¶

`obs_to_state(obs, reference_signal, time_step)` ¶

`reset()` ¶

`fit_plant_model(states, actions, next_states, *, batch_size=256, epochs=None, verbose=False)` ¶

`predict(obs, reference_signal=None, time_step=0, *, deterministic=True)` ¶

`learn(next_obs, reference_signal=None, time_step=0, *, dt=1.0)` ¶

`last_action()` ¶

`train_episode(env, *, max_steps=None, reference_signal=None)` ¶

`train(env, *, num_episodes=1, max_steps=None, reference_signal=None)` ¶

`get_param_env()` ¶

`save(path=None, *, save_gradients=False)` ¶

`from_pretrained(repo_name, access_token=None, version=None, state_transform=None, load_gradients=False)` `classmethod` ¶

`publish_to_hub(repo_name, folder_path, access_token=None)` ¶

`ETDHPActor(n_state, n_control, hidden_sizes=(10, 10), u_bound=1.0, init_scale=0.5)` ¶

`forward(state)` ¶

`ETDHPCritic(n_state, hidden_sizes=(10, 10), init_scale=0.5)` ¶

`forward(state)` ¶

`PlantModelNN(n_state, n_control, hidden_sizes=(10, 10), init_scale=0.5)` ¶

`forward(xu)` ¶

`EventTrigger(rho=0.1, trigger_first_step=True, min_floor=0.0)` ¶

`reset()` ¶

`threshold(step)` ¶

`should_trigger(state_measured, step)` ¶