Active-Adaptive Incremental Nonlinear Dynamic Inversion (AA-INDI)¶

AA-INDI is a fault-tolerant flight controller built on top of Incremental Nonlinear Dynamic Inversion (INDI). It combines a classical INDI control law with online Variable-Forgetting-Factor RLS identification of the control-effectiveness matrix so that the controller adapts quickly to actuator faults, and a lightweight sensor-filter surrogate that mimics the OTSEKF-HOSM branch of the reference paper. See also the nonlinear F-16 model: NonlinearLongitudinalF16.

Reference: Sun et al., "Active Incremental Nonlinear Dynamic Inversion for Sensor and Actuator Fault Diagnosis and Fault-Tolerant Flight Control", TU Delft Aerospace, research.tudelft.nl.

Key ideas¶

INDI control law: the applied control increment \(\Delta u = G^+ \cdot (\nu_{\text{des}} - \dot{\omega}_{\text{meas}})\) requires only the control-effectiveness matrix \(G\), not the full nonlinear dynamics \(f\). This eliminates model-uncertainty sensitivity.
Reference model: a second-order filter shapes the commanded angular rate into a smooth desired rate and its derivative \(\nu_{\text{des}} = \dot{\omega}_{\text{ref}}\).
VFF-RLS: the forgetting factor \(\lambda_k\) contracts toward a lower bound when the prediction residual grows (fast adaptation during faults/manoeuvres) and relaxes toward the upper bound in quiet operation (noise rejection).
Sensor-filter surrogate: a low-pass differentiator produces \(\dot{\omega}\) from raw \(\omega\), and a residual-based bias estimator yields a coarse IMU bias that the agent subtracts from measurements — a minimal stand-in for the paper's OTSEKF-HOSM stack.

Aspect	INDI	Adaptive INDI	AA-INDI
Control-effectiveness \(G\)	Offline / fixed	Online (basic RLS)	Online VFF-RLS
Sensor fault handling	None	None	Bias estimator (OTSEKF-HOSM surrogate)
Reaction to abrupt faults	Poor	Moderate	Fast (λ contracts under large residuals)
Noise rejection in nominal flight	Good	Moderate	Good (λ relaxes to max)

AA-INDI components¶

Component	Role	Implementation
VFFRLSEstimator	Online identification of \(G = \partial \dot{\omega}/\partial u\) with variable forgetting	`tensoraerospace.agent.aa_indi.VFFRLSEstimator`
LowPassDerivative	Causal differentiator (HOSM surrogate)	`tensoraerospace.agent.aa_indi.LowPassDerivative`
BiasEstimator	Exponential-forgetting IMU-bias estimator	`tensoraerospace.agent.aa_indi.BiasEstimator`
Reference model	2nd-order filter for \(\nu_{\text{des}}\)	Inline in `AAINDIAgent`
AAINDIAgent	Orchestrates INDI law, estimators, filter	`tensoraerospace.agent.aa_indi.AAINDIAgent`

Algorithm¶

On each control tick \(k\), given the measurement \(\omega_k\) and command \(r_k\):

Measurement conditioning. Subtract the current bias estimate (if enabled): \(\omega_k^c = \omega_k - \hat{b}\). The low-pass differentiator yields \(\dot{\omega}_k^{\text{meas}}\) (advanced inside learn() to avoid double-stepping).
Reference model. Second-order filter:

\[ \ddot{r} = -2\zeta\omega_n \dot{r} + \omega_n^2 (r_{\text{cmd}} - r), \qquad \nu_{\text{des}} = \dot{r}. \]

INDI law.

\[ \Delta u = G^{+} \cdot (\nu_{\text{des}} - \dot{\omega}^{\text{meas}}), \qquad u = \mathrm{clip}(u_{\text{prev}} + \Delta u,\ \pm u_{\max}), \]

with \(\Delta u\) first rate-limited to \(\pm\dot{u}_{\max} \cdot dt\). 4. VFF-RLS update. From \((\Delta u_k, \Delta \dot{\omega}_k)\):

\[ \varepsilon = \Delta \dot{\omega} - \theta^{\top} \Delta u,\qquad \lambda_k = \mathrm{clip}\bigl(e^{-\|\varepsilon\|^2/\sigma_\varepsilon^2},\ \lambda_{\min},\ \lambda_{\max}\bigr), \]

followed by the usual RLS gain / covariance recursion with forgetting factor \(\lambda_k\). 5. Bias update. Exponential moving average of the residual between \(\omega\) and its reintegration from \(\dot{\omega}\).

Quick start¶

import numpy as np
from tensoraerospace.agent.aa_indi import AAINDIAgent, AAINDIConfig

# Onboard model snapshot of the control-effectiveness matrix at design trim.
G_init = np.array([[-2.0, 0.1, 0.0],
                   [0.05, -1.5, 0.2],
                   [0.0,  0.05, -0.9]])

cfg = AAINDIConfig(
    dt=0.01,
    ref_wn=5.0,
    ref_zeta=0.7,
    u_magnitude_limit=25.0,
    u_rate_limit=200.0,
    vff_forgetting_min=0.9,
    vff_forgetting_max=0.999,
    vff_eps_sensitivity=2.0,
    sensor_cutoff_hz=50.0,
    enable_bias_correction=True,
    G_init=G_init,
    seed=0,
)
agent = AAINDIAgent(n_state=3, n_control=3, config=cfg)

omega = np.zeros(3)
ref = np.array([0.2, -0.1, 0.05])  # rad/s targets for roll/pitch/yaw rates

for k in range(500):
    u = agent.predict(omega, ref, k)
    # Plant step (placeholder — plug your environment here)
    omega = omega + cfg.dt * (G_init @ u)
    metrics = agent.learn(omega, ref, k)

Warm-start G_init matters

INDI needs a reasonable \(G\) on the first few ticks — with the default random init, the pseudo-inverse explodes and the actuator saturates before VFF-RLS has converged. Provide G_init from a linearised on-board model.

Hyperparameters¶

Reference model¶

Parameter	Default	Description
`ref_wn`	10.0	Natural frequency of the reference filter (rad/s). Higher → faster tracking, larger Δu.
`ref_zeta`	0.7	Damping ratio. 0.7 gives a critically-damped-ish response.

Actuator bounds¶

Parameter	Default	Description
`dt`	0.01	Control step (s)
`u_magnitude_limit`	25.0	Hard magnitude clamp per channel (same units as env action)
`u_rate_limit`	60.0	Max Δu per second per channel
`pinv_rcond`	1e-6	Cutoff for `np.linalg.pinv(G)`
`G_init`	None	Warm-start of shape `(n_state, n_control)`

VFF-RLS¶

Parameter	Default	Description
`vff_forgetting_min`	0.7	Lower bound on λ — fast-adaptation regime
`vff_forgetting_max`	0.999	Upper bound on λ — noise-rejection regime
`vff_eps_sensitivity`	1.0	Residual norm at which λ drops ~1/e
`vff_cov_init`	1e2	Initial covariance scale

Sensor filter¶

Parameter	Default	Description
`sensor_cutoff_hz`	10.0	Low-pass cutoff of the differentiator
`bias_forgetting`	0.99	EMA retention of the bias estimator
`enable_bias_correction`	True	Subtract bias estimate from ω before forming the INDI residual

Supported environments¶

Any Gymnasium env whose observation vector contains measurable angular rates (e.g. [alpha, wz] in NonlinearLongitudinalF16-v0 after light shaping, or a full [p, q, r] vector from a 6-DoF plant).

Persistence¶

Same API as the other adaptive-critic agents:

run_dir = agent.save("./checkpoints")        # creates <date>_AAINDIAgent/
restored = AAINDIAgent.from_pretrained(run_dir)
agent.publish_to_hub("me/my-aaindi", folder_path=run_dir, access_token="hf_...")

Saved artefacts:

config.json — full AAINDIConfig + n_state / n_control.
vff_rls.npz — RLS θ, covariance P, last forgetting factor λ, update counter.
bias_state.npz — exponential bias estimate.
deriv_state.npz — low-pass differentiator state.
loop_state.npz — reference-model state, PI integrator, last applied control, cached ω̇. Persisting these means a mid-episode save resumes bit-identically on reload (essential when ref_error_kp / ref_error_ki are non-zero).

API reference¶

`AAINDIAgent(n_state, n_control, config=None)` ¶

Active-Adaptive INDI control agent.

Parameters:

Name	Type	Description	Default
`n_state`	`int`	Dimension of the controlled angular state vector `ω` (e.g. 3 for roll/pitch/yaw rates).	required
`n_control`	`int`	Number of control channels `u` (e.g. aileron, elevator, rudder).	required
`config`	`AAINDIConfig \| None`	:class:`AAINDIConfig` instance. Defaults to moderately conservative values suitable for a sub-sonic fixed-wing aircraft.	`None`

`reset()` ¶

Clear the per-episode rolling state (keeps learned G estimate).

`predict(omega, reference, time_step=0, *, deterministic=True)` ¶

Compute the commanded control for the current step.

Parameters:

Name	Type	Description	Default
`omega`	`ndarray`	Measured angular state ω (shape `(n_state,)`).	required
`reference`	`ndarray`	Commanded angular rate ω_cmd. Accepted shapes are `(n_state,)` (a single command) or `(n_state, T)` / `(T,)` (a schedule, from which column `time_step` is read).	required
`time_step`	`int`	Current step index; used to slice a time-varying reference.	`0`
`deterministic`	`bool`	Unused — kept for API parity with stochastic agents.	`True`

Returns:

Type	Description
`ndarray`	Control command `u` of shape `(n_control,)`.

`learn(next_omega, reference, time_step=0)` ¶

Update the online estimators from the newly observed state.

Must be called once per environment step, after :meth:predict and the corresponding env.step(u) call.

Parameters:

Name	Type	Description	Default
`next_omega`	`ndarray`	Angular state measured at `t+1`.	required
`reference`	`ndarray`	Commanded reference (unused at learn time — accepted only for API parity with other agents).	required
`time_step`	`int`	Same index passed to :meth:`predict`.	`0`

Returns:

Type	Description
`dict[str, float]`	Dict of scalar metrics: prediction residual norm, current
`dict[str, float]`	forgetting factor, norm of the identified G, and current
`dict[str, float]`	bias estimate norm.

`get_param_env()` ¶

Build a JSON-serialisable config dict for :meth:save.

`save(path=None)` ¶

Write the agent to a directory.

Files produced

config.json — agent/config metadata.
vff_rls.npz — RLS parameter matrix theta, covariance P, last forgetting factor, update counter.
bias_state.npz — current bias estimate.
deriv_state.npz — low-pass differentiator internal state.
loop_state.npz — reference-model state, PI integrator, last applied control, cached ω̇. Persisting these means a saved agent resumes bit-identically on reload mid-episode (essential when ref_error_kp/ref_error_ki are non-zero).

Parameters:

Name	Type	Description	Default
`path`	`Union[str, Path, None]`	Base directory (`None` → CWD).	`None`

Returns:

Type	Description
`str`	Absolute path to the created run directory.

`from_pretrained(repo_name, access_token=None, version=None)` `classmethod` ¶

Load an agent from a local directory or Hugging Face Hub.

Parameters:

Name	Type	Description	Default
`repo_name`	`str`	Local folder path, or `namespace/repo_name` on the Hugging Face Hub.	required
`access_token`	`Optional[str]`	Hub access token for private repos.	`None`
`version`	`Optional[str]`	Hub revision / branch / tag.	`None`

Returns:

Name	Type	Description
`AAINDIAgent`	`'AAINDIAgent'`	Reconstructed agent.

`publish_to_hub(repo_name, folder_path, access_token=None)` ¶

Upload a :meth:save directory to the Hugging Face Hub.

Parameters:

Name	Type	Description	Default
`repo_name`	`str`	Target repository id, e.g. `"me/my-aaindi"`.	required
`folder_path`	`Union[str, Path]`	Local folder produced by :meth:`save`.	required
`access_token`	`Optional[str]`	Hub access token.	`None`

`AAINDIConfig(dt=0.01, ref_wn=10.0, ref_zeta=0.7, u_magnitude_limit=25.0, u_rate_limit=60.0, vff_forgetting_min=0.7, vff_forgetting_max=0.999, vff_eps_sensitivity=1.0, vff_cov_init=100.0, sensor_cutoff_hz=10.0, bias_forgetting=0.99, enable_bias_correction=True, pinv_rcond=1e-06, G_init=None, ref_error_kp=0.0, ref_error_ki=0.0, seed=None, history=dict())` `dataclass` ¶

Hyper-parameters for :class:AAINDIAgent.

Parameters:

Name	Type	Description	Default
`dt`	`float`	Simulation / control step [s].	`0.01`
`ref_wn`	`float`	Reference-model natural frequency [rad/s]. Higher values track aggressive reference changes faster at the cost of larger control increments.	`10.0`
`ref_zeta`	`float`	Reference-model damping ratio. Default `0.7` gives the critically-damped-ish response used throughout the paper.	`0.7`
`u_magnitude_limit`	`float`	Hard magnitude clamp on the control output (per-channel). Matches the actuator envelope of the plant.	`25.0`
`u_rate_limit`	`float`	Maximum Δu per step (per-channel). Limits how far the incremental law can move the actuator in one control tick.	`60.0`
`vff_forgetting_min`	`float`	Lower bound on the VFF-RLS forgetting factor. Smaller values react to faults sooner.	`0.7`
`vff_forgetting_max`	`float`	Upper bound on the VFF-RLS forgetting factor. Larger values tune how aggressively old data is kept for noise rejection.	`0.999`
`vff_eps_sensitivity`	`float`	Residual norm at which the forgetting factor has dropped to `1/e` of its peak.	`1.0`
`vff_cov_init`	`float`	Initial covariance scale for the RLS.	`100.0`
`sensor_cutoff_hz`	`float`	Cut-off of the low-pass differentiator used to produce ω̇_meas from raw ω.	`10.0`
`bias_forgetting`	`float`	Exponential-forgetting parameter of the bias estimator (closer to 1 → slower, smoother tracking).	`0.99`
`enable_bias_correction`	`bool`	When True, subtract the estimated IMU bias from the angular-rate measurement before forming the INDI residual.	`True`
`pinv_rcond`	`float`	Cut-off for the pseudo-inverse of `G` (passed to `numpy.linalg.pinv`). Prevents blowing up when `G` has near-zero singular values during the RLS warm-up.	`1e-06`
`G_init`	`Optional[ndarray]`	Optional warm-start for the control-effectiveness matrix, shape `(n_state, n_control)`. INDI needs a reasonable `G` on the first few ticks — in a real deployment this would come from a linearised on-board model. When `None` the RLS starts from tiny random weights and the controller is quiet for a handful of steps until the identifier has converged.	`None`
`ref_error_kp`	`float`	Proportional feedback gain that injects the reference-model tracking error `(r_ref − ω)` into the INDI desired acceleration. Without it pure INDI is a type-0 loop for `ω` and settles with a small residual offset (≈ 10 % on the F-16 longitudinal channel) driven by actuator lag / rate limiting. Set to a value of order `ref_wn` to close the gap; leave at `0.0` for the textbook pure-INDI behaviour.	`0.0`
`ref_error_ki`	`float`	Integral feedback gain on the reference tracking error. Eliminates residual offset from persistent modelling error / constant disturbances (e.g. a stuck trim). Start at `0` and raise cautiously — too large trades steady-state accuracy for wind-up.	`0.0`
`seed`	`int \| None`	Optional RNG seed.	`None`

`VFFRLSEstimator(n_y, n_u, forgetting_min=0.7, forgetting_max=0.999, eps_sensitivity=1.0, cov_init=100.0, theta_init_scale=0.001, seed=None)` ¶

VFF-RLS identifier for the control-effectiveness matrix G.

Internally stores the parameter matrix theta of shape (n_u, n_y) such that y ≈ θᵀ · φ where φ = Δu (length n_u) and y = Δω̇ (length n_y). Under that convention G = θᵀ, which is the usual row-action convention for INDI.

Parameters:

Name	Type	Description	Default
`n_y`	`int`	Dimension of the output Δω̇.	required
`n_u`	`int`	Dimension of the control increment Δu.	required
`forgetting_min`	`float`	Lower bound on λ — reached under strong innovations (fast adaptation).	`0.7`
`forgetting_max`	`float`	Upper bound on λ — reached under quiescent operation (noise rejection).	`0.999`
`eps_sensitivity`	`float`	Scale of the residual norm at which λ falls off significantly. Smaller values make the forgetting factor more reactive to transients.	`1.0`
`cov_init`	`float`	Initial scale of the covariance matrix `P₀ = cov_init · I`.	`100.0`
`theta_init_scale`	`float`	Standard deviation of a zero-mean Gaussian used to randomly initialise `theta`. Small non-zero values break symmetry for downstream matrix operations.	`0.001`
`seed`	`int \| None`	RNG seed for the initial `theta`.	`None`

`G` `property` ¶

Return the control-effectiveness matrix of shape (n_y, n_u).

`reset_covariance()` ¶

Restore P to its initial large-variance state.

`update(du, dy)` ¶

Run one VFF-RLS step.

Parameters:

Name	Type	Description	Default
`du`	`ndarray`	Control increment `u_t − u_{t-1}` (shape `(n_u,)`).	required
`dy`	`ndarray`	Measured output increment `ω̇_t − ω̇_{t-1}` (shape `(n_y,)`).	required

Returns:

Type	Description
`ndarray`	The prediction residual `ε = dy − θᵀ du` before the
`ndarray`	update.

`predict(du)` ¶

Predict Δω̇ from a candidate control increment Δu.

`LowPassDerivative(n, dt, cutoff_hz=10.0)` ¶

Causal finite-difference differentiator with a low-pass filter.

Computes ω̇_t from a sequence of ω_t readings using the first-order backward difference (ω_t − ω_{t-1}) / dt followed by an exponential filter with cut-off set by cutoff_hz. The filter is a discrete first-order IIR with α = dt · 2π · cutoff.

Parameters:

Name	Type	Description	Default
`n`	`int`	Dimension of the input signal.	required
`dt`	`float`	Sampling period [s].	required
`cutoff_hz`	`float`	Low-pass cut-off frequency [Hz]. Values in 5–20 Hz are typical for sub-sonic flight envelopes.	`10.0`

`reset()` ¶

Clear the internal filter state.

`step(x)` ¶

Ingest a new sample and return the filtered derivative estimate.

`BiasEstimator(n, forgetting=0.99)` ¶

Exponential-forgetting mean of an innovation signal.

Used by :mod:aa_indi to produce a scalar IMU bias estimate b̂ from the residual between the raw measurement and the reintegrated-from-derivative one. When an actual bias appears, the residual has a non-zero mean and b̂ tracks it with a time constant of roughly dt / (1 − lambda).

Parameters:

Name	Type	Description	Default
`n`	`int`	Dimension of the innovation.	required
`forgetting`	`float`	Exponential-moving-average retention (`0 < λ < 1`). Values near 1 average over long windows (slow bias tracking); smaller values react faster at the cost of noisier estimates.	`0.99`

`update(innovation)` ¶

Update the bias estimate with a new innovation sample.

Sources¶

Sun et al. "Active Incremental Nonlinear Dynamic Inversion for Sensor and Actuator Fault Diagnosis and Fault-Tolerant Flight Control", TU Delft Aerospace, research.tudelft.nl.
Smeur, Chu, de Croon. "Adaptive Incremental Nonlinear Dynamic Inversion for Attitude Control of Micro Air Vehicles", J. Guid. Control Dyn., 2016.
Fortescue, Kershenbaum, Ydstie. "Implementation of Self-Tuning Regulators with Variable Forgetting Factors", Automatica, 1981.

Active-Adaptive Incremental Nonlinear Dynamic Inversion (AA-INDI)¶

Key ideas¶

Differences from related methods¶

AA-INDI components¶

Algorithm¶

Quick start¶

Hyperparameters¶

Reference model¶

Actuator bounds¶

VFF-RLS¶

Sensor filter¶

Supported environments¶

Persistence¶

API reference¶

AAINDIAgent(n_state, n_control, config=None) ¶

reset() ¶

predict(omega, reference, time_step=0, *, deterministic=True) ¶

learn(next_omega, reference, time_step=0) ¶

get_param_env() ¶

save(path=None) ¶

from_pretrained(repo_name, access_token=None, version=None) classmethod ¶

publish_to_hub(repo_name, folder_path, access_token=None) ¶

VFFRLSEstimator(n_y, n_u, forgetting_min=0.7, forgetting_max=0.999, eps_sensitivity=1.0, cov_init=100.0, theta_init_scale=0.001, seed=None) ¶

G property ¶

reset_covariance() ¶

update(du, dy) ¶

predict(du) ¶

LowPassDerivative(n, dt, cutoff_hz=10.0) ¶

reset() ¶

step(x) ¶

BiasEstimator(n, forgetting=0.99) ¶

update(innovation) ¶

Sources¶

`AAINDIAgent(n_state, n_control, config=None)` ¶

`reset()` ¶

`predict(omega, reference, time_step=0, *, deterministic=True)` ¶

`learn(next_omega, reference, time_step=0)` ¶

`get_param_env()` ¶

`save(path=None)` ¶

`from_pretrained(repo_name, access_token=None, version=None)` `classmethod` ¶

`publish_to_hub(repo_name, folder_path, access_token=None)` ¶

`VFFRLSEstimator(n_y, n_u, forgetting_min=0.7, forgetting_max=0.999, eps_sensitivity=1.0, cov_init=100.0, theta_init_scale=0.001, seed=None)` ¶

`G` `property` ¶

`reset_covariance()` ¶

`update(du, dy)` ¶

`predict(du)` ¶

`LowPassDerivative(n, dt, cutoff_hz=10.0)` ¶

`reset()` ¶

`step(x)` ¶

`BiasEstimator(n, forgetting=0.99)` ¶

`update(innovation)` ¶