3. Background-averaged interactions¶

Purpose¶

Compute the Walsh–Hadamard transform (WHT) of a genotype–phenotype landscape.
This function automatically selects between:

1. Exact FWHT (Fast Walsh–Hadamard Transform) — used only when:

   • the landscape is complete (contains all $2^N$ genotypes),
   • and has no missing values (no NaNs).

2. Fallback projection — used when:

   • the landscape is incomplete (missing genotypes),
   • or contains NaNs.

The function provides a unified interface so the user doesn't have to choose manually unless desired.

Inputs¶

values : np.ndarray¶

Phenotype values of shape (M,) or (M, R).
If 1D, it is reshaped internally to (M, 1).
May contain NaNs (which triggers fallback mode).

states : np.ndarray¶

Genotype matrix of shape (M, N) with binary entries {0,1}.

N : int¶

Number of loci in the system.

strategy : {"auto", "fwht", "fallback"}, default "auto"¶

• "auto":
  Uses FWHT when the landscape is complete and NaN-free; otherwise uses fallback.
• "fwht":
  Forces exact FWHT; raises an error if impossible.
• "fallback":
  Forces approximate projection even if FWHT is available.

Outputs¶

The function returns a 4-tuple:

1. fs — Walsh–Hadamard coefficients¶

Shape (2^N,) or (2^N, R) depending on input.
If values was 1D, it is squeezed back to 1D.

2. s_bits — Mode bits¶

Array of shape (2^N, N) containing the binary representation of each Walsh mode.
Each row corresponds to a vector

\begin{equation} s \in \{0,1\}^N. \end{equation}

3. orders — Interaction order of each coefficient¶

Defined as

\begin{equation} \text{order}(s) = \sum_{i=1}^N s_i. \end{equation}

Thus:

• order 0 = baseline
• order 1 = additive
• order 2 = pairwise epistasis
• …
• order N = full N-way interaction

4. meta — Metadata¶

A dictionary describing what was done:

• "path" — "fwht" or "fallback"
• "reordered" — whether rows were lexicographically sorted
• "mask_rows" — boolean mask of included rows for fallback

Automatic logic¶

The "auto" strategy follows:

if M == 2^N and values contain no NaNs: use FWHT (exact) else: use fallback projection (approximate)

This guarantees the WHT is computable for any dataset.

Purpose¶

Compute Walsh–Hadamard coefficients $f_\mathbf{s}$ for all modes $\mathbf{s} \in \{0,1\}^N$, including:

• replicate-level coefficients,
• summary statistics (mean, experimental and biological SD),
• bootstrap-based uncertainty bands,
• null-model tests and signal-to-noise ratios,

mirroring the APIs of focal_effect and epistasis_k.

This function is the main entry point to analyze the global structure of the landscape in Walsh space.

Inputs¶

L : Landscape¶

Landscape object with:

• L.states — genotypes, shape $(M, N)$ with entries in $\{0,1\}$
• L.values — replicate phenotype values, shape $(M, R)$

At least 2 replicates are required if any bootstrap is enabled.

Robustness options¶

These are passed to the internal cleaning routine _clean_landscape_for_wh:

• missing_policy : {"error","drop"}, default "error"
  Reserved for future handling of incomplete landscapes; currently used only for basic sanity checks during cleaning.

• nan_policy : {"omit","propagate"}, default "omit"
  How to treat NaNs before computing the transform:

  • "omit": handle NaNs in a robust way (e.g. replacing by 0 for the transform).
  • "propagate": NaNs are allowed to propagate into the coefficients when appropriate.

Bootstrap options¶

Uncertainty bootstrap (around the inferred landscape $\hat F$)¶

• B_uncertainty : int, default $0$
  Number of bootstrap draws to estimate the sampling distribution of $f_s$.

• uncertainty_flavor : {"iid","wildcluster"}, default "iid"

  • "iid": bootstrap each state independently.
  • "wildcluster": preserve replicate-level covariance (recommended when biological correlations exist).

This produces:

• per-mode uncertainty CIs: ci_low, ci_high
• sign probabilities: sign_prob_pos, sign_prob_neg
  giving $\Pr(f_s > 0)$ and $\Pr(f_s < 0)$.

Null bootstrap (noise-only model in Walsh space)¶

• B_null : int, default 0
  Number of bootstrap samples drawn from an observed null model: landscapes with no biological signal but realistic measurement noise.

• consider_bio_corr : bool, default False

  • False: i.i.d. null — breaks cross-state correlations.
  • True: wild-cluster null — preserves empirical cross-state covariance.

• multipliers : {"rademacher","normal"}, default "rademacher"
  Distribution of wild bootstrap multipliers.

• ci_level : float, default 0.95
  Confidence level used for both uncertainty and null CIs.

• rng : np.random.Generator | None
  Optional RNG for reproducibility.

The null bootstrap yields:

• per-mode null CIs: null_ci_low, null_ci_high
• global variance test across modes:
  • var_obs — observed variance of mean across modes
  • var_null_median — median variance under null
  • p_value_var — (\Pr(\mathrm{Var}{\text{null}} \ge \mathrm{Var}{\text{obs}}))
  • effect_size_var — difference between observed and typical null variance
• per-mode signal-to-null-noise ratio: snr_null

Output options¶

• as_dataframe : bool, default True

Controls the return type:

• If True: returns a tidy DataFrame (pretty format) with one row per Walsh mode.
• If False: returns a WalshResults dataclass with all numerical arrays and metadata.

Returned object¶

Depending on as_dataframe:

1. If as_dataframe=True¶

A pd.DataFrame summarizing for each mode:

• Walsh index and binary pattern (s_bits, orders)
• Walsh coefficient (mean)
• Experimental SD, Biological SD
• Uncertainty CI (low), Uncertainty CI (high) (if B_uncertainty > 0)
• Null CI (low), Null CI (high) (if B_null > 0)
• Signal-to-Null-Noise (SNR)
• P-value (variance excess) and related null-test fields (if B_null > 0)
• Sign probabilities (Prob(Effect > 0), Prob(Effect < 0))

This format is ideal for plotting spectra, filtering modes, or exporting results.

2. If as_dataframe=False¶

A WalshResults object with:

• values — replicate-level coefficients (f_s^{(r)})
• mean, std_experimental, std_biological
• s_bits, orders — structural information about each mode
• ci_low, ci_high, null_ci_low, null_ci_high
• sign_prob_pos, sign_prob_neg, snr_null
• var_obs, var_null_median, p_value_var, effect_size_var
• meta — dictionary with all settings and bootstrap arrays (fs_unc_b, fs_null_b, etc.)
• feature_names — copied from L.feature_names if present

Purpose¶

Compute the epistasis amplitude spectrum
\begin{equation} \langle E^2 \rangle_k,\quad k = 1,2,\dots,N \end{equation} including:

• point estimates from the Walsh–Hadamard coefficients of the empirical landscape,
• uncertainty bootstrap (around the inferred phenotype),
• null bootstrap (noise-only epistasis spectrum),
• per–order confidence intervals, SNR, and global variance tests.

This is the amplitude decomposition used in the literature (e.g. Otwinowski et al., Neidhart et al., Husain & Murugan), and fully matches the statistical structure of epistasis_k, focal_effect, and walsh_coefficients.

Definition¶

For each interaction order $k \ge $1$$:

\begin{equation} \langle E^2 \rangle_S \;=\; \frac{4^k}{\binom{N}{k}} \sum_{\lvert s \rvert = S} f_s^2, \end{equation}

where:

• $f_s$ = Walsh–Hadamard coefficient of mode $s$,
• $\lvert s\rvert$ = interaction order $k$,
• the sum is over all modes with Hamming weight $k$,
• the prefactor $4^k$ rescales the Walsh variance to biological epistasis amplitude.

This produces one number per order, summarizing how much epistatic variance is present at each interaction scale.

Inputs¶

L : Landscape¶

Landscape with:

• L.states — genotype matrix (M, N)
• L.values — replicate matrix (M, R)

Requires R ≥ 2 if bootstraps are enabled.

Robustness options¶

• missing_policy : {"error","drop"}
  Used in cleaning; currently validation only.

• nan_policy : {"omit","propagate"}
  Forwarded to _clean_landscape_for_wh().

Uncertainty bootstrap¶

Parameters:

• B_uncertainty — number of bootstrap samples
• uncertainty_flavor : {"iid","wildcluster"}
• multipliers : {"rademacher","normal"}
• ci_level — confidence level
• rng — optional RNG

Produces for each order $S$:

• CI low, CI high — bootstrap confidence intervals
• distribution of amplitude fluctuations under measurement noise

This captures uncertainty in $\hat F$.

Null bootstrap¶

Parameters:

• B_null — number of null samples
• consider_bio_corr — preserve vs. remove biological covariance
• multipliers
• ci_level

Produces for each order $k$:

• Null CI low, Null CI high
• SNR (null) — detection strength:
  [ \text{SNR}_k = \frac{\langle E^2\rangle_k}{\sigma_k^{\text{null}}} ]
• global variance test across orders:
  • Variance (obs)
  • Variance (null median)
  • P-value var

This tests whether the amplitude structure exceeds what noise alone produces.

Output¶

If as_dataframe=True (default)¶

A tidy table:

| Order | Epistasis amplitude | CI low | CI high | Null CI low | Null CI high | SNR(null) | … |

One row per interaction order $k$.

If as_dataframe=False¶

A dict:

{
    "orders": array([...]),
    "amplitude": array([...]),
    "ci_low": ...,
    "ci_high": ...,
    "null_ci_low": ...,
    "null_ci_high": ...,
    "snr_null": ...,
    "var_obs": ...,
    "var_null_median": ...,
    "p_value_var": ...
}

This yields the epistasis spectrum, uncertainty, null model, and strengths of detection for every order $k$.

Purpose¶

Compute the variance decomposition by interaction order:

\begin{equation} V(S) \;=\; \sum_{\lvert s\rvert = S} f_s^2, \end{equation}

where $f_s$ are the Walsh–Hadamard coefficients and $S = 1,2,\dots,N$.
This gives the amount of phenotypic variance contributed by epistatic interactions of each order.

The function supports:

• uncertainty bootstrap (around the inferred phenotype $\hat F$),
• null bootstrap (noise-only landscapes),
• confidence intervals for both uncertainty and null,
• SNR per order,
• global variance test across orders,
• option to normalize the spectrum as

\begin{equation} V_{\mathrm{frac}}(S) = \frac{V(S)}{\sum_T V(T)} \end{equation}

producing a fractional epistasis spectrum.

Definitions¶

Variance by interaction order¶

\begin{equation} V(S) = \sum_{|s|=S} f_s^2 \end{equation}

This reflects how much epistasis exists at order (S).
It is the biologically standard variance decomposition.

Fraction of variance (optional)¶

If as_fraction=True:

\begin{equation} v_{\mathrm{frac}}(S) = \frac{V(S)}{\sum_T V(T)}. \end{equation}

This shows what percentage of total epistatic variance belongs to each order.

Inputs¶

Required¶

• L : Landscape
  • L.states — binary matrix (M, N)
  • L.values — replicates (M, R)
• Complete or incomplete landscapes supported (fallback projector is used).

Robustness options¶

• missing_policy : {"error","drop"}
  How to handle missing states for WH projection.
• nan_policy : {"omit","propagate"}
  How to treat NaNs before the transform.

Bootstrap options¶

Same API and meaning as in your other functions:

• B_uncertainty — bootstraps around (\hat F)
• uncertainty_flavor : {"iid","wildcluster"}
• B_null — noise-only bootstraps
• consider_bio_corr : bool
• multipliers : {"rademacher","normal"}
• ci_level
• rng

Output control¶

• as_fraction=True → return normalized spectrum (V_{\mathrm{frac}}(S)).
• as_dataframe=True → return tidy table (recommended).

Outputs¶

If as_dataframe=True, returns:

| Order | Variance V(S) or Fraction | CI low | CI high | Null CI low | Null CI high | SNR(null) | Variance (obs) | Variance (null median) | P-value var |

• Per-order uncertainty CI
• Per-order null CI
• SNR vs null
• Global variance excess test (across orders)

If as_dataframe=False, returns an equivalent raw dictionary.

Interpretation¶

• High V(S) → strong epistasis at order S
• High SNR(null) → order S is reliably detected above noise
• Null CIs tell what spectrum you'd expect from noise only
• Global p-value tests whether the pattern across S is more structured than noise

This is extremely useful to characterize:

• landscapes dominated by low-order interactions,
• globally epistatic landscapes (rank-1-like),
• idiosyncratic high-order epistasis,
• landscapes near your Kuramoto + exclusion or Ising-like generative models.

Notes

Order $\mathcal{S}=0$ (the global mean) is automatically removed.

Works seamlessly with incomplete landscapes via the fallback WH projection.

Statistical behaviour fully matches your other modules (epistasis, WH, null tests).