Methods: one saved point, two expectations

Object and provenance

This companion reconstructs one fixed parameter point, not a full training benchmark. The underlying new local run used one seed, a modern runtime, and the classical architecture found in the public archive. The authors' historical runtime, exact Iris bytes and shuffle order remain unknown. We have not regenerated their trials or executed a quantum network.

The exported point is the saved initial state, t=0; time here counts completed optimizer updates, not elapsed time. The direction is the saved first ADAM update divided by its Euclidean norm in the original eight-weight coordinate. It is not chosen to minimize the Hessian. The example was chosen after the existing results were inspected; no prevalence estimate follows.

Fully specified local input and model

Input: the exact iris.csv from scikit-learn 0.23.2, identified by SHA256 in SOURCES.md. Select source rows 0–99, with labels 0 (setosa) and 1 (versicolor), 50 each. Retain duplicate feature rows.
Preprocessing: divide each four-feature row by its L2 norm in float64; apply numpy.random.RandomState(0).permutation(100); cast to float32 and then float64. This rounding step is intentional. No imputation or standardization across the population.
Model: widths [4,1,1,1,2], no biases, LeakyReLU slope 0.01 at each hidden layer, raw output scores. The parameter order is four first-layer weights, one second-layer weight, one third-layer weight, and two output weights.
Parent run: PyTorch seed 0; weights initialized uniformly over [0,1]; mean full-batch cross-entropy; ADAM learning rate 0.1, betas (0.9,0.999), epsilon 1e-8, 100 updates. Numerical runtime: Python 3.13.5, NumPy 2.2.6, PyTorch 2.7.1+cpu, scikit-learn 1.7.2. The public companion starts from the exported weights; it does not claim to regenerate that initialization or all 100 updates.
Loss and parameter numbers here are dimensionless after this fixed preprocessing. A positive or negative curvature is not a good/bad performance label. No trace-normalized Fisher or effective-dimension calculation is performed by this companion.

The exact identity

For row $i$ , let $r_i=z_{i1}-z_{i0}$ , $p_i=\sigma(r_i)$ , and $y_i\in\{0,1\}$ . The mean fixed-label loss is

L(\theta)=\frac1N\sum_i\big[\operatorname{softplus}(r_i(\theta))-y_i r_i(\theta)\big].

For the empirical distribution of inputs but the model distribution of labels, the conditional Fisher is

F=\frac1N\sum_i p_i(1-p_i)\nabla r_i\nabla r_i^T.

This is not the outer product of the fixed, observed-label loss gradients. That alternative is often called an empirical Fisher and would be another quantity.

Differentiating the fixed-label loss gives

H=\nabla^2L=F+R,\qquad R=\frac1N\sum_i(p_i-y_i)\nabla^2r_i.

For the saved unit direction $d$ ,

d^THd=\frac1N\sum_i p_i(1-p_i)(D_dr_i)^2 +\frac1N\sum_i(p_i-y_i)D_d^2r_i.

All sums in the checkpoint use $N=100$ ; per-row contributions already include this denominator. An observed residual need not cancel even for a correctly implemented likelihood. Conversely, nonlinearity does not guarantee a nonzero total residual: cancellation is possible. The displayed inequality is measured at this selected point, not assumed from the model type.

Independent product-rule calculation

Inside a fixed activation branch, write

r_i=k_i a_i b c e, \quad a_i=x_i\cdot\theta_{0:4},\quad b=\theta_4, \quad c=\theta_5,\quad e=\theta_7-\theta_6.

Here $k_i$ is the product of the three local activation slopes. Set $A_i=x_i\cdot d_{0:4}$ , $B=d_4$ , $C=d_5$ , $D=d_7-d_6$ . Then

D_d^2r_i=2k_i(A_iBce+A_ibCe+A_ibcD+a_iBCe+a_iBcD+a_ibCD).

The six terms are retained separately for every row. Although the network is locally linear in one layer's weights with the others held fixed, it is not jointly affine in all eight weights. Active linear branches do not erase these mixed derivatives.

verify.py --input PATH compares this product rule against an automatically differentiated Hessian, the saved matrices, per-row contributions, loss and gradient. It also verifies that the saved next update matches the first ADAM update formula. Central differences of directional gradients provide a further check with step 1e-5; both displaced points must retain the original activation signs. The point must be more than 1e-12 from an activation kink in every recorded preactivation. Failing this test refuses the local identity check; it does not invent a derivative there.

As a control, the companion holds the model probabilities at the saved point fixed as soft targets and differentiates the resulting loss at that same point. Its residual term cancels, and its Hessian agrees with F. This is a changed target relation, not training on alternative labels.

What the companion can certify

The standard-library path checks hashes, matrix identities, unit direction, row identities, per-row algebra, sums and the displayed values. It alone does not reconstruct input features or establish the saved derivatives.

The optional NumPy/PyTorch path verifies those fixed-point derivatives from the exact public input. Its tolerances are atol=1e-10, rtol=1e-8, with atol=1e-8, rtol=1e-6 for the finite-difference check. Runtime versions are printed. Agreement within tolerance is not a bitwise reconstruction of another runtime.

Neither path verifies author intent, historical experiment identity, statistical typicality, an entire optimization trajectory, or a quantum/classical advantage. The initialization discrepancy is a source-level observation and must be checked against the separately linked supplement and code, not inferred from a passing numerical test.

The useful result is limited but direct: for this fixed model point and direction, substituting F for the fixed-label H would discard a signed residual large enough to change the curvature's sign.

Return to Cases →