Same Model, Different Curvatures

The question behind the number

How easy will a neural network be to train? A curve showing that a loss decreased tells us what happened in a particular run. Explaining why it happened requires another connection: between the quantity we inspect and the dynamics we want to understand.

Our starting point was The power of quantum neural networks, by Abbas and colleagues. The paper investigates effective dimension and Fisher information in comparisons of classical and quantum neural networks. Its discussion connects Fisher geometry with trainability through a conditional relationship to the Hessian, the matrix describing local loss curvature. The author-hosted manuscript says “for certain loss functions”; the publisher supplement also retains a qualification. This case does not attribute an unconditional equality to the authors. Paper; inspected manuscript, PDF page 2; source coordinates.

We followed a smaller question:

At the same model point, does the Fisher quantity describe the curvature of the loss calculated against the actual, fixed training labels?

Hold the model still; inspect both quantities

We made one new classical calculation using the small network architecture in the archived code and a pinned version of the Iris dataset. It has 8 trainable weights and uses 100 examples from two species. This is our modern-runtime run, not an authenticated replay of the authors' experiment. Archived code; exact method.

At the saved initial parameter point, we examined one direction: the actual first optimizer update, rescaled to unit length. The input rows, weights, labels, direction and averaging rule were held fixed.

The Fisher term is positive. The training-loss curvature is negative. Both refer to the same parameter point and direction; they are different quantities, not rival answers to an identical calculation. Full-precision values and the contributing rows are in the checkpoint. The recompute companion checks the model as well as the arithmetic.

This point was selected for explanation after inspecting the run. It establishes a concrete distinction, not how often the distinction occurs across seeds or models. Negative curvature here also does not identify a saddle point: we have not established stationarity.

Where the difference comes from

For binary cross-entropy, write the difference between the two output scores as $r$, the model probability as $p$, and the fixed label as $y$. At a smooth parameter point:

The extra term retains a relationship between the fixed labels, the model's current predictions, and how its output bends as several weights change together.

For the model-label expectation used to define this Fisher matrix, the mean of $p-y$ is zero at the same parameter point. For the fixed species labels in our batch, that cancellation is not guaranteed. In the saved example, the extra term is large enough to reverse the sign of the total curvature.

This is not recovered by calling both matrices “geometry.” The expectation and the labels are part of what each matrix means. The methods note gives the full identity, its smoothness condition, and a control in which the residual cancels.

What this does—and does not—say about the paper

Our calculation makes the conditional bridge concrete. It does not show that the authors asserted a universal identity, and it does not refute their reported quantum/classical results.

There is also a separate reproduction boundary. The publisher supplement describes initialization over $[-1,1]^d$, while the inspected classical training file initializes weights over $[0,1]$. Our run follows the latter and records its own choices. These materials alone do not establish which initialization produced the historical experiment, or why the specifications differ. Changing that interval now would create another run, not retroactively identify the old one. Supplement §3.3 and code coordinates.

Nor does a local second derivative explain an entire training trajectory. The gradient, update size, optimizer state and movement between parameter regions still matter. We have not demonstrated a cause of training speed, a ranking of architectures, or a consequence for quantum hardware.

A more useful handover

The lesson is not to discard Fisher information. It is to carry the conditions of its use along with it.

Before turning a geometric summary into an explanation of training, ask: Which loss, which labels or expectation, which parameter point, which coordinates—and what condition permits the substitution? If those relationships are not established, keep the two quantities separate.

Here, the additional calculation changes the question from “Is the geometric number correct?” to “Which relationship does this correct number preserve?” That is a smaller claim than judging an entire paper, but it is one a reader can actually inspect.

Return to Cases →