§5. Singular Value Decomposition (SVD)
While L1 Lasso regularization yielded substantial unstructured sparsity, it failed to translate into tangible reductions in physical storage or inference latency. Standard edge hardware lacks the specialized sparse libraries required to skip randomly distributed zero-weights. To achieve authentic physical compression, we transitioned from unstructured penalties to low-rank matrix approximations via Singular Value Decomposition (SVD).
Mathematically, SVD factors a dense weight matrix into three distinct components: an orthogonal input projection, a diagonal matrix of singular values representing feature importance, and an orthogonal output projection. By retaining only the top k singular values, we discard highly collinear weights and effectively neutralize multicollinearity, yielding a low-rank approximation:
In our PyTorch implementation, to avoid incurring the computational overhead of three sequential matrix multiplications at runtime, we "pre-baked" the scaling factor by absorbing the diagonal singular value matrix (Σk) into the output projection matrix (Uk). This offline algebraic simplification condenses the structure into a clean bottleneck of merely two sequential linear layers:
SVD was strictly applied to the fully connected dense layers. Applying SVD to convolutional 4D tensors requires flattening operations that inherently destroy the spatial inductive biases necessary for computer vision tasks. Furthermore, our baseline PCA audit indicated that 93% to 97% of the network's structural redundancy was localized exclusively within these terminal dense layers.
5.1 The 95% Variance Fallacy
Standard machine learning practice dictates retaining enough singular values to capture 95% of the matrix energy (variance). However, in massively over-parameterized neural networks, this heuristic completely collapses. When we attempted to compute the 95% energy rank for the VGG19 dense layers, the spectrum analyzer triggered severe capacity warnings:
The analyzer revealed that even keeping 1,024 ranks (a massive matrix) failed to capture 95% of the energy in the first layer. A rank-1024 bottleneck still costs nearly 30 million parameters—offering negligible compression. Why does this happen? The answer lies in the "Scree Plot" phenomenon of neural network weights.
5.2 Singular Value Flattening (The Scree Elbow)
If we extract the raw top singular values from the first two fully connected layers, a distinct pattern emerges:
Notice the violent drop immediately after the first singular value (from 5.13 to 3.79, and 6.62 to 3.42), followed by a long, extremely flat tail where values barely decay. The top singular vectors capture the core distinct features, while the flat tail consists of highly collinear, redundant noise. By rigidly adhering to a 95% energy threshold, we are mathematically forced to memorize thousands of these useless flat-tail noise dimensions.
5.3 Aggressive 20% Energy Truncation
To determine the optimal truncation rank k, we abandoned the 95% rule and conducted a deep empirical energy threshold sweep. Sweeping the energy retention threshold from 50% down to 10%, we identified that preserving merely 20% of the matrix energy offered the optimal sparsity-performance tradeoff. The remaining 80% of the energy was analytically discarded as collinear noise.
This aggressive truncation collapsed the mathematical rank of the primary fully connected layer from 4,096 down to just 44. To explicitly quantify this compression: the original dense layer required 25,088 × 4,096 = 102.76M parameters. By substituting it with the pre-baked rank-44 bottleneck, the parameter count dropped to (25,088 × 44) + (44 × 4,096) = 1.28M parameters.
Similarly, the subsequent fully connected layer's rank was reduced from 4,096 to 31. Across the entire classification head, we physically eradicated over 98.7% of the linear dimensions and active weights, successfully clearing the massive parameter bottleneck.
As anticipated, this severe structural disruption initially degraded the minimum per-class F1 score to approximately 88%. To recover diagnostic performance, we executed a brief 10-epoch adaptation phase. The 16 convolutional layers were strictly frozen to preserve the integrity of the learned feature extractors, confining gradient updates exclusively to the newly initialized low-rank SVD layers. The model recovered its baseline accuracy within a single training epoch.
5.4 Blind Test Set Results
Scientific Observation Table
| Cell Class | Baseline F1 | SVD F1 | Δ F1 | SVD Recall |
|---|---|---|---|---|
| Basophil | 99.59% | 99.18% | −0.41% | 98.77% |
| Eosinophil | 99.92% | 99.84% | −0.08% | 100.00% |
| Erythroblast | 98.41% | 99.19% | +0.78% | 98.71% |
| Immature Gran. | 96.28% | 97.06% | +0.78% | 96.89% |
| Lymphocyte | 98.78% | 98.37% | −0.41% | 99.18% |
| Monocyte | 97.90% | 98.59% | +0.69% | 98.24% |
| Neutrophil | 97.73% | 97.98% | +0.25% | 98.20% |
| Platelet | 100.00% | 100.00% | 0.00% | 100.00% |
Class-wise comparison: Baseline vs SVD — Min per-class F1 improved from 96.28% → 97.06%. SVD uniquely improved aggregate accuracy.
5.5 Compression Profile
The Latency Paradox
The final results were incredible. We shrank the physical model size from 532 megabytes down to just 82 megabytes. We had minimal per-class accuracy loss, and our Macro F1 actually increased by 0.2 percent because the SVD acted as a structural denoiser.
But we ran into a paradox. The model size dropped massively, but the latency barely moved. This happens because splitting one large matrix into two smaller ones doubles the GPU kernel-dispatch overhead, causing a bottleneck. To finally conquer the latency problem where the actual heavy computations happen—the convolutional base—we moved to our final method. I'll hand it over to my colleague to talk about L0 Stochastic Gates.