Research
Separable Shape Tensors
Separable Shape Tensors (SST) decompose large ensembles of segmented curves into complementary features of generalized scale and nonlinear undulation, enabling explainable binary classification of images without labeled training data.
Given a centered landmark matrix \(X \in \mathbb{R}^{n \times d}\) representing a discrete curve, the thin SVD \(X^\top = U\Sigma V^\top\) motivates a polar standardization separating undulation from scale,
\[ X = \tilde{X} P, \quad \tilde{X} := VU^\top, \quad P := U\Sigma U^\top \in \mathcal{S}^d_{++}, \]
where \([\tilde{X}] \in \mathrm{Gr}(d,n)\) encodes nonlinear undulation over the Grassmannian and \(P\) encodes generalized anisotropic scale over \(\mathrm{GL}^+(d,\mathbb{R})/\mathrm{SO}(d)\), independent of rotations and reflections.
The formalism is grounded in a composite linear operator (CLO),
\[ T[c](s) = c(s) - \int c(u)d\mu(u)\]
and\[k_{T}[c](s,u) = T[c](s) \cdot T[c](u),\]
which is invariant to permutations, rotations, reflections, and translations (PRRTI). Eigensolutions satisfy the Fredholm integral equation,
\[ \int k_{T}[c](s,u)\, v_j(u)\, d\mu(u) = \sigma_j^2\, v_j(s), \]
\(j=1,\dots,d\). A central result (Thm. 1) of this simple curve-PCA establishes that the eigenfunctions are orthonormal dual evaluation functionals of a curve in a vector-valued RKHS with flexible kernel-controlled regularity. After discretization, explainable binary classification (EBC) is performed via a product maximum mean discrepancy (MMD) over the separable parameter distributions, yielding provably logical statistical decisions in seconds--even with ensembles containing thousands of curves.
Dimension Reduction
Engineering models often depend on a large number of parameters \(x \in X \subseteq \mathbb{R}^m\), yet quantities of interest frequently only vary along a handful of directions. This observation motivates a manifold hypothesis: there exists a smooth surjection \(\varphi^{-1}: X \to M \subseteq \mathbb{R}^r\) with \(r \ll m\) and a function \(h: \mathbb{R}^r \to \mathbb{R}\) such that
\[ \| f(x) - h(\varphi^{-1}(x)) \|_{L^2_\rho(X)} \leq \epsilon \]
for small \(\epsilon \geq 0\). When the reduction is linear, we have \(\varphi^{-1} = U^\top\) for some orthonormal \(U \in \mathbb{R}^{m \times r}\), and important directions are revealed by the eigenspaces of the average outer product of the gradient,
\[ C = \int_X \nabla f(x)\, \nabla f(x)^\top \rho(x)\, dx = W \Lambda W^\top. \]
The dominant eigenvectors span an active subspace: a reduced-dimensional subspace along which \(f\) changes the most, on average. Orthogonal directions constitute an inactive subspace where the function is nearly constant. This framework was succesfully applied to the calibration of aero-engine thermal models and to shape sensitivity of transonic airfoils parameterized by PARSEC and class-shape transformations. However, in the latter, the resulting active subspace depends on the chosen shape parameterization which raises a natural question: can we define a dimension reduction that is intrinsic to the shapes themselves?
A space of shapes
Rather than selecting an engineering parameterization, we represent shapes directly as centered landmark matrices \(X_n \in \mathbb{R}^{n \times d}_*\) (full rank, not lines or points). An affine standardization via the thin SVD maps each translated shape to a representative element of the Stiefel manifold \(\mathrm{St}(d,n)\), modding scale variations as invertible linear deformations. The equivalence classes live on the Grassmannian \(\mathrm{Gr}(d,n)\), and a local section of the principal \(\mathrm{GL}_d\) fiber bundle recovers physical scales. For airfoils (\(d=2\)), a database of ~1500 shapes was standardized with \(n=1000\) landmarks each, and tangent Principal Components Analysis (PCA) at an approximated Karcher mean provided a data-driven basis for the tangent space of this learned manifold.
Active manifold-geodesics
Extending the active subspace to a Riemannian manifold \((M,g)\) requires replacing the component-wise Euclidean integral with one that respects the geometry. The key insight is that the Euclidean gradient outer product implicitly uses the identity as parallel transport. On a general manifold, we must explicitly transport gradients back to a central tangent space \(T_{p_0}M\) along unique geodesics given the Levi-Civita connection. This yields the Riemannian average outer product of the gradient,
\[ G_0 = \int_X P^{-1}_{t_0,t}\!\left[\nabla_M \hat{f}(x)\right] \otimes P^{-1}_{t_0,t}\!\left[\nabla_M \hat{f}(x)\right] \hat{\rho}(x)\, dx \]
where \(P^{-1}_{t_0,t}\) denotes parallel transport over the unique geodesic from \(p\) to \(p_0\), and normal coordinates identify tangent vectors with elements of \(\mathbb{R}^m\). By construction, the integrand is unique--naturality of the exponential map and the Riemannian connection guarantees that \(G_0\) commutes with any isometric embedding. In other words, one can compute intrinsic quantities using extrinsic matrix data.
The eigendecomposition \(G_0 = W_0 \Lambda_0 W_0^\top\) then defines active manifold-geodesics (AMG): the dominant eigenvectors \(w_1, \dots, w_r\) generate geodesic submanifolds
\[ \mathcal{A}_r = \left\{ \exp_{p_0}(\mathcal{V} \cap A) : A = \mathrm{Range}[W_r] \subset T_{p_0}M \right\} \]
over a normal neighborhood \(\mathcal{V}\). These are the nonlinear analogs of active subspaces--curves and surfaces in the manifold along which a quantity of interest changes the most, by a globalizing notion of the average that reduces exactly to the classical definition when \(M = \mathbb{R}^m\).
Shape sensitivity of transonic airfoils
The framework was applied to adjoint-based sensitivity analysis over a data-driven Grassmannian manifold of airfoils. The exponential and inverse exponential maps admit closed-form algebraic expressions over \(\mathrm{Gr}(2,n)\), making the approximation of \(G_0\) tractable even for \(n=1000\) landmarks. The resulting AMG eigenspaces produced the first global shape sensitivity analysis independent of any chosen parametrization--visualized as a scalar-valued activity score over the airfoil surface, revealing where shape perturbations have the most influence on aerodynamic forces.
Bi-Criteria Spectrum Sharing
When two networks share a band, there is no single objective to maximize: each has its own throughput, and improving one generally costs the other. Writing the two criteria as \(f_1, f_2 : X \subseteq \mathbb{R}^m \to \mathbb{R}\) over a shared parameter vector \(x\), the trade-off is captured by the convex scalarization
\[ g(x;\lambda) = \lambda f_1(x) + (1-\lambda) f_2(x), \qquad \lambda \in [0,1], \]
whose maximizers \(x^*(\lambda)\) sweep the Pareto set as \(\lambda\) varies. Solving this independently on a grid of weights is wasteful and produces a disconnected scatter of points rather than a curve.
Tracing rather than re-solving
The key move is to treat \(x^*\) as a curve in \(\lambda\) and differentiate its optimality condition. Stationarity \(\nabla_x g(x^*(\lambda);\lambda) = 0\) holds identically in \(\lambda\), so differentiating through gives a predictor for how the optimum moves,
\[ H(\lambda)\, \frac{dx^*}{d\lambda} + \big(\nabla f_1 - \nabla f_2\big) = 0, \qquad H(\lambda) = \lambda \nabla^2 f_1 + (1-\lambda)\nabla^2 f_2, \]
so that
\[ \frac{dx^*}{d\lambda} = -H(\lambda)^{-1}\big(\nabla f_1(x^*) - \nabla f_2(x^*)\big). \]
Numerically integrating this ODE traces the Pareto frontier as a continuous object, at the cost of gradients and a Hessian solve per step instead of a fresh optimization per weight.
Where the subspace enters
The tracing step needs \(H(\lambda)^{-1}\), which is exactly where a large \(m\) hurts. Each criterion carries its own active subspace, obtained from the eigendecomposition of its average gradient outer product,
\[ C_i = \int_X \nabla f_i(x)\, \nabla f_i(x)^\top \rho(x)\, dx = W_i \Lambda_i W_i^\top, \quad i = 1,2. \]
The two criteria do not agree on which directions matter, so neither subspace will serve on its own — a direction inactive for one may well be active for the other. Rather than stacking the two bases, the aggregate is found geometrically. The dominant \(r\)-dimensional eigenspaces are points \([W_1],[W_2] \in \mathrm{Gr}(r,m)\), and the geodesic between them
\[ [U(t)] = \exp_{[W_1]}\!\big(t \log_{[W_1]}[W_2]\big), \qquad t \in [0,1], \]
sweeps a one-parameter family of candidate subspaces running from one criterion's answer to the other's. A scalar sub-optimization then selects the point along that geodesic by a maximin criterion on the coefficient of determination \(R^2_i(t)\) of each criterion's ridge approximation,
\[ t^\star = \underset{t \in [0,1]}{\arg\max}\ \min\big\{\, R^2_1(t),\, R^2_2(t) \,\big\}, \qquad U := U(t^\star), \]
which lifts whichever criterion the shared subspace explains least well. The aggregate is therefore never allowed to serve one criterion at the other's expense: the guarantee is on the weaker of the two fits, not on their average.
Tracing then runs in the reduced coordinates \(y = U^\top x\), so the Hessian solve is \(r \times r\) rather than \(m \times m\). Choosing \(r = 2\) additionally buys explainability: the entire trade-off becomes something you can draw and inspect, rather than a black-box optimizer output. Note this is the same Grassmannian machinery as the shape work above — here the geodesic interpolates between two notions of which directions matter, rather than between two shapes.
Grey, Mosleh, Rezac, Ma, Coder, & Dienstfrey (2022) — IEEE Transactions on Communications