Open Problems in Integrable Systems — Geometry of caustics, invariant surfaces, and commuting billiard maps

Definitions and mathematical statement

For an ordinary Euclidean billiard in a convex body $\Omega\subset{\mathbb{R}}^d$ with sufficiently regular boundary, a customary fixed-speed collision phase space consists of pairs $(q,v)$ with $q\in\partial\Omega$, $|v|=1$, and $v$ pointing inward. The billiard map follows the straight segment to the next collision and applies specular reflection, \[ v_{\mathrm{after}}=v_{\mathrm{before}}-2\langle v_{\mathrm{before}},n\rangle n, \] where $n$ is the unit normal at that collision. Away from singular/grazing collisions this phase space has dimension $2d-2$. An invariant hypersurface, under a smooth embedded interpretation, is a nonempty codimension-one submanifold $\Sigma$ in the regular phase space satisfying $B(\Sigma)=\Sigma$. The source does not specify the required regularity or nontriviality of such a surface.

Questions attributed to M. Bialy. Are there multidimensional convex billiards other than ellipsoids whose phase spaces contain invariant hypersurfaces? Ask the same existence question for multidimensional outer billiards. For the inner problem, after fixing the intended admissible class and dimension range, the existential target is \[ \exists\,\Omega\text{ not an ellipsoid}\quad \exists\,\Sigma\subset\mathcal P_\Omega\text{ an admissible hypersurface}: B_\Omega(\Sigma)=\Sigma. \] Here an ellipsoid means a set $\{x:(x-c)^{\mathsf T}A(x-c)\leq1\}$ with $A$ positive definite. The comparison excludes such bodies but does not impose any additional geometric hypotheses.

For the outer problem, the analogous target is an invariant hypersurface for the intended outer-billiard map associated to a nonellipsoidal convex body. Higher-dimensional outer billiards require a specified construction, phase space, and sometimes additional geometric structure; none is defined in the excerpt. No planar construction is silently extended to arbitrary dimension.

The excerpt does not determine whether multidimensional means $d\geq2$ or $d\geq3$, whether the question is asked separately in every dimension, or which invariant hypersurfaces count. The fixed-speed collision convention above prevents energy-level hypersurfaces in a larger flow phase space from being inserted as unintended answers, but must itself be checked against the source.

The supplied research summary reports rigidity under strong hypotheses involving rational caustics, commuting/confocal structures, or smooth invariant foliations, and says the unrestricted inner/outer higher-dimensional question was not verified as settled. No precise rigidity statements or citations are supplied, so those reports are not substituted for an answer to the existential questions.

Short English statement

Bialy asks whether nonellipsoidal convex billiards in higher dimensions can have invariant hypersurfaces, and asks the analogous question for outer billiards. The intended dimension range, phase space, surface regularity, and higher-dimensional outer-billiard construction need clarification. Reported rigidity results concern stronger hypotheses.

Sources

• Local UnsolvedMath registry, permanent ID 2100407, source code catalogue entry, "Open Problems in Integrable Systems --- Geometry of caustics, invariant surfaces, and commuting billiard maps"; attribution: M. Bialy. Supplied status partially solved, classification PARTIAL-PROGRESS, updated 2026-08-06.
• Open Problems, Questions, and Challenges in Finite-Dimensional Integrable Systems (2018), Question 4.7, https://arxiv.org/abs/1804.03737. The record reports source-TeX extraction and access on 2026-07-29. It says the source presents an open item and current status requires release review. Rights: NEEDS_REVIEW; public arXiv source TeX inspected according to the record.
• The supplied research summary mentions rigidity under rational-caustic, commuting/confocal, or invariant-foliation hypotheses, but supplies no authors, paper titles, or theorem references for those results.