Nilpotent Orbits Abstract

### Anthony W. Knapp

Abstract of "Nilpotent Orbits and Some Small Unitary Representations of Indefinite Orthogonal Groups"

For 2 \leq m \leq l/2, let G be a simply connected Lie group with Lie algebra g_0 = so(2m,2l-2m), let g = k \oplus p be the complexification of the usual Cartan decomposition, let K be the analytic subgroup with Lie algebra k \cap g_0, and let U(g) be the universal enveloping algebra of g. This work examines the unitarity and K spectrum of representations in the "analytic continuation" of discrete series of G, relating these properties to orbits in the nilpotent radical of a certain parabolic subalgebra of g.

The roots with respect to the usual compact Cartan subalgebra are all \pm e_i \pm e_j with 1 \leq i \lt j \leq l. In the usual positive system of roots, the simple root e_m-e_{m+1} is noncompact and the other simple roots are compact. Let q = l \oplus u be the parabolic subalgebra of g for which e_m-e_{m+1} contributes to u and the other simple roots contribute to l, let L be the analytic subgroup of G with Lie algebra l \cap g_0, let L^ C = Int_g(l), let 2\delta(u) be the sum of the roots contributing to u, and let \bar{q}=l \oplus \bar{u} be the parabolic subalgebra opposite to q.

The members of u \cap p are nilpotent members of g. The group L^C acts on u \cap p with finitely many orbits, and the topological closure of each orbit is an irreducible algebraic variety. If Y is one of these varieties, let R(Y) be the dual coordinate ring of Y; this is a quotient of the algebra of symmetric tensors on u \cap p that carries a fully reducible representation of L^C.

For an integer s, let \lambda_s = \sum_{k=1}^m (-l+s/2)e_k. Then \lambda_s defines a one-dimensional (l,L) module C_{\lambda_s}. Extend this to a (\bar{q},L) module by having \bar{u} act by 0, and define N(\lambda_s+2\delta(u)) = U(g) \otimes_{\bar{q}} C_{\lambda_s+2\delta(u)}. Let N'(\lambda_s+2\delta(u)) be the unique irreducible quotient of N(\lambda_s+2\delta(u)). The representations under study are \pi_s = \Pi_S(N(\lambda_s+2\delta(u))) and \pi'_s=\Pi_S(N'(\lambda_s+2\delta(u))), where S = \dim(u \cap k) and \Pi_S is the S th derived Bernstein functor.

For s \gt 2l-2, it is known that \pi_s = \pi'_s and that \pi'_s is in the discrete series. Enright, Parthsarathy, Wallach, and Wolf showed for m \leq s \leq 2l-2 that \pi_s = \pi'_s and that \pi'_s is still unitary. The present paper shows that \pi'_s is unitary for 0 \leq s \leq m-1 even though \pi_s \neq \pi'_s, and it relates the K spectrum of the representations \pi'_s to the representation of L^C on a suitable R(Y) with Y depending on s. Use of a branching formula of D. E. Littlewood allows one to obtain an explicit multiplicity formula for each K type in \pi'_s. The chief tools involved are an idea of B. Gross and Wallach, a geometric interpretation of Littlewood's theorem, and some estimates of norms.

It is shown further that the natural invariant Hermitian form on \pi'_s does not make \pi'_s unitary for s \lt 0 and that the K spectrum of \pi'_s in these cases is not related in the above way to the representation of L^C on any R(Y).

A final section of the paper treats in similar fashion the simply connected Lie group with Lie algebra g_0=so(2m,2l-2n+1), 2 \leq m \leq l/2.