Journal of Lie Theory

Let $W_d=K^d$ be the $d$-dimensional vector space over a field $K$ of characteristic 0 with the canonical action of the general linear group $GL_d(K)$ and let $KX_d$ be the vector space of the linear functions on $W_d$. One of the main topics of classical invariant theory is the study of the algebra of invariants $K[X_d]^{SL_2(K)}$ of the special linear group $SL_2(K)$, when $KX_d$ is a direct sum of $SL_2(K)$-modules of binary forms. Noncommutative invariant theory deals with the algebra of invariants $F_d({\mathfrak V})^G$ of a group $G(K)$ acting on the relatively free algebra $F_d({\mathfrak V})$ of a variety of $K$-algebras $\mathfrak V$. Due to the noncommutativity it is more convenient to assume that $F_d({\mathfrak V})$ is generated by $W_d$ instead of by $KX_d$, with the corresponding action of $GL_d(K)$.\par In this paper we consider the free metabelian Lie algebra $F_d({\mathfrak A}^2)$ which is the relatively free algebra in the variety ${\mathfrak A}^2$ of metabelian (solvable of class 2) Lie algebras. We study the algebra $F_d({\mathfrak A}^2)^{SL_2(K)}$ and describe the cases when it is finitely generated. This happens if and only if as an $SL_2(K)$-module $W_d\cong K^2\oplus K \oplus\cdots\oplus K$ or $W_d\cong S^2(K^2)$ (and in the trivial case $K W_d\cong K\oplus\cdots\oplus K$). Here $SL_2(K)$ acts canonically on $K^2$, trivially on $K$, and $S^2(K^2)$ is the symmetric square of $K^2$. For small $d$ we give a list of generators even when $F_d({\mathfrak A}^2)^{SL_2(K)}$ is not finitely generated. The methods for establishing that the algebra $F_d({\mathfrak A}^2)^{SL_2(K)}$ is not finitely generated work also for other relatively free algebras $F_d({\mathfrak V})$ and for other groups $G$.