(a)
We know that,
$\left\{ \frac{n}{2}(n + 1)^{2} \right\} = (1 + 2 + ... + n)^{2}$
$= \sum_{i = 1}^{n}{x_{i}^{2} + 2\sum_{i < j}^{}{x_{1}\ x_{j}}}$
$\Rightarrow \sum_{i < j}^{}{x_{i}x_{j} = \frac{1}{2}\left\{ \frac{n^{2}(n + 1)^{2}}{4} - \frac{n(n + 1)(2n + 1)}{6} \right\}}$
$= \frac{n}{24}(n - 1)(n + 1)(3n + 2)$