Explanation

(b)
$A^{2} = \begin{bmatrix}
- 5 & - 8 & 0 \\
3 & 5 & 0 \\
1 & 2 & - 1
\end{bmatrix}\begin{bmatrix}
- 5 & - 8 & 0 \\
3 & 5 & 0 \\
1 & 2 & - 1
\end{bmatrix}$
$= \begin{bmatrix}
25 - 24 + 0 & - 40 - 40 + 0 & 0 + 0 + 0 \\
- 15 + 15 + 0 & - 24 + 25 + 0 & 0 + 0 + 0 \\
- 5 + 6 + 0 & - 8 + 10 - 2 & 0 + 0 + 1
\end{bmatrix}$
$= \begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix} = I$
Hence, the matrix $A$ is involutory.