[Solution by Dr. Abubacker] The determinant is $\begin{vmatrix}1&1&1\\1&2&5\\1&5&\lambda\end{vmatrix}=\lambda-17$, so for $\lambda=17$ the coefficient rows are dependent: $R_3=-3R_1+4R_2$. Consistency then needs $\mu=-3(6)+4(9)=18$. So there is no solution for $\lambda=17,\ \mu\ne18$. Answer (3).