(a)
Given, $y = x^{2}e^{mx} \Rightarrow \frac{dy}{dx} = 2xe^{mx} + mx^{2}e^{mx}$
$\Rightarrow \ \frac{d^{2}y}{dx^{2}} = 2\left( e^{mx} + mxe^{mx} \right) + m(2xe^{mx} - x^{2}me^{mx})\$
$\Rightarrow \frac{d^{2}y}{dx^{2}} = e^{mx}\left( m^{2}x^{2} + 4mx + 2 \right)$
$\Rightarrow \frac{d^{3}y}{dx^{3}} = e^{mx}\lbrack m^{3}x^{2} + 4m^{2}x + 2m + 2m^{2}x + 4m\rbrack$
$= e^{mx}\lbrack m^{3}x^{2} + 6m^{2}x + 6m\rbrack$