(a) Here, $\ F = 40 = \frac{k(3)(8)}{r^{2}} = \frac{24k}{r^{2}}$
$F^{'} = \frac{k(3 - 5)(8 - 5)}{r^{2}} = \frac{- 6k}{r^{2}}$
$\frac{F^{'}}{F} = {\frac{- 6k}{r^{2}} \times {\frac{r^{2}}{24k} = - \frac{1}{4}}}$
$F^{'} = - \frac{F}{4} = - \frac{40}{4} = - 10N$