Show that $\cos 3\theta = 4\cos^3 \theta - 3\cos \theta$. Hence, or otherwise, evaluate $\displaystyle \int_0^{\pi/6} \sin 2\theta \cos 3\theta\, d\theta$ exactly.

1. Split $3\theta$ into $(2\theta + \theta)$ and apply the cosine compound angle formula.

$\cos 3\theta = \cos(2\theta + \theta)$
$= \cos 2\theta \cos \theta - \sin 2\theta \sin \theta$

2. Apply the double angle formulas to express the equation in terms of single angles.

$= (2\cos^2 \theta - 1)\cos \theta - (2\sin \theta \cos \theta)\sin \theta$
$= 2\cos^3 \theta - \cos \theta - 2\sin^2 \theta \cos \theta$

3. Use the Pythagorean identity to convert all sine terms to cosine and simplify to the final form.

$= 2\cos^3 \theta - \cos \theta - 2(1 - \cos^2 \theta)\cos \theta$
$= 2\cos^3 \theta - \cos \theta - 2\cos \theta + 2\cos^3 \theta$
$= 4\cos^3 \theta - 3\cos \theta$

1. Substitute the identity from part (a) and the double angle formula for $\sin 2\theta$ into the integral, then expand.

$\int_0^{\pi/6} \sin 2\theta \cos 3\theta \, d\theta = \int_0^{\pi/6} (2\sin \theta \cos \theta)(4\cos^3 \theta - 3\cos \theta) \, d\theta$
$= \int_0^{\pi/6} (8\sin \theta \cos^4 \theta - 6\sin \theta \cos^2 \theta) \, d\theta$

2. Integrate the expression using the reverse chain rule for powers.

Using $\int [\mathrm{f}(x)]^n \mathrm{f}'(x) \, dx = \frac{[\mathrm{f}(x)]^{n+1}}{n+1} + c$:
$= \left[ -\frac{8}{5}\cos^5 \theta + 2\cos^3 \theta \right]_0^{\pi/6}$

3. Substitute the upper limit $\frac{\pi}{6}$ into the antiderivative.

$\text{At } \theta = \frac{\pi}{6}: \quad -\frac{8}{5}\left(\frac{\sqrt{3}}{2}\right)^5 + 2\left(\frac{\sqrt{3}}{2}\right)^3$
$= -\frac{8}{5}\left(\frac{9\sqrt{3}}{32}\right) + 2\left(\frac{3\sqrt{3}}{8}\right)$
$= -\frac{9\sqrt{3}}{20} + \frac{15\sqrt{3}}{20} = \frac{6\sqrt{3}}{20} = \frac{3\sqrt{3}}{10}$

4. Substitute the lower limit $0$ and subtract to find the exact value.

$\text{At } \theta = 0: \quad -\frac{8}{5}(1)^5 + 2(1)^3 = -\frac{8}{5} + 2 = \frac{2}{5}$
$\text{Value} = \frac{3\sqrt{3}}{10} - \frac{2}{5} = \frac{3\sqrt{3} - 4}{10}$