How do you determine stable vs. unstable equilibrium if $\left(\frac{d^2U}{dx^2}\right)_0 = 0$?

In this case you will need to look at higher-order terms in the Taylor expansion of $U(x)$.