Is there possible asymmetry in the error matrix, $\sigma^2_{uv} \ne \sigma^2_{vu}$?

No, by construction, the error matrix is symmetric. The off-diagonal elements are the covariances, and the covariances are defined as $\sigma^2_{uv} = \langle (u-\bar{u})(v-\bar{v})\rangle$. The covariance definition is symmetric between $u$ and $v$.