When I first started teaching classical electrodynamics, it rapidly became apparent to me that I was spending as much time teaching what amounted to remedial mathematics as I was teaching physics. After all, to even write Maxwell's equations down in either integral or differential form requires multivariate calculus - path integrals, surface integrals, gradients, divergences, curls. These equations are rapidly converted into inhomogeneous partial differential equations and their static and dynamic solutions are expanded in (multipolar) representations, requiring a knowledge of spherical harmonics and various hypergeometric solutions. The solutions are in many cases naturally expressed in terms of complex exponentials, and one requires a certain facility in doing e.g. contour integrals to be able to (for example) understand dispersion or establish representations between various forms of the Green's function. Green's functions themselves and Green's theorem emerge, which in turn requires a student to learn to integrate by parts in vector calculus. This culminates with the development of vector spherical harmonics, Hansen functions, and dyadic tensors in the integral equations that allow one to evaluate multipolar fields directly.
Then one hits theory of special relativity and does it all again, but now expressing everything in terms of tensors and the theory of continuous groups. It turns out that all the electrodynamics we worked so hard on is much, much easier to understand if it is expressed in terms of tensors of various rank2.1.
We discover that it is essential to understand tensors and tensor operations and notation in order to follow the formulation of relativity theory and relativistic electrodynamics in a compact, workable form. This is in part because some of the difficulties we have encountered in describing the electric and magnetic fields separately result from the fact that they are not, in fact, vector fields! They are components of a second rank field strength tensor and hence mix when one changes relativistic frames. Tensors are indeed the natural language of field theories (and much else) in physics, one that is unfortunately not effectively taught where they are taught at all.
The same is true of group theory. Relativity is best and most generally derived by looking for the group of all (coordinate) transformations that preserve a scalar form for certain physical quantities, that leave e.g. equations of motion such as the wave equation form invariant. There are strong connections between groups of transformations that conserve a property, the underlying symmetry of the system that requires that property to be conserved, and the labels and coordinatization of the physical description of the system. By effectively exploiting this symmetry, we can often tremendously simplify our mathematical description of a physical system even as we deduce physical laws associated with the symmetry.
Unfortunately, it is the rare graduate student that already knows complex variables and is skilled at doing contour integrals, is very comfortable with multivariate/vector calculus, is familiar with the relevant partial differential equations and their basic solutions, has any idea what you're talking about when you introduce the notion of tensors and manifolds, has worked through the general theory of the generators of groups of continuous transformations that preserve scalar forms, or have even heard of either geometric algebra or Hansen multipoles. So rare as to be practically non-existent.
I don't blame the students, of course. I didn't know it, either, when I was a student (if it can honestly be said that I know all of this now, for all that I try to teach it). Nevertheless filling in all of the missing pieces, one student at a time, very definitely detracts from the flow of teaching electrodynamics, while if one doesn't bother to fill them in, one might as well not bother trying to teach the course at all.
Over the years in between I've tried many approaches to dealing with the missing math. The most successful one has been to insert little minilectures that focus on the math at appropriate points during the semester, which serve to both prepare the student and to give them a smattering of the basic facts that a good book on mathematical physics would give them, and to also require that the students purchase a decent book on mathematical physics even though the ones available tend to be encyclopediac and say far too much or omit whole crucial topics and thereby say far too little (or even both).
I'm now trying out a new, semi-integrated approach. This part of the book is devoted to a lightning fast, lecture note-level review of mathematical physics. Fast or not, it will endeavor to be quite complete, at least in terms of what is directly required for this course. However, this is very much a work in progress and I welcome feedback on the idea itself as well as mistakes of omission and commission as always. At the end of I list several readily available sources and references that I'm using myself as I write it and that you might use independently both to learn this material more completely and to check that what I've written is in fact correct and comprehensible.