For a high-pass filter responding in the low frequency regime, the transfer function is approximately , which is equivalent to a (scaled) derivative function. Looking at it mathematically, the output signal will look like the derivative (the slope) of the input square wave. So for the rising edge, the output will be a sharp positive peak. The output will be zero where the square wave is flat, and it will be a sharp negative peak for the square wave's falling edge.

Looking at it physically: in the low frequency regime, the capacitor charges all the way up early in the cycle, and then passes no more current. When the voltage swings down, it discharges fully and charges up in the opposite direction, then stops passing current again. So you'll get only spikes of current (and corresponding spikes of voltage across the resistor) at the square wave rising and falling edges.