Is there a simple way to fix the common emitter amplifer to not invert the signal and only amplify it?

Yes, there are ways to make non-inverting amplifiers, and we'll see some. (In practice, you would usually use an op-amp to make an amplifier, and we'll see how to make both inverting and non-inverting amplifiers out of them).