Any ideas what to do to prevent this from happening?
You cannot prevent it entirely, but you can probably mitigate the problem by depressing the lever that squashes the filament against the hobbed gear of the extruder before starting to heat the nozzle.
In bowden extruders, the long portion of filament between the stepper motor and the nozzle is subject to compression during the print. Because of the hysteresis in the filament, and of the slack between filament and PTFE tube, this filament acts like a slow-releasing compression spring. When the nozzle cools down, the potential energy stored in the filament is "frozen" in place.
By depressing the lever, you allow the spring to extend "backward" towards the spool, rather than "forward" through the nozzle.
Some oozing is still bound to happen because of gravity and - as highlighted by others - thermal expansion, but it should be significantly less.
If you adopt the lever trick, remember to print with a skirt, as you will want the printer to recreate that "compression" in the filament before the model proper begins.
Another way to address the issue would be to add a little bit of retraction in the closing stanza of your GCODE (the part where you also tell the printer to unpower the steppers and stop heating). This will prevent any "compression" to be "frozen" in the first place.
This anwer is based on the assumption that the stepper motor is not actively spinning (i.e.: yours is not a hardware/firmware issue).