Why is my Ender 3 restarting spontaneously in mid print?

Question

tl;dr How can I test whether my sporadic in-print restarts are a thermal problem or a firmware issue?

The problem

I've had my Ender 3 (not V2 or Pro) for about a month, and gotten a number of excellent small prints with Amazon Basics and SunLu PLA, as well as Overture PETG, slicing with Cura Slicer 4.9.1; the printer has Marlin 1.0.1 according to the "About Printer" display. I've made no electrical changes to the original hardware, though I've added a few printed parts including a mainboard fan cover, alternative power supply mount, "original screws" display PCB cover, display cable clips, filament guides, storage drawers, and drag chains protecting the cables to the X stepper and extruder/hotend.

However, I've had trouble for most of the time I've had the machine with sporadic reboots in the middle of a multi-hour print. At a minimum, when I resume the print after one of these incidents, the printer will skip the rest of the current layer and start with the next, producing a weak layer (usually it also leaves a significant blob of plastic, and even if I just flatten that blob with the scraper blade as soon as the nozzle is clear, it will stick out of the print and have to be trimmed off later, and if I try to scrape the blob away while it's still soft, it'll result in layer shifts caused by skipping the Y-axis stepper).

Usually, this happens about four to six hours into a 10+ hour print, though I've seen it happen just a half hour or so after a previous occurrence, and near the end of a 65-minute print as well. It's becoming a big issue as I venture into larger prints, as it can ruin the appearance of a print (by skipping a large fraction of a layer that's the visible surface of the part) or causing a weak layer at a stress point -- which wastes a lot of machine time as well as filament.

I've also just upgraded my build surface to a Creality coated glass plate -- which is great in terms of flatness, sticks well, and the part comes off very easily when the plate cools. Unfortunately, if I'm not in the room when the printer restarts, and the bed gets below about 40 °C, the part will unstick and will start moving with the nozzle when I resume the print.

What I've tried

I had suspected issues with my 120 V power, so I purchased a small UPS to condition the power for the Ender 3 power supply, but without seeing improvement. Prints running overnight often do better, but not always, and this happens with both PLA (nozzle temp 200 °C, bed 50 °C) and PETG (nozzle 235 °C, bed 75-78 °C). I haven't attempted other filaments such as ABS.

I'm usually not watching the printer when it does this, but several times I've caught it immediately after (I can hear it running when I'm at my computer, so when it stops I can get to it in seconds), and once I happened to be watching when it stopped. The sequence is that (usually, but not always, when printing a layer with significant infill -- for which I usually use cubic, for best strength with minimal material and time) the printer will suddenly just stop with a normal info screen display for fifteen to thirty seconds (I don't have the exact time, and have only seen it early enough to have an idea a couple of times), then the display goes dark and within two or three seconds returns to the startup splash (Ender 3 dragon logo), from which it changes to the power failure recovery screen with "Resume Print" and "Stop Print" choices.

This is not a reproducible error (nor am I likely to attempt repeating an overnight or whole-day print multiple times to attempt to reproduce the problem). At no time have I noted the power supply cooling exhaust running hot, nor the steppers getting as warm as a 50 °C bed (I don't have an IR thermometer to check the actual stepper temperature). I used a photographic thermometer to check the temperature of cooling air leaving the mainboard housing, and found it never exceeded 30 °C during the print I tested.

My hypothesis

I'm aware of two significant possible causes: overheating the mainboard, or overloading the stepper pulse loop in the firmware. The former should (if things are designed sensibly) result in a complete shutdown, though with a hard switch on the power supply it might not be possible to prevent a restart as soon as the sensor reads below critical (which could be the fifteen to thirty seconds of "freeze" I've observed). The latter is something I've read about, where the Marlin firmware will restart if the stepper pulse interrupt loop gets so busy the mainboard can't service the sensor interrupt loop.

The only way I know to test which I'm seeing is to find a way to more efficiently cool the mainboard and see if that improves the situation -- but I'm reluctant to either remove the fan cover or run the printer with the mainboard cover off; the reason for installing the fan cover was to keep fragments of plastic from getting into the mainboard housing as, being directly under the build plate, it's prone to collect strings, hairs, and scrapings.

Update

I thought I had solved this problem -- I reduced the print speed (using the "tune" menu while the print was running) to 90% (=45 mm/s instead of Cura's default 50 mm/s) and started getting longer prints without stoppages; that would have pointed to firmware loop overload as the likely cause. Unfortunately, I still got a freeze about 10.5 hours in to a 13 hour print, but only one rather than the three or so I'd have expected; this may be a clue. Since then I've had freezes in under an hour into a new print once again, once with the bed running to Ymin (a movement not in the gcode) instead of stopping in place.

Due to the required level of disassembly, I haven't attempted to test continuity while in motion on the stepper or hotend wiring, and I cut away the damaged portion of the Bowden tube when I replaced the hotend and nozzle; I get good extrusion and, when they don't stop in the middle, good smooth prints.

Update 18 June 2021

As suggested in comments, I have initiated a "dry run" of a print that previously generated two or three of these stoppages over the course of about fourteen hours. Original print was Overture clear PETG, nozzle at 235 °C and bed at 75 °C. For this "print" I've retracted the PLA loaded into the machine well clear of the hotend and unplugged the extruder stepper; when I get home from work, I'll be able to see if the machine is still running, and if it has restarted, resuming the print will allow me to tell how long it ran before the incident.

Test Result

This is unexpected. I thought I'd either find the machine still running, or with the power failure resume screen asking to "Resume Print" or "Stop Print" -- instead, I came home from work, some 10 hours after starting the dry run test, to find the printer idle, as if I'd had it properly shut off and then turned the power on. No way to tell when it stopped or why.

Further Update, June 20

I had another of the new type of failure on an overnight print; I was printing PETG (Sun Lu brand), 240 °C nozzle and 75 °C bed, back on my magnetic textured build sheet because the PETG was warping and coming up from the coated glass -- and when I got up this morning the printer should have still been running, at around 90% complete, but instead it was sitting at the Info Screen, nozzle and bed both reading room ambient temperature with set point 0 °C.

I can only presume that either the "resume print" option has a limited life (though I've seen it at what must have been eight or nine hours after the print stopped and print progress in this case suggests it had only stopped four hours or so before I came back), or I'm getting a second interrupt from the resume screen that's causing the printer to return to "fresh power on" state.

I'm reluctant to pursue further warranty help from the Amazon seller because of the hassle of returning a machine I've customized and my feeling that this is an easy fix if I can just determine what to fix.

Followup, June 21

I'm starting to believe my machine is haunted. Over the past twenty-four hours, I've had about twenty-one continuous hours of printing, completely successful (in terms of the machine continuing to run; I've got some corner lifting on the print that was running when I left for work this morning, but that's print settings and/or material vs. build surface). I was more than ten hours into a twelve-plus hour print when I left this morning, and completely a nine and a half hour print yesterday without incident.

In other words, it almost seems as if the problem has resolved.

My partner half jokingly suggested the cats might be somehow resetting the machine, but I don't see any way they could. Just poking at the Ender 3 control knob won't do anything, and even if they managed to turn the knob and poke it again, if they initiated a "stop print" the machine would retract, lift the nozzle, and home the X and Y. If they batted at the power switch (which is after the UPS) and got it to momentarily break power, I'd expect to find the "resume print" screen -- and further, this has happened when I'm in the same room and the sequence isn't compatible with a power break.

Additional Followup 22 June 2021

I thought the machine was back to trying to make a liar out of me, after getting around 27 hours of failure-free prints -- and then it stopped at about 60% of a 2 1/2 hour print in PLA, glass bed, at 200 °C nozzle and 50 °C bed, 10% gryroid infill. It was printing fine when I went to bed, and when I got up I found it at the info screen (not the resume print choice screen), with the nozzle stuck to the infill.

Additional information 26 June 2021

I was able to take a few minutes to open the main board housing while installing belt tensioners today -- I was slightly startled to discover that despite shipping with Marlin 1.0.1 (correction: it's 2.0.1-V1.0.1, 32-bit build and the newest available from Creality for this mainboard), the mainboard is the 32-bit version 4.2.2. While this is not the most current version, it is the board that was introduced with the Ender 3 V2 in 2020 and almost immediately added to the Ender 3 Pro, and can be updated through the MicroSD or via the USB port.

I'm told (via discussion on the Ender 3 group at Thingiverse) that this rules out firmware overload, as the 32-bit boards are both much faster computationally, and have significantly larger on-board RAM and EEPROM.

I also went over the electrical connections on the mainboard; I found two of the screw terminals weren't as tight as I'd make them (this is probably a result of using tinned wire instead of crimped ferrules); I also was able to push down one of the stepper driver outputs (not sure which one), which wasn't fully seated (though surely making contact, since I've had all steppers operational any time the machine wasn't resetting itself).

Unfortunately, tightening the slightly loose screw connections helped nothing; though I was able to complete a 1 1/2 hour print flawlessly, I had two consecutive failures several hours into a 7 1/2 hour print.

My next thing to check is internals in the power supply.

June 29 update

Power supply check showed perfect 24.0 V output, but I found the two spade terminals on the power switch itself were loose enough to move around with a little tug on the wire. They were making good contact, as far as I could tell, and required effort to pull off the spade on the switch, but once things heat up after some time under load (running X and Y steppers and bed heater at the same time, for instance, along with near-constant nozzle heat) they might well have gotten looser -- and since I moved the power supply off the vertical extrusion to a pair of printed brackets to improve clearance between the magnetic build sheet and power supply, it does move around a bit when the machine is printing (especially with the rapid direction changes of infill).

On that basis, I disconnected both wires and squeezed the terminals with pliers to close them up a little; they were very tight when I reconnected them.

Too early to be sure, but the machine finished a 7.5 hour print flawlessly after this.

Update June 30

It was better -- I got two 7.5 hour prints in a row without a failure. Printed a spacer to allow quick switch between magnetic build surface sheet and glass (clips onto X gantry plate to trigger the Z-stop with the nozzle higher by exactly the extra thickness of the glass plate). Too thick; easier to reprint than to sand or scrape enough.

Machine stopped at the same spot while printing the skirt three attempts in a row. This is less than one minute into the print. Further, doesn't reboot, just freezes until I power it off, then comes back up with "resume" dialog. Tried to reslice, thinking it might be a bad g-code file, and the microSD won't mount on my desktop computer, though it still seems to read okay in the printer and another (full size) SD card mounts on the same USB port and hub (and the problem card doesn't mount even in the microSD slot in the other reader).

This last doesn't appear to be related to the original problem: the microSD had seemingly gotten disconnected without "safely remove" procedure, and Ubuntu apparently isn't capable of recovering from this; that resulted in a corrupted gcode file causing these most recent, very early stoppages, as well as the inability to mount the storage device. Reformatting the storage device corrected that problem.

Question?

What other tests can I run to differentiate between a thermal restart and a firmware overload?

Answer 1

This problem (of spontaneous restarting during printing) appears to be solved.

The Cause

The apparent cause was a combination of push-on spade connectors on the power switch that were looser than ideal, likely specified that way in order to facilitate mass production (the tighter such a connector is, the more difficult it is to assemble, leading to slower work and an increase in damaged parts that then require reworking an assembly), and moving the power supply off its direct mount to the machine frame (in order to increase clearance between the PSU and the heat bed after installation of a magnetic build sheet).

The brackets I used to remount the power supply are free to slide in the 4040 extrusion of the frame base, and have no mechanical connection to the PSU housing or even its plastic extension (which houses the wire connections, power switch, and power cord connector). This allows the PSU to move much more under frame accelerations (Newtonian reaction to movements of the bed and hot end) than with the original mount -- and this movement sometimes pulls against the wires inside the plastic wiring cover.

Combine that with spade connectors that are loose enough to move when the wire is tugged pretty gently, and you have a formula for momentary disconnects. Such a disconnect will reset the main board (like a power flicker can do to a computer with no UPS), resulting in the freeze (actually mainboard startup sequence before it begins to update the display), then a startup.

The Complication

I can't prove it, but I hypothesize that the transition from restarting to the "resume print" screen, to restarting as if freshly powered up after a normal shutdown occurred because the connection issue had progressed to the point of producing two or more disconnects close enough together that the second restart lost the "print in progress" flag.

The Solution

What appears to have solved the problem was to remove the wiring cover on the PSU (ensure power is off and power cord disconnected, mains voltage is present inside this cover), disconnecting the spade terminals at the power switch, and (with pliers) carefully squeezing the female receptacles so they are tighter on the male blade. This must be done carefully, as squeezing too hard will close up the female terminal and prevent reconnection, requiring either much more difficult adjustment with knife and miniature screwdriver, or replacement of the terminal.

Since applying this fix, I've completed a few longish prints (about 7 1/2 hours each) and a number of shorter ones without a single glitch, aside from an unrelated problem that caused a corrupted gcode file.

Adding to this, I did get one in-print reset after this answer; it happened while printing gyroid infill at (IIRC) 40%, which is some of largest ongoing acceleration (=> machine frame movement) of anything you'd do.

My final solution has been to add a third bracket to the power supply, so three corners are supported. Now the power supply doesn't wiggle and wobble when the machine shifts a little; it's back to a near-rigid mount to the frame (if anything, a little sturdier than the original two screws through an upright). I haven't tried gyroid infill again, but I've had 100% reliability (including a couple large prints at 7+ hours) since installing this third bracket.