I had the same problem with my Ender-3 V2.
You need to check if the feeder bracket is square like explained in this YouTube
If that is not the problem maybe you need a custom bracket to change the spacing between the Z-motor and the frame like this, this or this
I have improved mine a lot using:
- This Z-motor mount
- Using glue stick instead of squeezing the first layer
- Leveling the bed using a feeler gauge instead of a piece of paper
- Initial fan speed 100 %, all layers the same
- Same flow rate, line width, line height on all layers
- Build plate 40 °C, nozzle 205 °C
- Seam Corner Preference: Hide or Expose Seam