Actually we don't.
Under the assumption that the same extruder is used, the hot end doesn't matter. The question is unclear about the reuse of the same extruder. If there are different hot ends that have the extruder incorporated into the hot end design, the gearing solution used for these different extruders explain why the E-steps per mm are different. But, if the same (Bowden) extruder is used, you don't need to change the E-steps.
To explain, the E-steps per mm is a property that belongs to the extruder, not the hot end. The E-steps per mm expresses how many steps need to be send to the extruder stepper to extrude a mm of filament. This filament is pushed while it is 1.75 mm or 2.85 mm. For this reason the nozzle diameter doesn't have a part in the equation either; the slicer will calculate the amount of length to extrude a volume is needed, the firmware recalculates this extrusion length (or volume if the filament area is taken into account) and expresses this with the E-steps per mm into an amount of steps for the stepper.
You can calibrate your extruder by disconnecting the hot end or unmounting the nozzle (and telling the firmware to extrude at low temperature using G-code M302; you just need to make sure when 100 mm extrusion is requested, 100 mm is spit out. Any deviations from that can be fixed with the extrusion multiplier.
In the early days, when extruders didn't use dual gear filament gear solutions (which many these days do now), the filament was pushed by a single gear and used a counter bearing that pressed onto the filament with a spring like mechanism. What frequently happened with the cheaper extruder solutions is that friction in the path up to the nozzle caused some slip, so it may look like more steps per mm were needed, but in fact this was just an extruder issue.
Also beware of soft filament and pressure on the filament. See e.g. the following image of 2 different (blue and red) filament types or pressures on the filament sketched in the same image:

This image shows 2 types of filament or pressures on the filament. You see that the softer filament (red) or more pressed on (resulting in the gear teeth digging deeper into the filament), has an effective radius of R2 while the harder (blue) or less pressed on has an effective diameter of R1. R2 is smaller than R1, so for the same amount of steps per mm less filament is extruded (if the gear makes one turn, the extrusion difference is $ E_{diff}= 2\times\pi\times(R_1-R_2)$. Such effects should preferably be adjusted with the extrusion modifier in the slicer.