I'd say polarity and voltage are the biggest things -- about all you can do is double- and triple-check everything; then check again....
Be very careful about where you feed in 12V power (mainly for heaters and fans), vs. 5v (for the Arduino).
In many cases I found it unclear which way +/- went going out from the board (the inputs on my boards were at least marked clearly!). I found that if I plug in a limit switch connector backwards, that connects 5v straight to ground, which will probably make your power supply cut out (assuming it's properly protected -- otherwise it could potentially catch fire). Stepper motors, on the other hand, just run the wrong direction if you plug them onto RAMPS backwards (good design!).
Another thing to watch for only comes up if you're powering the Arduino via USB (I do that a lot, since I'm plugged in anyway to send files and such). A short can try to draw too much power from USB. In that case my Mac shuts down USB and gives a message; I don't know what would happen with other hardware.
I have heard that plugging a stepper-driver board onto RAMPS backwards is likely to fry a board, and that plugging or unplugging a stepper motor while powered is also risky. But I haven't messed those up yet, so I don't now for sure. :)
I may be overly cautious, but I never leave the power supply plugged in when I'm not around.