To improve server security, I usually do

  1. Keep system healthy
    At least the server should not have obvious vulnerability. Strengthen password, update patches regularly, setup firewall, and minimize open services.
    Attacks may be carried through legitimate services. For example, if your smtp service is an open relay, it is inevitably used by spammers.
    Some simple tools help a lot sometimes. Such as denyhosts, it may effectively thwart password cracks.
  2. Detect intrusion
    When intrusion happens, the intruder may need to plant bots to exploit other computers, and open a backdoor for his revisit. In this case, the sysadmin should be able to detect these activities by checking system files integrity and network traffic pattern.

This section is about how to achieve above goals.