I'll start with a few simple ones to get us rolling.
Base64 has a 2/3 chance of requiring one or two trailing
= signs - if you spot any of these, there's a very good chance you're dealing with Base64 encoded text.
A Bacon cipher is composed of 2 binary bits, which can be represented by anything (eg. could be upper/lowercase as in one recent question, or could be based on whether a cat is white/black, as seen in another). The tell-tale sign is if the ciphertext length is divisible by 5, as each letter requires 5 binary bits of Bacon. Yum.
Frequency analysis is most useful for Caesar ciphers (also known as Caesarian shifts) or similar ciphers where a letter is always encoded as itself (ie. if an
E encodes to a
K anywhere in the text, it will encode to a
K everywhere in the text). Many websites have frequency tables for how often you should expect a letter to appear, and if you have a reasonably large block of ciphertext, this may be a good way to get some initial letters.
A substitution cipher is a very simple numerical cipher where letters are replaced with their numeric position in the alphabet (eg.
A is 1,
B is 2, etc) - this may also be combined with a Caesarian shift. If you have a sequence of numbers where none are over 26, try a quick substitution and see if you get anywhere. Numbers above 26 may still be substitutable using modulo, but it's less common.
Playfair ciphers are composed of digrams (pairs of letters) so the pair of letters
HI may encode to something like
PK. If you have an even-length ciphertext, split it into digrams and look for duplicates. If you spot the same digram appearing multiple times, you should begin to suspect a Playfair cipher (or other digram-based cipher).
Polybius ciphers (or Polybius squares) are composed of the same 5 characters over and over again (for example,
E) and also happen to be digram-based so it must be an even-length ciphertext. If you have only 5 repeated characters and can split them into digrams, it's highly likely you're dealing with Polybius.
One more common-sense point - if you have a keyword then you can usually safely rule out any cipher which doesn't require a keyword. For example, Base64 is just an algorithm which you put text in and it gets spat out the other end, while a Playfair cipher uses a keyword to make the cipher stronger.