The solution is the following:
We define the notion of a class of infinite sequence of hats. All sequences in this class differ from each other by only a finite number of elements. There are an infinite number of classes, each containing a infinite number of elements.
Why? As differences between classes are of infinite length, there is an infinite number of possible distances, which means there is an infinite number of classes.
For each such class we select a "representative" which is a single element of the class (a single infinite sequence of hats). So suppose we have the class "infinitely alternating red and blue, with a finite number of non-alternating hats in the first 20 elements". We can set the representative to be:
EDIT: Apparently, after digging a bit, a good way to define the classes is:
Two sequences are equivalent if they are identical after a finite number of items
Choice of a representative can be easily defined as choosing the sequence where the differences appear at the beginning of the sequence and are lexicographically smaller.
For example, the sequence
rbrbrbrbrbrb...
And, the sequence:
r[r]rb[br]rbrbrb... ([] denotes mismatches)
is a member of that class. Each dwarf can see the infinite sequence of hats in front of it and can recognize the class. The dwarves can see the mismatches in front of them, but do not know if their own hat is not a mismatch.
But, the difference is finite, and a modification of the standard rules for guessing a finite sequence can be applied. The first dwarf says "red" if there is an odd number of hats that are different in the "sequence", compared to the "representative" (or "blue" if the mismatches are even). From there on, each dwarf has sufficient info to tell the colour of their hat.
Note that, the number of classes is infinite, however, since we are talking about infinite dwarves, I assume this is acceptable.
Finally, since the first dwarf "guesses", the first dwarf has a 50% chance of survival, all other dwarves survive.
Oh, and because of the infinite number of classes, this can be directly applied to the problem with an infinite number of colours.
Summary for the naysayers
Two sequences are equivalent if they are identical after a finite number of items.
Choice of a representative can be easily defined as choosing the sequence where the differences appear at the beginning of the sequence and are lexicographically smaller.
As the sequence is infinite, and differences are finite, there exists an infinite subsequence, that can uniquely define the class of the sequence.