How can a library contain less information than one of the books in it? The library is intended to be a set of books on shelves. The information content is the least number of bits to describe the data.
The books in the library contains all binary strings of length 1 million, one per book, sorted in lexicographic order. An individual book takes a million bits to specify, but this description of the whole library is much shorter.
This is indeed possible if we define information content as the least number of bits necessary to describe the data. Consider the following example (in human readable language):
- Book 1: Jim knows all animals whose name starts with a letter between A and L.
- Book 2: Jim knows all animals whose name starts with a letter between M and Z.
Then the information in the whole library would be:
- Jim knows all animals.
The paradox is solved if we consider the source of an information as an information itself (i.e. which book contains which information). In this case, the library, if not empty, will always contain at least as much information as any of its books.
The library contains all possible combinations of letters conceivable. (It is a very large library) with no ordering of books at all. Perfect entropy. Each of its book contains a subset only, which therefore contains some order and some information. (The library information is 'complete/all' which can be described with a single bit true)
Sorry if this is not in proper mathematical terms, I am not a mathematician...