The Mathematics of Communication
5. Exercises
- How many bits are needed to encode 4000 equally likely symbols?
How about 5000?
- Suppose an alphabet has 4 symbols: bear, raven, eagle, whale,
occurring with probabilities p(bear) = 1/2, p(raven) = 1/4,
p(eagle) = p(whale) = 1/8. What is the entropy of this set
of probabilities? Devise an optimal binary code for this alphabet.
Check that this is more efficient than using 00, 01, 10, 11 for the
four symbols.
- Suppose that in the bear, raven, eagle, whale language the
probabilities of digraphs (2-symbol combinations) occur as follows:
|
first\second |
bear |
raven |
eagle |
whale |
|
bear |
7/16 |
1/16 |
0 |
0 |
|
raven |
0 |
1/8 |
1/8 |
0 |
|
eagle |
0 |
1/16 |
0 |
1/16 |
|
whale |
1/16 |
0 |
0 |
1/16 |
Explain why BREW can occur, but BERW cannot (B = bear, etc).
What is the entropy of this set of probabilities in bits/digraph?
Explain why an optimal coding by digraphs can be more efficient
than an optimal coding by symbols. (This is an example of data
compression. For statistics on digraphs in English text, see
Soukoreff and MacKenzie).
@ Copyright 2000, American Mathematical Society.