COMSATS Institute of Information Technology (CIIT), Abbottabad Department of Electrical Engineering

EEE-354

Telecommunication Systems Engineering Problem Set 1 Source Coding and Data Compression

Spring-2014

Question No. 1:
A communication system compresses the data using Huffman coding. The Huffman codes are as shown in Table 1. The resulting bit-stream is sent over the transmission medium. The receiver receives the following bit-stream: 1100011110101011111001100100 1. 2. 3. 4. 5. Draw a Huffman tree on Figure 1, showing the construction of code words given in Table 1. (Tip: Use only the required branches in Figure 1) What message will the receiver recover? (Show your working) Symbol Code What is the average length (bits/symbol) of the Huffman code used? E 0 What is the entropy of the message? D 100 Justify the use of Huffman code over a fixed length code. R 110 N 111 0 1 G 1010 I 1011

Figure 1

Question No. 2:
For the message REENGINEERING , 1. Use Shannon-Fano coding to compress the message 2. What is the average length (bits/symbol) of the code used? 3. What is the entropy of the message? 4. Compare the value of average length of this code with that resulting from compression of the message by a Huffman code. 5. Can the average length of a Huffman or Shannon-Fano code be less than the value of entropy for a given input stream? Discuss reasons. 6. Can the average length of a Huffman or Shannon-Fano code be equal to the value of entropy for a given input stream? If no, give reasons, if yes, give conditions when they will be equal.

Question No. 3:
1. 2. 3. Find the entropy of a source that produces 8 symbols with probabilities of {0.25, 0.25, 0.125, 0.125, 0.0625, 0.0625, 0.0625, 0.0625} Find the number of bits required to represent each symbol of a source producing the numbers 0 to 99 for a fixed length code. Given that the entropy of a source is 3.0, how many equally likely symbols can it produce?

Question No. 4:
1. 2. 3. 4. 5. Use LZW coding to compress the input text IDIDITDIDINOTDOIT# given that the initial dictionary entries are as shown in Table 2 Also list down all the dictionary entries that will be created Calculate the entropy of the message. Symbol Dictionary Entry What will be the compression ratio achieved? # 000 Decode the resulting bit-stream to see if you are able to D 001 recover the original message! I 010 N 011 O 100 T 101