Information Theory and Coding 01

Chapter One Introduction

1. Basic requirements

• Master the basic concepts of news, signals and information
• Clarify the research object, research purpose and research content of information theory
• Master the communication system model

1. What is information? What are news and signals?

Definition of Shannon information: Information refers to the way in which various things move and change their state.
The basic concept of the information system is its uncertainty. Anything that has been determined contains no information.
Message: refers to the language, text, images, etc. that contain information.
Signal: The physical system of the message.

2. Shannon is the founder of information theory
3. Communication System Model
   Source ——> Channel ——> Sink
4. Three performance indicators of communication system

Effectiveness
Reliability
Security

5. Three types of coding

Source coding,
channel coding,
encryption coding

6. Homework exercises

The basic concept of information lies in its uncertainty .
The purpose of information theory research is to find the common law of information transmission, to improve the reliability, effectiveness, confidentiality and authentication of information transmission, and to optimize the information transmission system.
The performance indexes of the communication system are mainly economical, effective, reliable and safe.
Source coding-effectiveness
Channel coding-reliability
Encryption coding-security

Chapter 2 Sources and Source Entropy 1

1. According to whether the parameter set and the value range are discrete or continuous intervals, the Venus show is classified:

1. Discrete information source in time discrete space: T discrete, Ex discrete. (Discrete sources) such as text, numbers, data, letters, etc.
2. Continuous source of time discrete space: T discrete, Ex continuous. (Continuous sources) such as language, images, video, etc.
3. Discrete sources of time continuous space: T continuous, Exl discrete.
4. Time continuous space continuous source: T continuous, Ex continuous. (Waveform source)

2. Classification according to the memory characteristics of the sequence

1. There are sources of memory: related
2. No memory source: independent of each other

3. Typical stationary signal: white noise. Typical non-stationary signal: radar signal.
4. Random processes can be divided into stationary random processes and non-stationary random processes. The most common stationary random processes are traversal processes.
5. In theory, the frequency spectrum of any time-limited function is infinite. On the contrary, time is unlimited.
6. Markov Source

When the memory length of the source is m + 1, the memory source is called m-order Markov source. That is, the symbols sent by the source each time are related to the first m symbols, and not related to the previous symbols.

7. Self-information and uncertainty

Self-information: the amount of information provided to the recipient after the symbol appears.
Uncertainty: The uncertainty contained before the symbol appears.