Finite blocklength information theory

Information theory
Entropy Differential entropy Conditional entropy Joint entropy Mutual information Conditional mutual information Relative entropy Entropy rate Limiting density of discrete points
Asymptotic equipartition property Rate–distortion theory
Shannon's source coding theorem Channel capacity Noisy-channel coding theorem Shannon–Hartley theorem
v t e

Finite block-length information theory is a branch of information theory that analyzes maximum channel coding rate under the finite length frame.^[1][2][3] The Shannon–Hartley theorem developed under the hypothesis of an infinite length frame and to approach Shannon capacity it is necessary to use codes with large block length. However, in the presence of URLLC-enabled wireless communication networks, sending information with an infinite-block length regime is impracticable. As a result, short-packet data transmission is used to meet both the reliability and latency requirements of wireless communication networks, which has been theoretically studied using Finite block-length information theory.^[4] Furthermore, finite block-length information theory provides a precise framework for determining the relationship between wireless communication latency and reliability.^[5] The maximal achievable channel coding rate [math]\displaystyle{ \left ( \bar{R} \right ) }[/math] with given block error probability [math]\displaystyle{ \left ( \epsilon \right ) }[/math] and block-length [math]\displaystyle{ \left ( n \right ) }[/math] (for binary Additive white Gaussian noise (AWGN) channels, with short block lengths) , closely approximated by Polyanskiy, Poor and Verdú (PPV) in 2010, is given by

[math]\displaystyle{ \bar{R} \approx C-\sqrt{\frac{V}{n}}Q^{-1}\left ( \epsilon \right ) }[/math]

where [math]\displaystyle{ Q^{-1} }[/math] is the inverse of the complementary Gaussian cumulative distribution function, [math]\displaystyle{ C }[/math] is the channel capacity and [math]\displaystyle{ V }[/math] is a characteristic of the channel referred to as channel dispersion.

See also

• Rate–distortion theory
• Channel capacity
• Shannon–Hartley theorem
• Decorrelation
• Rate–distortion optimization
• Source coding
• 5G

References

0.00 (0 votes)