Appreciation of the paper - John Costas: Linear System Coding

© 1952 JP Costas
© 2023 Conmajia

Author brief introduction John Peter Costas (1923-2008), an American electrical engineer, invented the Costas ring and the Costas array. Costas participated in World War II, and after the war entered the Massachusetts Institute of Technology to study for a doctorate, and then worked for General Electric until retirement. In 1965, Costas was elected a fellow of IEEE.

World Lines Convergence Costas's "Linear System Coding" paper was published in 1952 and appears on the second half of page 1101 of IRE ¹ Proceedings of the IRE, September 1952. Interestingly, the previous article in this article—the one that ends on page 1101 in the first half—was entitled "A Method of Constructing Minimal Redundancy Codes" by David Albert Herr Huffman: Yes, it was the article that gave birth to the famous Huffman ² algorithm in information science.

Archeology Many contents of this article, such as impulse response, etc., seem to be common sense to engineers who have studied the "Signals and Systems" course. However, in the 1950s when this article was written, these "common senses" required specific definitions and references in the text.

---

The following is the paper.

---

Abstract This paper considers message transmission over noisy channels. Two linear networks are designed: one to process the message before transmission; the second to filter the processed message plus channel noise at the receiver. The method presented in this paper can minimize the mean square error between the actual output and the expected output of the transmission circuit under a given allowable average signal power through proper network design. The paper also presents and discusses the results of numerical examples.

Linear System Coding

I. Introduction

The author discusses in a forthcoming Ref. 4 the concept of the importance of statistical methods pioneered by Wiener (Ref. 1, 2) and developed by Lee (Ref. 3) in filter design. In short, for a system whose input is a given sum of message and noise as shown in Figure 1, we wish to find a filter characteristic that provides the best performance. By "best", we mean such that the actual output $f_o\left(t\right)$ and desired output$f_d\left(t\right)$ minimizes the mean square error between such filters. That is to say, if$\epsilon$ represents the mean square error of filtering, that is,

$$e=\underset{T\to \infty }{\lim }\frac{1}{2T\_}{\int }_{-T}^T{\left[f_o(t)-f_d(t)\right]}^2\mathrm{d}t.\text{\hspace{1em}(1)}$$

Figure 1. Traditional filter problem

Such an error criterion is certainly sound from a physical point of view, but contrary to popular belief, it is by no means the only well-tested mathematical treatment for error measurement (ref. 5).

The desired filter output is usually the message function $f_m\left(t\right)$ . However, output other than messages may be desired. For example, one might require a network design to be able to filter, predict messages from noise, and differentiate outcomes in seconds. Thus, we can ask for prediction, filtering, and differentiation in one operation (Ref. 3). In this sense, a (filter) network can be viewed as an operator rather than just a filter (Ref. 1).

If only linear systems are considered, the statistical parameters that need to be designed are called correlation functions. Random function $f_1(t)$、$f_2\left(t\right)$ between the cross-correlation function${\varphi }_{12}\left(\tau \right)$ is defined as

$${\varphi }_{12}\left(t\right)=\underset{T\to \infty }{\lim }\frac{1}{2T\_}{\int }_{-T}^T{f}_1(t)f_2(t+\tau )\mathrm{d}t.\text{\hspace{1em}(2)}$$

Random function $f_1\left(t\right)$ autocorrelation function${\varphi }_{11}\left(\tau \right)$ is defined as

$${\varphi }_{11}\left(t\right)=\underset{T\to \infty }{\lim }\frac{1}{2T\_}{\int }_{-T}^T{f}_1(t)f_1(t+\tau )\mathrm{d}t.\text{\hspace{1em}(3)}$$

Fourier transform on $g(t)$、$G\left(\omega \right)$ is

$$G\left(\omega \right)=\frac{1}{2p.m}{\int }_{-\infty }^{\infty }g(t)e^{-j\omega t}dt\text{\hspace{1em}(4)}$$

and

$$g(t)={\int }_{-\infty }^{\infty }G\left(\omega \right)e^{+j\omega t}d\omega .\text{\hspace{1em}(5)}$$

Laplace transform also uses (4), (5) two formulas, $\omega$ is changed to

$$l=oh+\mathrm{j}\sigma .\text{\hspace{1em}(6)}$$

An important theorem of Wiener states that the random function $f_1\left(t\right)$ The power spectral density is given by the Fourier transform of its autocorrelation function (References 3, 6), namely

$${Phi}_{11}\left(oh\right)=\frac{1}{2p.m}{\int }_{-\infty }^{\infty }{\varphi }_{11}\left(t\right)e^{-j\omega \tau }d\tau .\text{\hspace{1em}(7)}$$

Similarly, we can define the random function $f_1(t)$、$f_2\left(t\right)$ cross power spectrum${Phi}_{12}\left(\omega \right)$ is

$${Phi}_{12}\left(oh\right)=\frac{1}{2p.m}{\int }_{-\infty }^{\infty }{\varphi }_{12}\left(t\right)e^{-j\omega \tau }d\tau .\text{\hspace{1em}(8)}$$

We define the unit impulse function $u\left(t\right)$ is

$$u(t)=\underset{a\to \infty }{\lim }\frac{a}{\sqrt{Pi}}{e}^{-a^{2t}{\_}^2}.$$

So, we can use formula (4) to get u ( t ) u(t)Transformation of$u$$($$t$$)$$\cup \left(\omega \right)$ is

$$\cup \left(\omega \right)=\frac{1}{2p.m}.\text{\hspace{1em}(9)}$$

Now, if $h\left(t\right)$ is the response of a linear system to a unit impulse, and it can be proved that the linear system responds to an arbitrary input$f_i\left(t\right)$ output$f_o\left(t\right)$ is given by (ref. 3)

$$f_o(t)={\int }_{-\infty }^{\infty }h\left(\sigma \right)f_i(t-s)ds.\text{\hspace{1em}(10)}$$

Let ${ϵ}_o\left(t\right)$ is the linear system for the transient input${ϵ}_i\left(t\right)$ transient output. Define the system function H ( ω ) H(\omega)of the linear system$H\left(\omega \right)$ to satisfy

$$H\left(\omega \right)=\frac{E_o(oh}{E_i\left(oh\right)}\text{\hspace{1em}(11)}$$

We know that $H\left(\omega \right)$ and$h\left(t\right)$ has the following relationship

$$H\left(\omega \right)={\int }_{-\infty }^{\infty }h(t)e^{-j\omega t}dt\text{\hspace{1em}(12)}$$

and

$$h(t)=\frac{1}{2p.m}{\int }_{-\infty }^{\infty }H\left(\omega \right)e^{+j\omega t}d\omega ,\text{\hspace{1em}(13)}$$

divide by $2\; Except\; for\; the\; position\; ofthe\; \pi$ item, it is the same as (4) and (5).

II. Transmission Problems

The filter design problem shown in Figure 1 has been studied in great detail by Wiener (Ref. 1) and Li (Ref. 3) and verified experimentally by CA Stater (Ref. 8). Therefore, in this report we will consider the more general case as shown in Figure 2. In most communication systems, there is an opportunity to modify or "encode" the information to be transmitted before it is introduced into the transmission channel. Therefore H ( ω ) H(\omega) must be designed in such a way as to "predistort" or "precode" the message$H\left(\omega \right)$ network such that the "decoding" or filtering network$G\left(\omega \right)$ produces an output that is a better mean square approximation of the desired output than the unprocessed message.

Figure 2. Transmission circuit

fo ( t ) f_o(t) in Figure 2$f_o(t)$、$f_d\left(t\right)$ The mean square error between

$$e=\underset{T\to \infty }{\lim }\frac{1}{2T\_}{\int }_{-T}^T{\left[\begin{array}{rl}& {\int }_{-\infty }^{\infty }d\sigma g\left(\sigma \right)f_n(t-s)\\ & +{\int }_{-\infty }^{\infty }d\sigma g\left(\sigma \right){\int }_{-\infty }^{\infty }dvh\left(v\right)f_m(t-p-v)\\ & -f_d(t)\end{array}\right]}^2.\text{\hspace{1em}(14)}$$

Expand (14) and rewrite it in the relevant functional form to get

$$\begin{array}{rl}e=& {\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }d\sigma dvg\left(\sigma \right)g\left(v\right){\varphi }_{nn}(p-v)\\ & +2{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }d\xi d\sigma dvg\left(\xi \right)g\left(\sigma \right)h\left(v\right){\varphi }_{nm}(x-p-v)\\ & +{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }d\psi d\xi d\sigma dvg\left(\psi \right)h\left(\xi \right)g\left(\sigma \right)h\left(v\right){\varphi }_{mm}(p+X-p-v)\\ & -2{\int }_{-\infty }^{\infty }d\sigma g\left(\sigma \right){\varphi }_{nd}\left(p\right)\\ & -2{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }d\sigma dvg\left(\sigma \right)h\left(v\right){\varphi }_{md}(p+v)+{\varphi }_{dd}(0).\end{array}\text{\hspace{1em}(15)}$$

To make (15) where ε \varepsilonTo minimize$\epsilon$$g(t)$、$h\left(t\right)$ , ie

$$g(t),h(t)=0\text{to}t<0.\text{\hspace{1em}(16)}$$

In addition, the encoding network $H\left(\omega \right)$ imposes an additional constraint on the average transmitted signal power, but this is left alone here.

First we try to assume $H\left(\omega \right)$ is a fixed value, solve its optimal value$G\left(\omega \right)$ . This can be obtained by ordering (15) where$g\left(t\right)$ takes an acceptable variable value$ϵ\eta \left(t\right)$ , where

$$h\left(t\right)=0\text{to}t<0\text{\hspace{1em}(17)}$$

And the parameter $ϵ$ relative to$\eta$、$h$ independent. That is, we use$g(t)+ϵ\eta \left(t\right)$、$e+\delta \epsilon$ replaced g ( t ) g(t)in (15) respectively$g\left(t\right)$ ,$\epsilon$ . Now, if a certain$g\left(t\right)$ can give the minimum mean square error, then the optimal$g\left(t\right)$ must satisfy at the same time

$${\frac{\partial (e+de}{\partial ϵ}|}_{ϵ=}=0.\text{\hspace{1em}(18)}$$

Using formula (15) to expand formula (18), we can get

$$\begin{array}{rl}& {\int }_{-\infty }^{\infty }dvg\left(v\right){\varphi }_{nn}(p-v)+{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }d\xi dvh\left(v\right)g\left(\xi \right){\varphi }_{nm}(x-p-v)\\ +& {\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }d\xi dvh\left(v\right)g\left(\xi \right){\varphi }_{nm}(p-X-v)\\ +& {\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }d\psi d\xi dvh\left(\xi \right)h\left(v\right)g\left(\psi \right){\varphi }_{mm}(p+X-p-v)\\ -& {\varphi }_{nd}\left(p\right)-{\int }_{-\infty }^{\infty }dvh\left(v\right){\varphi }_{md}(p+v)\\ =& q(\sigma \end{array}\text{\hspace{1em}(19)}$$

where $The\; q\left(\sigma \right)$ function is defined as

$$q\left(s\right)=0\text{against}you>0.\text{\hspace{1em}(20)}$$

Now for (20) where σ \sigmaThe Laplace transform of$\sigma \; can\; be\; obtained$

$$G\left(\lambda \right)F\left(\lambda \right)-{Phi}_{nd}\left(l\right)-H(-\lambda )F_{md}\left(l\right)=Q(\lambda )\text{\hspace{1em}(21)}$$

where $F\left(\lambda \right)$ function is defined as

$$F\left(\lambda \right)=H\left(\lambda \right)H(-\lambda ){\Phi }_{mm}\left(l\right)+H\left(\lambda \right)F_{nm}\left(l\right)+H(-\lambda )F_{mn}\left(l\right)+{Phi}_{nn}\left(l\right).\text{\hspace{1em}(22)}$$

We assume that $F\left(\lambda \right)$ can be factorized as

$$F\left(\lambda \right)=F^{+}\left(\lambda \right)\cdot F^{-}\left(\lambda \right)\text{\hspace{1em}(23)}$$

where $F^{+}\left(\lambda \right)$containsλ \lambdaAll poles and zeros of the upper half of the$\lambda$$F^{-}\left(\lambda \right)$contains$The\; lower\; half\; of\; the\; lambda$ plane with all its poles and zeros. With the help of formulas (4), (5), and (6), formula (21) can be rewritten as

$$\begin{array}{rl}& G\left(\lambda \right)F^{+}\left(\lambda \right)-\frac{1}{2p.m}{\int }_0^{\infty }{e}^{-\mathrm{j}\lambda t}\mathrm{d}t{\int }_{-\infty }^{\infty }\frac{{Phi}_{nd}\left(oh\right)+H(-\omega )F_{md}(oh}{F^{-}\left(\omega \right)}{e}^{+j\omega t}d\omega \\ -& \frac{1}{2p.m}{\int }_{-\infty }^0{e}^{-j\lambda t}dt{\int }_{-\infty }^{\infty }\frac{{Phi}_{nd}\left(oh\right)+H(-\omega )F_{md}(oh}{F^{-}\left(\omega \right)}{e}^{+j\omega t}d\omega \\ =& \frac{Q(\lambda )}{F^{-}\left(\lambda \right)}.\end{array}\text{\hspace{1em}(24)}$$

The first two terms on the left side of (24) include $All\; possible\; poles\; in\; the\; upper\; half\; of\; the\; \lambda$ plane, and the third term contains the poles atλ \lambdaAll poles in the lower half of the$lambda\; plane.$Since the right side of (24) has only the poles of the lower half plane, the sum of the first two terms on the left side must be a certain constant. It can be shown that this constant is zero, so that

$$G\left(\lambda \right)=\frac{1}{2\pi F^{+}\left(\lambda \right)}{\int }_0^{\infty }{e}^{-\mathrm{j}\lambda t}\mathrm{d}t{\int }_{-\infty }^{\infty }\frac{{Phi}_{nd}\left(oh\right)+H(-\omega )F_{md}(oh}{F^{-}\left(\omega \right)}{e}^{+j\omega t}d\omega .\text{\hspace{1em}(25)}$$

For fixed $For\; H\left(\lambda \right)$ network, formula (25) gives the optimal transfer function of the decoding network. The system function G ( λ ) G(\lambda)given by (25)$G\left(\lambda \right)$ is always achievable.

(25) has two interesting special cases. One is to select $H\left(l\right)=1$ , we have

$$\begin{array}{rl}F\left(\lambda \right)& ={Phi}_{mm}\left(l\right)+{Phi}_{nm}\left(l\right)+{Phi}_{mn}\left(l\right)+{Phi}_{nn}\left(l\right)\\ & ={Phi}_{ii}\left(l\right)\end{array}\text{\hspace{1em}(26)}$$

Thus, $F\left(\lambda \right)$ becomesGGFourier transform of the autocorrelation function input to the$G\; network.$For$G\left(\lambda \right)$ , we have

$$G\left(\lambda \right)=\frac{1}{2p{F}_{ii}^{+}\left(l\right)}{\int }_0^{\infty }{e}^{-\mathrm{j}\lambda t}\mathrm{d}t{\int }_{-\infty }^{\infty }\frac{{Phi}_{id}(oh}{{Phi}_{ii}^{-}\left(oh\right)}{e}^{+j\omega t}d\omega \text{\hspace{1em}(27)}$$

Remove ${Phi}_{id}\left(\lambda \right)$ represents the cross-power spectrum between the filter input and the desired output. Equation (27) is the optimal filter equation obtained by Wiener and Li, and it is also the solution to the problem in Fig. 1.

The second interesting special case occurs when the noise function in Figure 2 is zero. At this point (25) becomes

$$G\left(\lambda \right)=\frac{1}{2\pi H^{+}\left(l\right)F_{mm}^{+}\left(l\right)}{\int }_0^{\infty }{e}^{-\mathrm{j}\lambda t}\mathrm{d}t{\int }_{-\infty }^{\infty }\frac{H(-\omega )F_{md}(oh}{H^{-}\left(\omega \right)F_{mm}^{-}\left(oh\right)}{e}^{+j\omega t}d\omega \text{\hspace{1em}(28)}$$

in

$$H^{+}\left(\lambda \right)H^{-}\left(\lambda \right)=H\left(\lambda \right)H(-\lambda )\text{\hspace{1em}( 28a )}$$

This is the so-called "optimum compensator formula". (28) was derived by YW Li, although he has never published it before.

If consider $G\left(\omega \right)$ is fixed, then using the same method used above, we can find the optimal$H\left(\lambda \right)$ is given by

$$H\left(l\right)=\frac{1}{2\pi G^{+}\left(l\right)F_{mm}^{+}\left(l\right)}{\int }_0^{\infty }{e}^{-\mathrm{j}\lambda t}\mathrm{d}t{\int }_{-\infty }^{\infty }\left[\frac{G(-\omega ){\Phi }_{md}(oh}{G^{-}\left(\omega \right)F_{mm}^{-}\left(oh\right)}-\frac{G^{+}\left(o\right)F_{mn}(oh}{{Phi}_{mm}^{-}\left(oh\right)}\right]e^{+j\omega t}d\omega \text{\hspace{1em}(29)}$$

in

$$G^{+}\left(\lambda \right)G^{-}\left(\lambda \right)=G\left(\lambda \right)G(-\lambda ).\text{\hspace{1em}( 29a )}$$

If the channel noise is zero, or the cross-correlation between the message and the noise is zero, then (29) reduces to an optimal compensator formula.

(25) gives the fixed H ( λ ) H(\lambda)The optimal G ( λ ) G(\lambda)under$H$$($$\lambda$$)$$G\left(\lambda \right)$ , while (29) gives the fixedG ( λ ) G(\lambda)The optimal H ( λ ) H(\lambda)under$G$$($$\lambda$$)$$H\left(\lambda \right)$ . If equations (25) and (29) are solved simultaneously, the optimal encoding-decoding network pair of the transmission circuit shown in Fig. 2 can be obtained. However, before attempting this solution, it is convenient to first solve for a fixed$H\left(\lambda \right)$ and optimized$G\left(\lambda \right)$ the resulting mean square error. Substitute (19) into (15) to get

$$e_{\text{min}}^{(\text{Fixed H }}={\varphi }_{dd}(0)-{\int }_{-\infty }^{\infty }d\sigma g\left(\sigma \right){\varphi }_{nd}\left(p\right)-{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }d\sigma dvg\left(\sigma \right)h\left(v\right){\varphi }_{md}(p+v).\text{\hspace{1em}(30)}$$

This equation can be rewritten in Fourier transform form,

$$e_{\text{min}}^{(\text{Fixed H }}={\int }_{-\infty }^{\infty }\mathrm{d}\omega \left[F_{dd}\left(oh\right)-{Phi}_{nd}\left(\omega \right)G(-\omega )-G(-\omega )H(-\omega ){\Phi }_{md}\left(o\right)\right].\text{\hspace{1em}(31)}$$

It must be remembered that g ( σ ) g(\sigma) in (30), (31)$g\left(\sigma \right)$、$G\left(\omega \right)$ is not chosen arbitrarily, but the solution of formula (19).

III. Long Delay Solutions

When solving the optimal network pair shown in Figure 2, we assumed zero cross-correlation between message and channel noise. We further assume that the solution is long-delayed, that is, the desired output is the delayed message. So there is

$$f_d(t)=f_m(t-a),a\to \infty \text{\hspace{1em}(32)}$$

and

$${Phi}_{md}\left(oh\right)={Phi}_{mm}\left(o\right)e^{-\mathrm{j}\omega a},a\to \infty .\text{\hspace{1em}(33)}$$

Under this condition, (25) can be rewritten as

$$G\left(\omega \right)\to \frac{H(-\omega )F_{mm}\left(o\right)e^{-j}}{{|H(\omega )|}^2{Phi}_{mm}\left(oh\right)+{Phi}_{nn}\left(oh\right)},a\to \infty .\text{\hspace{1em}(34)}$$

Since the best filtering result can be obtained when a long delay is allowed (Document 3), substituting (34) into (31) will get the smallest error, which is the so-called irreversible error. Therefore, we end up with

$$e_{\text{irr}}^{(\text{Fixed H }}={\int }_{-\infty }^{\infty }\frac{{Phi}_{mm}\left(o\right)F_{nn}(oh}{{|H(\omega )|}^2{Phi}_{mm}\left(oh\right)+{Phi}_{nn}\left(oh\right)}\mathrm{d}\omega .\text{\hspace{1em}(35)}$$

Note that the non-cancellable error only depends on the transfer function $The\; magnitude\; of\; H\left(\omega \right)$ , not the phase. From formula (34), we can see that because of fixing$Any\; phase\; contribution\; caused\; by\; H\left(\omega \right)$ will be absorbed by the optimal decoding network$G\left(\omega \right)$ removal.

For a given $H\left(\omega \right)$ , (35) will give the network G ( ω ) G(\omega)designed according to (34)The transmission error generated by$G$$($$\omega$$)\; .$Therefore, we must find the H ( ω ) H(\omega)that minimizes the error in (35)$H\left(\omega \right)$ while keeping the average transmitted signal power constant. That is, given${|H(\omega )|}^2{Phi}_{mm}\left(\omega \right)$ represents the power spectral density output by the encoding network, we must make

$${\int }_{-\infty }^{\infty }{|H(\omega )|}^2{Phi}_{mm}\left(\omega \right)d\omega =c_1.\text{\hspace{1em}(36)}$$

Ruoling

$${\left[y\left(\omega \right)\right]}^2={|H(\omega )|}^2{Phi}_{mm}\left(oh\right)\text{\hspace{1em}(37)}$$

Then equations (35) and (36) can be rewritten as

$$\frac{e_{\text{irr}}}{2}={\int }_0^{\infty }\frac{{Phi}_{mm}\left(o\right)F_{nn}(oh}{{\left[y\left(\omega \right)\right]}^2+{Phi}_{nn}\left(oh\right)}\mathrm{d}\omega \text{\hspace{1em}(38)}$$

and

$${\int }_0^{\infty }{\left[y\left(\omega \right)\right]}^{2}d\omega =\frac{c_1}{2}.\text{\hspace{1em}(39)}$$

What we are looking for now is the real function $y\left(\omega \right)$ . This is the isoperimetric condition of the so-called variational method. By applying a commonly used technique (Ref. 9), one obtains

$${|H(\omega )|}^2{Phi}_{mm}\left(oh\right)=-F_{nn}\left(oh\right)+\frac{1}{\sqrt{c}}\sqrt{{Phi}_{mm}\left(o\right)F_{nn}\left(oh\right)}\text{\hspace{1em}(40a)}$$

and

$${|H(\omega )|}^2{Phi}_{mm}\left(oh\right)=0\text{\hspace{1em}(40b)}$$

where $\gamma$ is a constant that satisfies the formula (39). If the right side of formula (39) is greater than zero, formula (40a) is used; otherwise, formula (40b) must be used. Physically, this means that$H\left(\omega \right)$ may contain cut-off bands. However, due to the assumption that through$H\left(\omega \right)$ to$G\left(\omega \right)$ has an infinite delay time, so the existence of this cut-off band does not violate the Paley-Wiener theorem (References 1, 10, 11).

IV. Discussion of Results

As a check on the results of Section III, we have the noise spectrum

$${Phi}_{nn}\left(oh\right)=a^2\text{\hspace{1em}(41)}$$

and message spectrum

$${Phi}_{mm}\left(oh\right)=\frac{b/}{{oh}^2+b^2}.\text{\hspace{1em}(42)}$$

Take2 $2a{\_}^{2}b=1/5\pi$，$c_1=1$ , the mean square error given by (38) is$0.285$ . Without encoding and using the same average signal power, let${|H(\omega )|}^2=1$ , the mean square error can be calculated from formula (35) as$0.302$ . Therefore, the optimal encoding network has some contribution to improve the transmission performance, but not much. For the special case cited, when over$oh=After\; 8.45\beta$ ,$H\left(\omega \right)$ cannot be transmitted; when the noise level increases by 5 times, this upper cut-off frequency drops to$oh=3.25b$ .

It can be seen that Figure 2 can represent a communication circuit using amplitude modulation (see Reference 4). The use of frequency modulation complicates matters, but it has also been suggested that

$${Phi}_{nn}\left(oh\right)=a^2{o}^2\text{\hspace{1em}(43)}$$

The noise spectrum of may be of interest. It was found that using the message spectrum in (42) and the noise spectrum in (43) yields only modest improvements for networks using optimal coding or "pre-emphasis".

The above modest improvement in mean squared error is partly due to the assumption of a specific frequency spectrum. Therefore, in some cases, considerable improvements may be achieved with an appropriate encoding network.

references

1. N. Wiener: The Extrapolation, Interpolation, and Smoothing of Stationary Time Series with Engineering Applications, John Wiley, New York, 1949
2. N. Wiener: Cybernetics, John Wiley, New York, 1948
3. Y. W. Lee: Course 6.563, Statistical Theory of Communication, Class Notes, M.I.T. unpublished
4. J. P. Costas: Interference Filtering, Technical Report No. 185, Research Laboratory of Electronics, M.I.T. March 1951
5. Y. W. Lee: Memorandum on Error Criteria in Filter Design, unpublished
6. N. Wiener: Generalized Harmonic Analysis, Acta Math. 55, 117-258, 1930
7. Y. W. Lee, C. A. Stutt: Statistical Prediction of Noise, Techinical Report No. 129, Research Laboratory of Electronics, M.I.T. July 1949
8. C. A. Stutt: An Experimental Study of Optimum Linear Systems, Doctoral Thesis, Dept. of Electrical Engineering, M.I.T. May 15, 1951
9. H. Margenau, G. M. Murphy: The Mathematics of Physics and Chemistry, Van Nostrand, New York, 1943
10. G. E. Valley, Jr., H. Wallman: Vacuum Tube Amplifiers, McGraw-Hill, New York, 1948
11. R. Paley, N. Wiener: Fourier Transforms in the Complex Domain, Am. Math. Soc. Colloq. Pub. 19, 1934

thank you

The author would like to thank Prof. YW Lee, Dr. CA Stutt, and Mr. CA Desoer, MIT Research Laboratory of Electronics, for their many helpful comments and suggestions.

© 1952 JP Costas
© 2023 Conmajia

---

1. IRE (Institute of Radio Engineers) was established in 1912 as the predecessor of IEEE (Institute of Electrical and Electronics Engineers). In 1963, IRE merged with AIEE (American Institute of Electrical Engineers) to form IEEE. ↩︎

2. Due to differences in the level of translators and folk transliteration habits, David Huffman is often translated as "David Huffman". According to the "(Foreign) Personal Names Translation Handbook" series published by the Commercial Press, the official translated name should be "David Huffman". ↩︎