Sometimes signals can be stubbornly non-periodic. This can make digital signal processing so much more annoying. The supposedly simple act of plotting the spectrum ends up being a game of trial-and-error, or requires making a decision that involves prior knowledge of what one would actually like to know in the first place. We’ll have a look at why this is so and what to do about it.

#### About leakage and dynamic range

Basically, leakage is a consequence of looking at only a finite segment of a signal instead of the whole signal extending over all time, from negative to positive infinity. In the most simple case, the signal segment $s^{\prime}(t)$ is isolated by multiplying a signal $s(t)$ with a box function such that

$$s^{\prime}(t) = s(t) \cdot w_\mathrm{rect}(t) \quad\text{with} \quad w_\mathrm{rect}(t) = \begin{cases} 1 & -\frac{T}{2} \le t \lt \frac{T}{2} \\ 0 & \vphantom{\frac{T}{2}}\mathrm{else} \end{cases}\tag{1}$$

This multiplication in the time domain corresponds to a convolution in the frequency domain, i.e. each frequency component contained in $s(t)$ is convolved with the Fourier transform of the box function, which is

$$\tilde w_\mathrm{rect}(f) = T \,\, \mathrm{sinc} \bigl(f \, T\bigr) \quad\text{with}\quad \mathrm{sinc}\,x = \frac{\sin \pi x}{\pi x} \tag{2}$$

where the tilde denotes a Fourier-transformed quantity. Part of the energy contained in that frequency component is distributed over the entire frequency domain. This is referred to as *leakage*. One disadvantage of such strong leakage is that small signals in the vicinity of strong signals can be completely masked if the leakage energy at some frequency is larger than the signal energy at that frequency. This greatly reduces the dynamic range of the spectral analysis. The box window has a very low dynamic range.

#### Windows

As long as the segment $s^{\prime}(t)$ is to be of finite duration, leakage cannot be prevented. However, by using functions other than the box, the distribution of leakage energy in the spectrum can be shaped, since that is described by the Fourier transform of the window. To make things a bit simpler, we will normalize window functions so that $T=1$. We will also follow Eric Weisstein and call their Fourier transforms *instrument functions* [1]. Figures 1 through 3 illustrate $w_\mathrm{rect}(t)$ and its instrument function, as well as the Hann and flat-top windows, two often used windows. The energy spectral density (ESD), shown in the figures, is defined as the squared magnitude of the instrument function.

The Hann window is defined as

$$w_\mathrm{hann}(t) = \begin{cases} \cos^2 \bigl(\pi t \bigr) & -\frac{1}{2} \le t \lt \frac{1}{2} \\ 0 & \vphantom{\frac{1}{2}}\mathrm{else} \end{cases} \tag{3}$$

which is actually the product of a raised cosine function with the box window $w_\mathrm{rect}(t)$. No matter the function that describes the shape of the window, it must always be bounded in time by multiplication with the box – that is also the reason why there is always leakage: the instrument function of $w_\mathrm{hann}(t)$ is the convolution of the infinitely extended $\tilde w_\mathrm{rect}(f)$ with another function (most often it’s a number of Dirac delta peaks resulting from a sum of cosines). The flat-top window is often defined as

$$\begin{gather}

w_\mathrm{flat}(t) = \begin{cases} \sum_{n=0}^4 a_{n} \cos \bigl(2 n \pi t \bigr) & -\frac{1}{2} \le t \lt \frac{1}{2} \\ 0 & \vphantom{\frac{1}{2}}\mathrm{else} \end{cases} \tag{4}\\

\text{with} \vphantom{\frac{1}{2}} \quad a_0 = 0.21557895 \quad a_1 = 0.41663158 \quad a_2 = 0.277263158\\

\quad a_3 = 0.083578947 \quad a_4 = 0.006947368

\end{gather}$$

The sidelobes of both, the Hann window and the flat-top window, are significantly lower than those of the rectangular window, increasing the dynamic range in the vicinity of a strong peak. In the case of the Hann window they also fall off much more rapidly, their slope is steeper. The flat-top falls off more slowly, but it has a very large dynamic range over a broad spectral range.

I have compiled an overview of a much larger number of common window functions on a separate page, Window Function Reference.

#### Resolution

Comparing the three windows, we also notice that their main lobe width differs significantly. Main lobe width can be defined either as the 3dB (half-)width or the width to the first spectral zero. Within the main lobe the dynamic range is very low – if two spectral components of a signal are separated by less than their main lobe (3dB) width, they are virtually indistinguishable. The minimum separation for detectability determines the *resolution* of the window.$^1$ The box window has the highest possible resolution (at the expense of low dynamic range). The Hann window is a middle-ground trade-off between resolution and dynamic range. The flat-top window has very low resolution owing to its significant main lobe width. Other windows usually weight resolution and dynamic range somewhere between the box and the flat-top.

#### Discrete Signals

Spectral leakage occurs for continuous and discrete signals alike, as long as they are not of infinite duration. However, there are some interesting peculiarities when working with discrete signals (such as the apparent absence of leakage in some signals), for which reason we’ll look at these a bit more in detail.

The discrete Fourier transform (DFT) takes a discrete sample sequence and transforms it into another discrete sample sequence. If the first sequence is interpreted as consecutive samples of a time signal (segment), the second sequence corresponds to the sampled Fourier spectrum of the time signal, with some caveats.$^2$ When there are $N$ time samples within the period $T$, the sample duration is

$$\Delta t = \frac{T}{N}$$

The resulting frequency range spanned by the discrete spectrum and spacing between frequency samples (or “bins”) are

$$F = \frac{1}{\Delta t} = \frac{N}{T} \quad \text{and} \quad \Delta f = \frac{1}{T}$$

respectively. See also my DFT cheat sheet [2] for illustrated relations between the various time and frequency quantities.

Looking at the instrument function (2) of the box window, we see that it has zeros at all non-zero integer multiples of $1/T = \Delta f$ – which is the same spacing as between the samples of the DFT frequency spectrum. This means that if a spectral component of $s(t)$ corresponds exactly to one of the frequencies sampled by the DFT spectrum, all other frequency samples coincide with zeros of the box window instrument function, and thus it appears as if there is no leakage at all! However, this is really just an artifact of the sampled nature of the DFT spectrum. The particular frequency components for which this occurs (those at integer multiples of $\Delta f$) are those corresponding to an integer number of full oscillations within $T$. Hence, any signal segment for which all frequency components (and thus the segment itself) are periodically extensible without jumps (for all its frequency components) is transformed leakage-free and thus exact.

With real-world signals, the segment $s’(t)$ will be properly periodic only in the rarest of cases, and almost always we will have to deal with leakage.

#### Scalloping

Another effect peculiar to discrete spectra is called *scalloping*. If a frequency component of $s(t)$ falls outside any frequency that is sampled by the DFT spectrum, then its convolution with the instrument function is not sampled at the (known) maximum of the resulting curve but slightly offset. The maximum possible offset is half the separation between two DFT samples, or $\Delta f / 2$. The maximum possible scalloping is thus the instrument function difference between $f=0$ and $f=\Delta f/2$. When scalloping is large then the obtained energy reading of a frequency component depends on the frequency of that spectral component relative to the DFT sampling points. This scalloping is largest for the box window – its instrument function falls off rapidly, which also results in its excellent resolution. The flat-top window – hence the name – falls off very slowly, and scalloping is usually not an issue, though it results in poor resolution.

Since by means of scalloping the energy in a frequency component is distributed into several DFT bins, it can be used to approximate the exact frequency of that component even if it does not coincide with a DFT bin, if the instrument function is known. When multiple frequency components are superposed in a signal, this is generally not possible, however, since their overlapping contributions to the DFT spectrum cannot be separated.

Figure 4 shows the relative energy detected in neighboring DFT bins (and hence the scalloping) when the frequency of a single spectral component is swept between the bin frequencies for the box, Hann, and flat-top windows. The 3dB half-width of the main lobe is also shown.

#### Equivalent Noise Bandwidth

The ENBW has its very own post here.

#### Agony of choice

So which window is the best one – box, Hann, flat-top, or any of the other windows profiled on the overview page or somewhere else? The answer is: that depends. It depends on the kind of signal we want to measure.

If the exact energy of the frequency components in the signal is important, windows with very little scalloping such as the flat-top are usually the best choice. However, when multiple frequency components can occur close to each other, the low resolution of these windows will blur them into a single peak without usable information about the energy content of the single components.

If resolution is important, we usually need to sacrifice dynamic range near strong peaks, and vice versa. The box window offers the highest resolution, but has very low dynamic range. The flat-top has a high dynamic range, but poor resolution. The Hann window balances resolution and dynamic range and is often a good starting point for spectral analysis.

Figure 5 illustrates the advantages and disadvantages of our three window types. A time signal is composed of various frequency components (grey peaks in each graph) and analyzed used each window. The energy spectral density values were normalized so that $\tilde w(0)$ (the main lobe height) corresponds to one. This is necessary to obtain the correct energy information since each window has a deterministic correction factor that depends on the energy of the window function itself (window functions $w(t)$ that have small values most of the time reduce the energy in the windowed signal) and the leakage properties (the portion of the instrument function energy that is contained in the main lobe and the width of the main lobe determine its height).

The rectangular box window clearly separates the closely spaced frequency components, even though one of them is severely affected by scalloping. The three weaker signal are not detected at all due to strong leakage. The Hann window can still separate the two strong signals, but not as clearly; also scalloping is reduced. With it, the strongest of the weak signals is also detectable, and due to the steep slope of its sidelobes the weakest signal, well separated from the strong peaks, is also detectable. Only one signal is masked by leakage. The flat-top window cannot properly separate both strong signals due to its poor resolution, but can detect the two strongest of the weak signal, both with correct amplitude owing to the absence of scalloping. The weakest signal is not properly detectable, since the sidelobe slope of the flat-top window is rather flat.

Since the choice of the best window depends to a large extent on the spectral properties of the signal, and the latter is usually the property we’d like to know, it is normally a good idea to try different window functions and to look for weak signals, signals in close proximity, etc. with the proper window. The Hann window is usually a good starting point because of its dynamic range vs. resolution balance, but it is not the right window for every job.

#### Notes on signal processing

Maybe I should emphasize an issue with the DFT here, since that wasn’t really clear to me (in the proper way) until recently: Using the DFT for signal processing is usually not a very good idea, unless one is dealing with signals that are mostly periodic (except for noise and/or other weak spectral components of no interest). These are usually generated by an inverse DFT (like the symbols in OFDM communication or signals in optical communication simulators).

Using a window when doing computations on a signal in the frequency domain is usually also not a good idea, since windowing significantly changes the time-domain signal and the operations in the frequency domain do not only affect the signal itself, but also the window function, so that a simple multiplication with the inverse of the window after transformation back into the time domain will not yield the correct result.

With the right kind of signal, the DFT can be used very effectivly e.g. in the calculation of signal propagation in optical fibers DFT algorithms can be used to calculate the effects of chromatic dispersion in the frequency domain very quickly. However, with segments of real-world signals, leakage will introduce distortions at the segment boundaries that may be problematic – in those cases it would be prudent to use FIR filter implementations.

**1** Resolution in this context seems to be a poorly defined term. In optics, there is the Rayleigh criterion that defines if some optical instrument is able to resolve two distinct sources or not. I have not found anything similar in a (brief) search on the topic.

**2** The sampling in the frequency domain corresponds to a periodic extension of the time signal, hence it’s actually the transform of the periodically extended signal segment. Also, the sampled time signal corresponds to a periodic extension of the signal spectrum. If the spectrum is wider than the repetition period (which is called undersampling), the repeated spectra will overlap – this overlap cannot be undone. See the Nyquist theorem.

[1] Instrument Function [MathWorld]

[2] DFT Cheat Sheet [Nerd Rage]

[3] Window Function Reference [Nerd Rage]

last posts in signal processing:

[…] energy is shown in the top left graph of the window infographics that were also used in the original post – it is -4.26dB for the Hann […]