Essentials of Time Series for Financial Applications

Essentials of Time Series for Financial Applications

Massimo Guidolin

Professor of Finance, Bocconi University and Research Fellow, BAFFI-CAREFIN Centre

Manuela Pedio

Teaching Fellow, Bocconi University and Fellow, BAFFI-CAREFIN Centre

Table of Contents

Cover image

Title page

Copyright

List of Figures

List of Tables

Preface

Chapter 1. Linear Regression Model

Abstract

1.1 Inference in Linear Regression Models

1.2 Testing for Violations of the Linear Regression Framework

1.3 Specifying the Regressors

1.4 Issues With Heteroskedasticity and Autocorrelation of the Errors

1.5 The Interpretation of Regression Results

References

Appendix 1.A

Appendix 1.B Principal Component Analysis

Chapter 2. Autoregressive Moving Average (ARMA) Models and Their Practical Applications

Abstract

2.1 Essential Concepts in Time Series Analysis

2.2 Moving Average and Autoregressive Processes

2.3 Selection and Estimation of AR, MA, and ARMA Models

2.4 Forecasting ARMA Processes

References

Appendix 2.A

Chapter 3. Vector Autoregressive Moving Average (VARMA) Models

Abstract

3.1 Foundations of Multivariate Time Series Analysis

3.2 Introduction to Vector Autoregressive Analysis

3.3 Structural Analysis With Vector Autoregressive Models

3.4 Vector Moving Average and Vector Autoregressive Moving Average Models

References

Chapter 4. Unit Roots and Cointegration

Abstract

4.1 Defining Unit Root Processes

4.2 The Spurious Regression Problem

4.3 Unit Root Tests

4.4 Cointegration and Error-Correction Models

References

Chapter 5. Single-Factor Conditionally Heteroskedastic Models, ARCH and GARCH

Abstract

5.1 Stylized Facts and Preliminaries

5.2 Simple Univariate Parametric Models

5.3 Advanced Univariate Volatility Modeling

5.4 Testing for ARCH

5.5 Forecasting With GARCH Models

5.6 Estimation of and Inference on GARCH Models

References

Appendix 5.A Nonparametric Kernel Density Estimation

Chapter 6. Multivariate GARCH and Conditional Correlation Models

Abstract

6.1 Introduction and Preliminaries

6.2 Simple Models of Covariance Prediction

6.3 Full, Multivariate GARCH Models

6.4 Constant and Dynamic Conditional Correlation Models

6.5 Factor GARCH Models

6.6 Inference and Model Specification

References

Chapter 7. Multifactor Heteroskedastic Models, Stochastic Volatility

Abstract

7.1 A Primer on the Kalman Filter

7.2 Simple Stochastic Volatility Models and their Estimation Using the Kalman Filter

7.3 Extended, Second-Generation Stochastic Volatility Models

7.4 GARCH versus Stochastic Volatility: Which One?

References

Chapter 8. Models With Breaks, Recurrent Regime Switching, and Nonlinearities

Abstract

8.1 A Primer on the Key Features and Classification of Statistical Model of Instability

8.2 Detecting and Exploiting Structural Change in Linear Models

8.3 Threshold and Smooth Transition Regime Switching Models

References

Chapter 9. Markov Switching Models

Abstract

9.1 Definitions and Classifications

9.2 Understanding Markov Switching Dynamics Through Simulations

9.3 Markov Switching Regressions

9.4 Markov Chain Processes and Their Properties

9.5 Estimation and Inference for Markov Switching Models

9.6 Forecasting With Markov Switching Models

9.7 Markov Switching ARCH and DCC Models

9.8 Do Nonlinear and Markov Switching Models Work in Practice?

References

Appendix 9.A Some Notions Concerning Ergodic Markov Chains

Appendix 9.B State-Space Representation of an Markov Switching Model

Appendix 9.C First-Order Conditions for Maximum Likelihood Estimation of Markov Switching Models

Chapter 10. Realized Volatility and Covariance

Abstract

10.1 Measuring Realized Variance

10.2 Forecasting Realized Variance

10.3 Multivariate Applications

References

Appendix A. Mathematical and Statistical Appendix

A Fundamental Statistical Definitions

B Matrix Algebra

C Uncorrelatedness and Independence

D Bootstrapping

Index

Copyright

Academic Press is an imprint of Elsevier

125 London Wall, London EC2Y 5AS, United Kingdom

525 B Street, Suite 1650, San Diego, CA 92101-4495, United States

50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States

The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom

Copyright © 2018 Elsevier Inc. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

ISBN: 978-0-12-813409-2

For Information on all Academic Press publications visit our website at https://www.elsevier.com/books-and-journals

Publisher: Candice Janco

Acquisition Editor: Scott Bentley

Editorial Project Manager: Jaclyn Truesdell

Production Project Manager: Anusha Sambamoorthy

Cover Designer: Mark Rogers

Typeset by MPS Limited, Chennai, India

List of Figures

List of Tables

Preface

This textbook is the result of more than 10 years spent in the classroom, teaching MSc-level financial econometrics to a very diverse crowd of students, at Bocconi University, Italy and University of Manchester, United Kingdom. During these years, one of the authors has managed to change side of the classroom and blossom in a well-liked instructor. The other author proudly claims to be responsible of such an awesome shift of her role. In any event, we have both retained and developed over time a keen awareness of the fact that similarly to Stephen Hawking’s famous quote—and contrary to what students and colleagues may think before approaching applied statistics—also in econometrics there is no unique picture of reality. As a result, in Essentials of Time Series for Financial Applications we have wholeheartedly applied this principle and—intentionally and unintentionally—presented a range of models and inferential techniques that just aim at providing different and concurring answers to three basic questions: Can we forecast the conditional mean of one or more time series? Can we forecast the conditional variances and covariances of the same series? What are the trade-offs between these two key applications and approaches? As simple as these questions may sound, the book witnesses the existence of a wide variety of methods and techniques and therefore a range of potential answers. These all represents different, equally valid and faithful pictures of the same reality, the data—in our case, financial time series data.

In our treatment, this feature is greatly emphasized by the constant interplay between theoretical definitions and results and a large number of fully developed examples that are used to either illustrate the methodologies at work or to openly discuss of their limitations in actual cases. In fact, additional examples that could not find space in the book have been available in the book’s website (at http://essentialoftimeseries.wordpress.com), where also the data and EViews worksheets are made available to a Reader. In fact, practitioners and researchers in empirical finance approaching Essentials of Time Series for Financial Applications may even find it productive to work through the book backwards, i.e., past some introductory notions and definitions in each chapter, to focus first on the examples to then better understand the theoretical concepts.

The book has a rather linear structure and goes from simple and introductory topics, such as Chapter 1 on the linear regression model and Chapter 2 on ARMA models, to more advanced but still MSc-level topics, such as Vector autoregressions in Chapter 3 and ARCH models in Chapter 5, to more advanced topics in multivariate (like in Chapter 6 on conditional heteroskedastic models) and nonlinear (like in Chapters 8 and 9, on threshold and Markov switching models with regimes) econometrics analysis. The latter set of topics and chapters may find use in advanced, elective MSc courses in financial econometrics or even as a background Reader in a first-year doctoral course in the econometrics of finance. However, the book is sufficiently rich and structured in a modular fashion to allow a use that could be either mostly focused on models of the conditional mean under stationary linear processes (Chapters 2 on ARMA and 3 on VARMA), followed by cointegrated vector error correction models (Chapter 4), and finally by nonlinear regime switching models (Chapters 8 and 9). Alternatively, one can imagine a pedagogical path focused instead on modeling and forecasting conditional variances and covariances, and more generally conditionally higher-order moments, and therefore centered on Chapters 5 (on ARCH), 6 (on multivariate GARCH and dynamic conditional correlation approaches), 7 (on stochastic volatility), 9 (on Markov switching variances and correlations), and 10 (on realized variance). Both these keys to potentially read and present the material in Essentials of Time Series for Financial Applications are strengthened by a number of additional concept boxes including additional material and explanations that, for pure reasons of space, could not be included but that are available at the book’s website (at http://essentialoftimeseries.wordpress.com).

The book developed out of our lectures, tutorials, and thesis supervisions. We are grateful first and foremost to many students over the years whose questions, detailed error-hunting, and comments have shaped the course of the manuscript and helped us to put in greater focus the trickiest points. We are grateful to Elena De Gioannis, Serena De Lorenzi, and Valentina Massagli for their precious work as research assistants and to Beatrice Franzolini for her help with Chapter 1 that reflects portion of her background research and notes. We also thank Federico Mainardi for volunteering to proofread a large portion of the book. All errors remain our own of course. As for Bocconi’s unparalleled research and teaching environment, our praise cannot escape quoting Frederik Nietsche’s from his 1889 Twilight of the Idols, or, How to Philosophize with a Hammer: Was dich nicht umbringt, macht dich starker.

Chapter 1

Linear Regression Model

Abstract

This chapter introduces the linear regression model used in applied time series analysis to investigate relations among variables. In Section 1.1, the basic tools and assumptions underlying the model are presented; then the chapter shows how to derive point estimates of the parameters using three possible estimation methods, that is, ordinary least square, generalized least squares, and maximum likelihood; moreover, a range of methods to build tests of hypotheses on the parameters are developed. In Section 1.2, the chapter explains how to deal with violations of the hypotheses of the linear regression framework. Section 1.3 deals with the selection of the appropriate regressors and presents some of the issues related with the process of model specification, such as multicollinearity and errors in the measurement of the regressors. Section 1.4 explains how to address potential violations of two initial assumptions on the regression errors of the classical linear regression model, such as heteroskedasticity and autocorrelation. Finally, Section 1.5 provides some intuition on how to interpret the results from an estimated regression.

Keywords

Linear regression; estimation methods; ordinary least squares; generalized least squares; maximum likelihood; tests of hypotheses; confidence intervals; model specification

The roots of education are bitter, but the fruit is sweet.

Aristotle

, called the dependent variable (or regressand), and by one or more explanatory variables , also called independent variables (or regressors). The aim of a regression model (when there is little ambiguity, we shall drop the reference to its linear nature) is to explain (or predict) the values assumed by the dependent variable exploiting the information summarized by the values taken by the regressors.

Despite the arguable simplicity of the model, it is crucial to consider which underlying hypotheses are routinely made to make inference possible and to appreciate the different properties of alternative inferential approaches. In fact, there is little doubt that a superficial use of the model may often lead to misguided or at least unreliable conclusions.

Therefore, in this chapter, we present the basic structure of a classical regression model and a range of suitable estimation methods along with their properties. In addition, in order to retain a strong applied orientation, we also provide some pointers as to how to safely interpret the results of an analysis based on the estimation of linear regression models.

1.1 Inference in Linear Regression Models

Common statements reported by average users of econometric modeling strategies are that they would like to understand from the observed data, say, daily returns on a stock portfolio, from where they came or better how they have been generated. Such a language is not completely random, because also in a technical sense, when we collect and observe samples of data, we interpret their values as realizations of an unobservable data generating process (DGP), which is the object we are really interested in (the population). In simpler words, the underlying, albeit often implicit, assumption is that there exists a stochastic process, which is a sequence (or collection) of random variables from which the data came, so that every sample value that we observe simply represents a single realization of a random variable. Therefore, we cannot directly observe the random variables themselves but—for each of them—only a number of their realizations (often only one). For instance, when it comes to the closing S&P 500 index price on August 4, 2017, we can collect (at most, assuming the market has not been halted or closed out of extraordinary circumstances, as it occasionally happens) one observation: we have no way to rewind history or to ponder different realizations of alternative histories. However, if we consider the random variable defined as the price of the first security traded at market open on every first day of the week, then we have multiple realizations for the same random variable.

Our goal is therefore to investigate the nature of the DGP, that is, to capture the features of the (collection or sequence of) random variables from which the data were generated. To perform this task, first, some assumptions on the DGP have to be cautiously introduced (i.e., we say that we select a statistical model); second, a sample of data—often a simple random sample made of independent and identical random variables, although other, more complex circumstances may also be addressed—must be collected to estimate the parameters in the model using inferential procedures.

over a sampling period of length T.¹ A linear regression model is then:

(1.1)

-dimensional vector of parameters called regression coefficients . Note that , the model is called the classical linear regression model.

To make Eq. (1.1) more transparent, we can rewrite in its extended, algebraic form to provide the following system of equations:

(1.2)

. As shown in can be expressed as a linear combination of the observable values assumed by the explanatory variables plus an unobserved, zero-mean error term.

Example 1.1

Suppose that you have collected daily returns on a stock portfolio over the two most recent months. A way to perform a quantitative analysis of the performance of this portfolio is to estimate a regression that explains stock returns using the returns of the market portfolio. This is the well-known market model: a simple statistical regression in which the dependent variable is the return of a security of interest and there is only one explanatory variable, the return on the market portfolio. The assumption behind the market model is that the performance of any security can be additively factored as a component related to the movement of the market and a residual portion, uniquely related to the behavior or properties of the specific firm (sometimes called idiosyncratic risk).

. Clearly, while stating the basic equation of the model in Eq. (1.1), we are already making specific (in fact, restrictive) assumptions listed below over the functional form of the process.

• Linearityis just a linear combination of the values of the regressors.

• Zero-mean errors. Thus, the effects of the unobservable or omitted factors can be either positive or negative for a single observation, but their expected value is 0.

• Homoskedasticity.

• Zero serial correlation.

• Normally distributed errors: The errors are normally distributed. This assumption is needed mostly to perform hypothesis testing and to compute confidence intervals in small samples but, as we shall see later in this chapter, this can be generalized or even removed, at the cost of either introducing alternative assumptions or resorting to large-sample (only asymptotically valid) results.

Finally, we also introduce the (rather implausible) assumption below.

• Correctly selected regressors collects all the relevant regressors, which means that we are neither forgetting any relevant observable regressor that contributes to explain the dependent variable nor including any redundant regressor.

, they are not a realization of a random variable. However, this restriction shall be removed in Section 1.1.6.

From (we also call it scalar matrix):

(1.3)

), we are able to compute the estimates of the dependent variable resulting from the model in Eq. (1.1), the so-called fitted valuesthe T-dimensional vector that collects all the fitted values at time t. Using the estimated fitted values, it is also possible to compute the sample counterpart of the error term, the vector of residuals. Based on this framework, we are now able to introduce a few key inference tools.

1.1.1 The Ordinary Least Squares Estimator

The most widely used estimation method applied to a regression is the ordinary least squares (OLS) :

(1.4)

The logic is that, if you accept to measure the distance between the sample values of the dependent variable and the fitted values produced by the model with their squared difference, then to be compensated by positive ones), it is symmetric (i.e., it does not favor any sign), and it is everywhere differentiable, what we sometimes refer to as being a smooth function, the OLS estimator also minimizes the sum of the squared residuals,

(1.5)

:

(1.6)

(also called the gradientto a vector of zeros:

(1.7)

It follows that the OLS estimator is:

(1.8)

presents the following properties:

• It is linearof the values of the dependent variable.

• It is unbiased, meaning that its expected value equals the true (and yet unknown) parameter, that is,

(1.9)

, that is,

(1.10)

• It is normally distributed, being a linear combination of the random variables that compose the random sample on the dependent variable, each of which is normally distributed, that is,

(1.11)

• It is the best linear unbiased estimator (BLUE)has the minimum variance; this last property is usually referred to as the Gauss–Markov theoremis the (uniformly) minimum variance estimator within the linear class, that is, the simplest possible class.

Result 1.1

(Gauss–Markov theorem)

If in the linear regression model is BLUE.

Proof and then in comparing this result with the covariance matrix of the OLS estimator.

holds, so that:

(1.12)

To satisfy . Putting these facts together, we have:

(1.13)

, then it also follows that

Along with these important properties of the OLS estimator, the least squares procedure ensures also three additional, desirable algebraic properties of the sample residuals, provided that the model has an intercept:

.

.

.

To prove these properties is fairly easy as it only requires algebraic manipulations, and therefore this is left as an exercise for the Reader.

as a sum of the correspondent fitted value and the OLS residual:

(1.14)

It is straightforward to prove that is equal to the sample average of the fitted values. These results will be soon very important when we will define and measure the goodness of fit of a model.

, as the function minimized according to the OLS criterion in Eq. (1.4) does not depend on it. The OLS estimation of the residual variance parameter is simply

(1.15)

is characterized by the following, useful properties:

is unbiased, that is,

(1.16)

.

.

• It is a quadratic form that involves normally distributed variables and as a result:

(1.17)

Notably, the OLS method does not require the introduction of any assumption on the distribution of the error term. In particular, we have not used the normal distribution hypothesis to derive either the estimator in Eq. (1.8) or the one in Eq. (1.15). In other words, the normality assumption only helped us to characterize the distribution of the estimators, not their identity or formulation.

1.1.2 Goodness of Fit Measures

. As already emphasized by the model can explain, we use as a measure of variability the sum of the squared deviations from the mean, it is called total sum of squares (SST):

(1.18)

When we compute the same measure on the fitted values and the residuals, we obtain the explained sum of squares (SSE) and the residual sum of squares (SSR), which are defined respectively as:

(1.19)

(1.20)

:

(1.21)

can be explained through its linear relation with the regressors, we can compute the R² (also called coefficient of determination) of the model, which is defined as:

(1.22)

The ratio in can only take values between 0 and 1. The higher the Rdoes not provide a reliable index to compare models built on a different number of explanatory variables. To overcome this limitation, it is common to apply a correction to Eq. (1.22) to obtain the adjusted R²:

(1.23)

The adjusted R² varies between 0 and 1, but it can no longer be interpreted as the percentage of explained variability to the adjusted Rmeasure so that it declines when weak, almost-irrelevant regressors are added. Of course, the penalization applied to the R² is based on a sensible but arbitrary functional modification. In Chapter 2 we shall introduce additional indices of goodness fit that are based on similar principles. Eq. (1.23) is then commonly used to compare the goodness of fit of alternative models, based on a different number of regressors.

Example 1.2

Table 1.1 shows the results of OLS estimation of a linear regression model, in which the dependent variable is the daily excess return on a portfolio composted by stocks of US firms operating in the food industry and where the regressors are the so-called Fama–French factors, which are supposed to pick up systematic influences of general aggregate economic conditions on the equity prices and hence returns. The returns are in excess of the daily return on the 1-month Treasury Bill. In the table, the regressors are labeled as Excess MKT, SMB (Small Minus Big), and HML (High Minus Low).

Table 1.1

The OLS estimates of the regression coefficients are reported in the column Coefficient, meaning that the estimated regression is:

is the excess return on the food portfolio at time tis the excess return on the market portfolio at time tis the return of a portfolio long on firms with high book-to-market value and short on firms with low book-to-market value.

The R² of the model is 0.672, meaning that the three Fama–French factors explain roughly 67% of the variability of the excess returns on food portfolio, while the remaining 33% is captured by the error terms. The adjusted R² is almost identical, and, in any case, it would be useful only to compare alternative linear models. In Table 1.1, the statistics reported in the column S.E. happens to be estimated at 0.453 in this application. The output also informs us that:

Indeed, it turns out that 102.579/312.96 gives the 32.8% fraction of total variation that is left unexplained by the three regressors assumed here. Other portions of this output will be explained as we move on with our presentation of the material.

1.1.3 The Generalized Least Squared Estimator

Up to this point, we have always assumed that the variance–covariance matrix of the error terms was spherical, that is, a diagonal matrix with all the elements on the main diagonal identical, also known as a scalar matrix. Although in many empirical applications this is a plausible assumption, OLS estimation method is impractical in many other situations in which we would have reasons—either theoretical or empirical, on the basis of the behavior of the data—to believe that autocorrelation and/or heteroskedasticity (which means that the variances and covariances of the error terms are not constant over time and across different observations) in the error terms may exist. To start taking these aspects into consideration, we now extend the model to the case in which the covariance matrix is some known general matrix. Later, we shall further extend the resulting estimation tools to the case of unknown covariance matrix. Moreover, Chapters 5 and 6, will be devoted to models of conditional heteroskedasticity.

, the (classical) linear regression model becomes:

(1.24)

, will fail to display all the properties that we have listed and proven in would still be unbiased, but it will not have minimum variance. Indeed, under Eq. (1.24), the BLUE estimator is no longer the OLS, but instead the so-called generalized least squared (GLS) estimator, which we are now going to derive.

is an orthogonal matrix(see on both sides of the model in Eq. (1.24), we obtain:

(1.25)

, we have:

(1.26)

, are now Gaussian independently and identically distributed (IID) with identity covariance matrix because:

(1.27)

The model in , it is sufficient to apply the OLS estimation procedure to the model in Eq. (1.26), deriving:

(1.28)

To re-express this estimator as a function of the variables originally appearing in the generalized model is known) GLS estimator:

(1.29)

In summary, the estimator in is left unchanged after the transformation.

is linear, normally distributed and unbiased:

(1.30)

is just the OLS estimator for Eq. (1.26).

However, the GLS estimator in Eq. (1.29) is often called the unfeasible , and we only entertain a series of doubts concerning the hypotheses of homoskedasticity and zero-autocorrelation. To solve this problem, it is usually advised to perform a few tests to assess whether the suspicion of heteroskedasticity and/or autocorrelation of the errors finds any empirical backing and, if needed, to proceed to:

• either adopt feasible versions of and a clear understanding of whether and how the resulting feasible GLS estimator may be valid in practice, being at least unbiased and consistent, given that such a two- or multi-step estimator is by construction unlikely to be the most efficient one;

.

We shall return to these options with more details later, in Section 1.4 of this chapter, when we will consider the role played by violations of the hypotheses concerning the errors in a more practical perspective.

1.1.4 Maximum Likelihood Estimator

is available; or, alternatively, that a researcher may be ready to impose such an assumption. When this is sensible, then an additional estimation method—in fact, one guaranteed to return the uniformly most efficient estimator—becomes available, the maximum likelihood estimator (MLE).

).

is given by:

(1.31)

For the purpose of ML estimation, we shall consider , obtaining the so-called likelihood function, and find its maximum with respect to exactly those parameters. The likelihood function is therefore defined as:

(1.32)

no longer represents a random variable of which we care of the joint density function as in being the variables with respect to which the maximization needs to be performed. To simplify the calculations, it is common to perform the maximization not of Eq. (1.32) as such but of the logarithm of the likelihood function (called log-likelihood function):

(1.33)

, to find a maximum we now compute the score function,

(1.34)

and equate it to 0 to obtain a system of necessary first-order conditions (FOCs):

(1.35)

unknowns, we obtain:

(1.36)

(1.37)

The result in is biased:

(1.38)

could be biased, one can prove that MLE always presents the following properties:²

• Invariancesatisfies a few conditions (such as being one-to-one, or being at least continuous).

• Consistency: As the sample size T increases (technically, over a sequence of random samples of increasing size T.

• Asymptotic normality: As the sample size T is the asymptotic covariance matrix.

• Asymptotic efficiencyis equal to the Cramér–Rao lower variance bound, and therefore the MLE is the uniformly most efficient estimator among the ones that asymptotically are characterized by a zero bias.³

1.1.5 Hypotheses Testing, Confidence Intervals, and Predictive Intervals

In Sections 1.2.1–1.2.3, we have shown how to derive point estimates of the parameters in the linear regression model, under a variety of different assumptions. However, in empirical analyses, it is often crucial to combine point estimates of coefficients with some indicators of the associated degree of confidence in this very point estimates, that is, how reliable and accurate the point estimate we have found may be. To this purpose, it is typical to compute confidence intervals, which are intervals for the unknown parameters, to which it is possible to associate a probability such that the true but unknown parameter value falls in the interval defined around some point estimate with the stated confidence; this can also be interpreted as frequency of the intervals including the true parameters over a long sequence of samples of identical size T.⁴ To fully grasp what follows, let us remind ourselves that a point estimator is nothing but a random variable (with an expected value and a variance). When we compute a point estimated value, we substitute the observed values of the dependent and explanatory variables in the appropriate formula (e.g., Eq. 1.36) and obtain a value that is however just one specific realization of the random variable.

and therefore:

(1.39)

. At this point, from Eq. (1.39) it follows that:

(1.40)

This formula defines a (1−α, indeed we can state that the true parameter belongs to

with a confidence of 1−α.⁵

Note that it is in principle necessary to know the true value of σ to compute the lower and upper bounds, and once more this appears odd, that while K+1 parameters