🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:

Response:

Calculated Shard: 126 (from laksa036)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa126.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties\')), getAhrefsUnparsedNoserviceFromURL(\'https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties\'))) FORMAT JSONEachRow'

Response:

{"found_url":"https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties","crawl_time":1775011611,"first_indexed_time":1462434386,"http_code":200,"src_unparsed":"com,statlect!www,\/fundamentals-of-statistics\/OLS-estimator-properties s443","src_root_hash":"3586688910177265926","history_drop_reason":null,"meta_title":"Properties of the OLS estimator | Consistency, asymptotic normality","meta_descriptions":["Learn what conditions are needed to prove the consistency and asymptotic normality of the OLS estimator."],"attrs_boilerpipe_text":"In this lecture we discuss under which assumptions the OLS (Ordinary Least\r\n   Squares) estimator has desirable statistical properties such as consistency\r\n   and asymptotic normality.\nTable of contents\nThe regression model\nMatrix notation\nThe estimator\nWriting the estimator in terms of sample means\nConsistency of the OLS estimator\nAssumption 1 - Convergence of sample means to population means\nAssumption 2 - Full rank\nAssumption 3 - Orthogonality\nProof of consistency\nAsymptotic normality of the OLS estimator\nAssumption 4 - CLT condition\nProof of asymptotic normality\nConsistent estimation of the variance of the error terms\nAssumption 5 - Regularity of error terms\nProof of consistency\nConsistent estimation of the asymptotic covariance matrix\nProof of consistency\nConsistent estimation of the long-run covariance matrix\nAssumption 6 - No serial correlation\nMore explicit formulae for the long-run covariance\nProof of consistency under Assumption 6\nFormula for the covariance matrix of the OLS estimator under Assumption 6\nAssumption 7 - Conditional homoskedasticity\nProof of consistency under Assumption 7\nFormula for the covariance matrix of the OLS estimator under Assumptions 6 and 7\nWeaker assumptions\nHypothesis testing\nReferences\nThe regression model\nConsider the\nlinear\r\n   regression\nmodel\nwhere:\nthe outputs are denoted by\n;\nthe associated\nvectors of inputs are denoted by\n;\nthe\nvector of regression coefficients is denoted by\n;\nare unobservable error terms.\nMatrix notation\nWe assume to observe a sample of\nrealizations, so that the vector of all outputs\nis\r\n   an\nvector, the\ndesign\r\n   matrix\nis\r\n   an\nmatrix, and the vector of error\r\n   terms\nis\r\n   an\nvector.\nThe estimator\nThe OLS estimator\nis the vector of regression coefficients that minimizes the sum of squared\r\n   residuals:\nAs proved in the lecture on\nLinear\r\n   regression\n, if the design matrix\nhas full rank, then the OLS estimator is computed as\r\n   follows:\nWriting the estimator in terms of sample means\nThe OLS estimator can be written as\nwhere\nis\r\n   the\nsample mean\nof the\nmatrix\nand\nis\r\n   the sample mean of the\nmatrix\n.\nConsistency of the OLS estimator\nIn this section we are going to propose a set of conditions that are\r\n   sufficient for the\nconsistency\nof the OLS estimator, that is, for the convergence in probability of\nto the true value\n.\nAssumption 1 - Convergence of sample means to\r\npopulation means\nThe first assumption we make is that the sample means in the OLS formula\r\n   converge to their population counterparts, which is formalized as follows.\nAssumption 1 (convergence)\n: both the sequence\nand the sequence\nsatisfy sets of conditions that are sufficient for the\nconvergence in probability\nof their sample means\r\n   to the population means\nand\n,\r\n   which do not depend on\n.\nFor example, the sequences\nand\ncould be assumed to satisfy the conditions of\nChebyshev's Weak Law of Large Numbers for\r\n   correlated sequences\n, which are quite mild (basically, it is only required\r\n   that the sequences are\ncovariance stationary\nand\r\n   that their auto-covariances are zero on average).\nAssumption 2 - Full rank\nThe second assumption we make is a rank assumption (sometimes also called\r\n   identification assumption).\nAssumption 2 (rank)\n: the square matrix\nhas\nfull rank\n(as a\r\n   consequence, it is\ninvertible\n).\nAssumption 3 - Orthogonality\nThe third assumption we make is that the regressors\nare orthogonal to the error terms\n.\nAssumption 3 (orthogonality)\n: For each\n,\nand\nare orthogonal, that\r\n   is,\nProof of consistency\nIt is then straightforward to prove the following proposition.\nProposition\nIf Assumptions 1, 2 and 3 are satisfied, then the OLS estimator\nis a consistent estimator of\n.\nProof\nAsymptotic normality of the OLS estimator\nWe now introduce a new assumption, and we use it to prove the asymptotic\r\n   normality of the OLS estimator.\nAssumption 4 - CLT condition\nThe assumption is as follows.\nAssumption 4 (Central Limit Theorem)\n: the sequence\nsatisfies a set of conditions that are sufficient to guarantee that a Central\r\n   Limit Theorem applies to its sample\r\n   mean\nFor a review of some of the conditions that can be imposed on a sequence to\r\n   guarantee that a Central Limit Theorem applies to its sample mean, you can go\r\n   to the lecture on the\nCentral Limit\r\n   Theorem\n.\nIn any case, remember that if a Central Limit Theorem applies to\n,\r\n   then, as\ntends to\r\n   infinity,\nconverges\r\n   in distribution\nto a\nmultivariate normal\r\n   distribution\nwith mean equal to\nand covariance matrix equal\r\n   to\nProof of asymptotic normality\nWith Assumption 4 in place, we are now able to prove the asymptotic normality\r\n   of the OLS estimator.\nProposition\nIf Assumptions 1, 2, 3 and 4 are satisfied, then the OLS estimator\nis asymptotically multivariate normal with mean equal to\nand asymptotic covariance matrix equal\r\n      to\nthat\r\n      is,\nwhere\nhas been defined above.\nProof\nAs in the proof of consistency, the\r\n   dependence of the estimator on the sample size is made explicit, so that the\r\n   OLS estimator is denoted by\n.\r\n   First of all, we have\nwhere,\r\n   in the last step, we have used the fact that, by Assumption 3,\n.\r\n   Note that, by Assumption 1 and the Continuous Mapping theorem, we\r\n   have\nFurthermore,\r\n   by Assumption 4, we have\r\n   that\nconverges\r\n   in distribution to a multivariate normal random vector having mean equal to\nand covariance matrix equal to\n.\r\n   Thus, by Slutski's theorem, we have\r\n   that\nconverges\r\n   in distribution to a multivariate normal vector with mean equal to\nand covariance matrix equal to\nConsistent estimation of the variance of the error terms\nWe now discuss the consistent estimation of the variance of the error terms.\nAssumption 5 - Regularity of error terms\nHere is an additional assumption.\nAssumption 5\n: the sequence\nsatisfies a set of conditions that are sufficient for the convergence in\r\n   probability of its sample\r\n   mean\nto\r\n   the population mean\nwhich\r\n   does not depend on\n.\nProof of consistency\nIf this assumption is satisfied, then the variance of the error terms\ncan be estimated by the sample variance of the\r\n   residuals\nwhere\nProposition\nUnder Assumptions 1, 2, 3, and 5, it can be proved that\nis a consistent estimator of\n.\nProof\nConsistent estimation of the asymptotic covariance matrix\nWe have proved that the asymptotic covariance matrix of the OLS estimator\r\n   is\nwhere\r\n   the long-run covariance matrix\nis defined\r\n   by\nUsually, the matrix\nneeds to be estimated because it depends on quantities\r\n   (\nand\n)\r\n   that are not known.\nProof of consistency\nThe next proposition characterizes consistent estimators of\n.\nProposition\nIf Assumptions 1, 2, 3, 4 and 5 are satisfied, and a consistent estimator\nof the long-run covariance matrix\nis available, then the asymptotic variance of the OLS estimator is\r\n      consistently estimated\r\n      by\nProof\nThus, in order to derive a consistent estimator of the covariance matrix of\r\n   the OLS estimator, we need to find a consistent estimator of the long-run\r\n   covariance matrix\n.\r\n   How to do this is discussed in the next section.\nConsistent estimation of the long-run covariance matrix\nThe estimation of\nrequires some assumptions on the covariances between the terms of the sequence\n.\nAssumption 6 - No serial correlation\nIn order to find a simpler expression for\n,\r\n   we make the following assumption.\nAssumption 6\n: the sequence\nis\nserially\r\n   uncorrelated\n, that\r\n   is,\nand\nweakly stationary\n, that is,\ndoes\r\n   not depend on\n.\nRemember that in Assumption 3 (orthogonality) we also ask\r\n   that\nMore explicit formulae for the long-run covariance\nWe now derive simpler expressions for\n.\nProposition\nUnder Assumptions 3 (orthogonality), the long-run covariance matrix\nsatisfies\nProof\nThis is proved as\r\n   follows:\nProposition\nUnder Assumptions 3 (orthogonality) and 6 (no serial correlation), the\r\n      long-run covariance matrix\nsatisfies\nProof\nThe proof is as\r\n   follows:\nProof of consistency under Assumption 6\nThanks to assumption 6, we can also derive an estimator of\n.\nProposition\nSuppose that Assumptions 1, 2, 3, 4 and 6 are satisfied, and that\nis consistently estimated by the sample\r\n      mean\nThen,\r\n      the long-run covariance matrix\nis consistently estimated\r\n      by\nProof\nFormula for the covariance matrix of the OLS\r\nestimator under Assumption 6\nWhen the assumptions of the previous proposition hold, the asymptotic\r\n   covariance matrix of the OLS estimator\r\n   is\nAs a consequence, the covariance of the OLS estimator can be approximated\r\n   by\nwhich\r\n   is known as\nheteroskedasticity-robust\r\n   estimator\n.\nAssumption 7 - Conditional homoskedasticity\nA further assumption is often made, which allows us to further simplify the\r\n   expression for the long-run covariance matrix.\nAssumption 7\n: the error terms are\nconditionally\r\n   homoskedastic\n:\nProof of consistency under Assumption 7\nThis assumption has the following implication.\nProposition\nSuppose that Assumptions 1, 2, 3, 4, 5, 6 and 7 are satisfied. Then, the\r\n      long-run covariance matrix\nis consistently estimated\r\n      by\nProof\nFormula for the covariance matrix of the OLS\r\nestimator under Assumptions 6 and 7\nWhen the assumptions of the previous proposition hold, the asymptotic\r\n   covariance matrix of the OLS estimator\r\n   is\nAs a consequence, the covariance of the OLS estimator can be approximated\r\n   by\nwhich\r\n   is the same estimator derived in the\nnormal\r\n   linear regression model\n.\nWeaker assumptions\nThe assumptions above can be made even weaker (for example, by relaxing the\r\n   hypothesis that\nis uncorrelated with\n),\r\n   at the cost of facing more difficulties in estimating the long-run covariance\r\n   matrix.\nFor a review of the methods that can be used to estimate\n,\r\n   see, for example, Den and Levin (1996).\nHypothesis testing\nThe lecture entitled\nLinear\r\n   regression - Hypothesis testing\ndiscusses how to carry out\nhypothesis tests\non the coefficients of a linear regression model in the cases discussed above,\r\n   that is, when the OLS estimator is asymptotically normal and a consistent\r\n   estimator of the asymptotic covariance matrix is available.\nReferences\nHaan, Wouter J. Den, and Andrew T. Levin (1996). \"Inferences from parametric\r\n   and non-parametric covariance matrix estimation procedures.\" Technical Working\r\n   Paper Series, NBER.\nHow to cite\nPlease cite as:\nTaboga, Marco (2021). \"Properties of the OLS estimator\", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties.","attrs_markdown":"![Search for probability and statistics terms on Statlect](data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAABkAAAAXCAMAAADJPRQhAAACXlBMVEX\/\/\/8zZpkyZZktY5guY5gxZZk0Z5kyZpkyZZgjWZAtYpchXJQcVo8tY5f\/\/PgiWpE2XIkvY5ctZJkwZZksY5gmXpUlXJM0ZpklXZQkXJOpssJNbpbt5+csYpcvZJcmXZP79vQ6Y5H59fQjWpImXJItXpLX2N1OcpqUp78kV4wXUYuDk6saVI11h6NNbpUdVo8eV4+hrsFEbJf\/\/\/7i4+d4kK6PrMmWpbgZVI6ToLW0vcoiU4cmVon69\/fEx88YT4clWIxAa5oZUotAYo3Pz9Tt7O2vs8AZVY8nVYghWZExZZjt7fAHQn81Z5okXJSBlrAzXIovZJkoU4TEydMZVZAfVIrJ0NubrMAjWZEkWpD78OyuusseV5AeVIsrYJTe5e2ywdMjWpDu8fS2wtImWo\/T2+Oqucqrucrx7Ovd2dsSUY20w9MdVIyxu8ppgqMoX5UjV4wQQ3szZprn5uk\/ZpItZJgTTIcpYJZTdJpshqYcVo4iWpJ\/katBZ5McU4q9xM\/28\/ImVooWUo5whKHn4uQ6Yo8mX5ZLbZX\/+\/k5Y5EvZJglXpYZTIJtiambqr6ouMuQorlNcJkTTYcuZJkrYpgoWIn18vIbToUeWJEpX5UjWpEeU4oxZpovV4f59vXg4ucjVIkuZJonVIa7wMtHa5QUTIYnX5X99vMvY5gsYZYfWZEdUIZxiqjR198nWIuGmbEkUYPd3+LO0tlsiatIcJw9aZg8bZ5AaZZUeqOVorfs7O7Cy9gkV4stZJojT4Ho5+n8+ff7\/P3\/+fYdSn359fNEa5YeS3769\/X\/\/vz69\/Y1HbsLAAABU0lEQVR4Xm3JY5MrURiF0fN2x+bQtm3btm3btm1c2rbxr25V+mTSM5X96dm1EG02tXY+vo1G6PoMAl0trUx1xBGRUVfAwnBADgkMoQkBxg6ONDAz1wOmm7WHbbi9PrCdnDXiogeKIPdEhDy9vA+PwM9fDQECYMQF4xMSCsywTiySbqI4GqkXE6sb\/4RKUVJySirSLC1dnkFVZhZk59AkN4\/IL1BVYRGnpJQmZWJOeYWqKgVQVU2TGi7BrVNVfQOrqZkmLa1EG872DrJLA3d6oLcPd78SBocuZVgBI6O4x8YJmJikempaCTOzCG9ufgF0F5eWV1bX1jc2YWtbhGFnV8jcY7DY+\/oHOjwWm398guH0jDy\/uHHz1m2eVHr33v0HDxHeo8ek7OkzhJ6\/ePnq9Zu3796r4cNH8tPnL1R\/\/fYdaSbhy378RFr36\/cfpH1\/\/2mF\/6EqUMbQOA8lAAAAAElFTkSuQmCC)\n\n[StatLect](https:\/\/www.statlect.com\/)\n\n[Index](https:\/\/www.statlect.com\/) \\> [Fundamentals of statistics](https:\/\/www.statlect.com\/fundamentals-of-statistics\/)\n\n# Properties of the OLS estimator\nby [Marco Taboga](https:\/\/www.statlect.com\/about\/#author), PhD\n\nIn this lecture we discuss under which assumptions the OLS (Ordinary Least Squares) estimator has desirable statistical properties such as consistency and asymptotic normality.\n\n![Infographic providing an overview of the assumptions needed to prove the consistency and asymptotic normality of the OLS estimator.](https:\/\/www.statlect.com\/images\/OLS-properties.png)\n\n![Table of Contents](data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAB0AAAAYCAIAAACJPGHrAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAABJSURBVEhLY\/z\/\/z8DDQATlKY2GGrmgsL3yJEjUB4Dg42NDZBEFiEeQPRCwGj4QsBQS7+j6QECRtMDBIymBwgYTQ8QMLTMZWAAAEw6JxXCQPIuAAAAAElFTkSuQmCC)\n\nTable of contents\n\n1. [The regression model](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid2)\n2. [Matrix notation](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid3)\n3. [The estimator](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid4)\n4. [Writing the estimator in terms of sample means](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid5)\n5. [Consistency of the OLS estimator](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid6)\n   1. [Assumption 1 - Convergence of sample means to population means](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid7)\n   2. [Assumption 2 - Full rank](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid8)\n   3. [Assumption 3 - Orthogonality](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid9)\n   4. [Proof of consistency](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid10)\n6. [Asymptotic normality of the OLS estimator](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid11)\n   1. [Assumption 4 - CLT condition](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid12)\n   2. [Proof of asymptotic normality](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid13)\n7. [Consistent estimation of the variance of the error terms](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid14)\n   1. [Assumption 5 - Regularity of error terms](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid15)\n   2. [Proof of consistency](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid16)\n8. [Consistent estimation of the asymptotic covariance matrix](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid17)\n   1. [Proof of consistency](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid18)\n9. [Consistent estimation of the long-run covariance matrix](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid19)\n   1. [Assumption 6 - No serial correlation](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid20)\n   2. [More explicit formulae for the long-run covariance](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid21)\n   3. [Proof of consistency under Assumption 6](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid22)\n   4. [Formula for the covariance matrix of the OLS estimator under Assumption 6](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid23)\n   5. [Assumption 7 - Conditional homoskedasticity](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid24)\n   6. [Proof of consistency under Assumption 7](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid25)\n   7. [Formula for the covariance matrix of the OLS estimator under Assumptions 6 and 7](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid26)\n   8. [Weaker assumptions](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid27)\n10. [Hypothesis testing](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid28)\n11. [References](https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties#hid29)\n## The regression model\nConsider the [linear regression](https:\/\/www.statlect.com\/fundamentals-of-statistics\/linear-regression) model![\\[eq1\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)where:\n\n- the outputs are denoted by ![\\$y\\_{i}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==);\n- the associated ![\\$1 imes K\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) vectors of inputs are denoted by ![\\$x\\_{i}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==);\n- the ![Kx1](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) vector of regression coefficients is denoted by ![\\$&#8;eta \\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==);\n- ![\\$&#11;arepsilon \\_{i}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) are unobservable error terms.\n## Matrix notation\nWe assume to observe a sample of ![\\$N\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) realizations, so that the vector of all outputs\n\n![\\[eq2\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)is an ![\\$N imes 1\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) vector, the [design matrix](https:\/\/www.statlect.com\/glossary\/design-matrix)![\\[eq3\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)is an ![\\$N imes K\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) matrix, and the vector of error terms![\\[eq4\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)is an ![\\$N imes 1\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) vector.\n\n## The estimator\nThe OLS estimator ![\\$widehat{&#8;eta }\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is the vector of regression coefficients that minimizes the sum of squared residuals:![\\[eq5\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\nAs proved in the lecture on [Linear regression](https:\/\/www.statlect.com\/fundamentals-of-statistics\/linear-regression), if the design matrix ![X](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) has full rank, then the OLS estimator is computed as follows:![\\[eq6\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\n## Writing the estimator in terms of sample means\nThe OLS estimator can be written as ![\\[eq7\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)where ![\\[eq8\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)is the [sample mean](https:\/\/www.statlect.com\/glossary\/sample-mean) of the ![\\$K imes K\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) matrix ![\\$x\\_{i}^{ op }x\\_{i}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\\[eq9\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)is the sample mean of the ![Kx1](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) matrix ![\\$x\\_{i}^{ op }y\\_{i}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==).\n\n## Consistency of the OLS estimator\nIn this section we are going to propose a set of conditions that are sufficient for the [consistency](https:\/\/www.statlect.com\/glossary\/consistent-estimator) of the OLS estimator, that is, for the convergence in probability of ![\\$widehat{&#8;eta }\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) to the true value ![\\$&#8;eta \\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==).\n\n### Assumption 1 - Convergence of sample means to population means\nThe first assumption we make is that the sample means in the OLS formula converge to their population counterparts, which is formalized as follows.\n\n**Assumption 1 (convergence)**: both the sequence ![\\[eq10\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and the sequence ![\\[eq11\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) satisfy sets of conditions that are sufficient for the [convergence in probability](https:\/\/www.statlect.com\/asymptotic-theory\/convergence-in-probability) of their sample means to the population means ![\\[eq12\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\\[eq13\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), which do not depend on ![i](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==).\n\nFor example, the sequences ![\\[eq14\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\\[eq15\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) could be assumed to satisfy the conditions of [Chebyshev's Weak Law of Large Numbers for correlated sequences](https:\/\/www.statlect.com\/asymptotic-theory\/law-of-large-numbers#cheby2), which are quite mild (basically, it is only required that the sequences are [covariance stationary](https:\/\/www.statlect.com\/glossary\/covariance-stationary) and that their auto-covariances are zero on average).\n\n### Assumption 2 - Full rank\nThe second assumption we make is a rank assumption (sometimes also called identification assumption).\n\n**Assumption 2 (rank)**: the square matrix ![\\[eq16\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) has [full rank](https:\/\/www.statlect.com\/matrix-algebra\/rank-of-a-matrix) (as a consequence, it is [invertible](https:\/\/www.statlect.com\/matrix-algebra\/inverse-matrix)).\n\n### Assumption 3 - Orthogonality\nThe third assumption we make is that the regressors ![\\$x\\_{i}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) are orthogonal to the error terms ![\\$&#11;arepsilon \\_{i}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==).\n\n**Assumption 3 (orthogonality)**: For each ![i](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), ![\\$x\\_{i}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\\$&#11;arepsilon \\_{i}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) are orthogonal, that is,![\\[eq17\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\n### Proof of consistency\nIt is then straightforward to prove the following proposition.\n\nProposition If Assumptions 1, 2 and 3 are satisfied, then the OLS estimator ![\\$widehat{&#8;eta }\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is a consistent estimator of ![\\$&#8;eta \\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==).\n\nProof\n\nLet us make explicit the dependence of the estimator on the sample size and denote by ![\\[eq18\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) the OLS estimator obtained when the sample size is equal to ![\\$N.\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) By Assumption 1 and by the [Continuous Mapping theorem](https:\/\/www.statlect.com\/asymptotic-theory\/continuous-mapping-theorem), we have that the probability limit of ![\\[eq19\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is ![\\[eq20\\]](https:\/\/www.statlect.com\/images\/OLS-estimator-properties__47.png)Now, if we pre-multiply the regression equation![\\[eq21\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)by ![\\$x\\_{i}^{ op }\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and we take expected values, we get![\\[eq22\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)But by Assumption 3, it becomes![\\[eq23\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)or![\\[eq24\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)which implies that![\\[eq25\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\n## Asymptotic normality of the OLS estimator\nWe now introduce a new assumption, and we use it to prove the asymptotic normality of the OLS estimator.\n\n### Assumption 4 - CLT condition\nThe assumption is as follows.\n\n**Assumption 4 (Central Limit Theorem)**: the sequence ![\\[eq26\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) satisfies a set of conditions that are sufficient to guarantee that a Central Limit Theorem applies to its sample mean![\\[eq27\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\nFor a review of some of the conditions that can be imposed on a sequence to guarantee that a Central Limit Theorem applies to its sample mean, you can go to the lecture on the [Central Limit Theorem](https:\/\/www.statlect.com\/asymptotic-theory\/central-limit-theorem).\n\nIn any case, remember that if a Central Limit Theorem applies to ![\\[eq26\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), then, as ![\\$N\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) tends to infinity,![\\[eq29\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) [converges in distribution](https:\/\/www.statlect.com\/asymptotic-theory\/convergence-in-distribution) to a [multivariate normal distribution](https:\/\/www.statlect.com\/probability-distributions\/multivariate-normal-distribution) with mean equal to ![0](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and covariance matrix equal to![\\[eq30\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\n### Proof of asymptotic normality\nWith Assumption 4 in place, we are now able to prove the asymptotic normality of the OLS estimator.\n\nProposition If Assumptions 1, 2, 3 and 4 are satisfied, then the OLS estimator ![\\$widehat{&#8;eta }\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is asymptotically multivariate normal with mean equal to ![\\$&#8;eta \\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and asymptotic covariance matrix equal to![\\[eq31\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)that is,![\\[eq32\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)where ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) has been defined above.\n\nProof\n\nAs in the proof of consistency, the dependence of the estimator on the sample size is made explicit, so that the OLS estimator is denoted by ![\\[eq33\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). First of all, we have ![\\[eq34\\]](https:\/\/www.statlect.com\/images\/OLS-estimator-properties__67.png)where, in the last step, we have used the fact that, by Assumption 3, ![\\[eq35\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). Note that, by Assumption 1 and the Continuous Mapping theorem, we have![\\[eq36\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)Furthermore, by Assumption 4, we have that![\\[eq37\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)converges in distribution to a multivariate normal random vector having mean equal to ![0](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and covariance matrix equal to ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). Thus, by Slutski's theorem, we have that![\\[eq38\\]](https:\/\/www.statlect.com\/images\/OLS-estimator-properties__73.png)converges in distribution to a multivariate normal vector with mean equal to ![0](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and covariance matrix equal to ![\\[eq39\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\n## Consistent estimation of the variance of the error terms\nWe now discuss the consistent estimation of the variance of the error terms.\n\n### Assumption 5 - Regularity of error terms\nHere is an additional assumption.\n\n**Assumption 5**: the sequence ![\\[eq40\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) satisfies a set of conditions that are sufficient for the convergence in probability of its sample mean![\\[eq41\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)to the population mean ![\\[eq42\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)which does not depend on ![i](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==).\n\n### Proof of consistency\nIf this assumption is satisfied, then the variance of the error terms ![sigma^2](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) can be estimated by the sample variance of the residuals![\\[eq43\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)where ![\\[eq44\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\nProposition Under Assumptions 1, 2, 3, and 5, it can be proved that ![\\[eq45\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is a consistent estimator of ![sigma^2](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==).\n\nProof\n\nLet us make explicit the dependence of the estimators on the sample size and denote by ![\\[eq46\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\\[eq47\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) the estimators obtained when the sample size is equal to ![\\$N.\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) By Assumption 1 and by the [Continuous Mapping theorem](https:\/\/www.statlect.com\/asymptotic-theory\/continuous-mapping-theorem), we have that the probability limit of ![\\[eq47\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is ![\\[eq49\\]](https:\/\/www.statlect.com\/images\/OLS-estimator-properties__89.png)where: in steps ![\\$ rame{A}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\\$ rame{C}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) we have used the Continuous Mapping Theorem; in step ![\\$ rame{B}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) we have used Assumption 5; in step ![\\$ rame{D}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) we have used the fact that ![\\[eq50\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)because ![\\[eq51\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is a consistent estimator of ![\\$&#8;eta \\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), as proved above.\n\n## Consistent estimation of the asymptotic covariance matrix\nWe have proved that the asymptotic covariance matrix of the OLS estimator is![\\[eq52\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)where the long-run covariance matrix ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is defined by![\\[eq53\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\nUsually, the matrix ![\\[eq54\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) needs to be estimated because it depends on quantities (![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\\[eq16\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)) that are not known.\n\n### Proof of consistency\nThe next proposition characterizes consistent estimators of ![\\[eq56\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==).\n\nProposition If Assumptions 1, 2, 3, 4 and 5 are satisfied, and a consistent estimator ![\\$widehat{V}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) of the long-run covariance matrix ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is available, then the asymptotic variance of the OLS estimator is consistently estimated by![\\[eq57\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\nProof\n\nThis is proved as follows![\\[eq58\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)where: in step ![\\$ box{A}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) we have used the Continuous Mapping theorem; in step ![\\$ box{B}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) we have used the hypothesis that ![\\$widehat{V}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is a consistent estimator of the long-run covariance matrix ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and the fact that, by Assumption 1, the sample mean of the matrix ![\\$x\\_{i}^{ op }x\\_{i}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is a consistent estimator of ![\\[eq59\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), that is![\\$QTR{rm}{,}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)![\\[eq60\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\nThus, in order to derive a consistent estimator of the covariance matrix of the OLS estimator, we need to find a consistent estimator of the long-run covariance matrix ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). How to do this is discussed in the next section.\n\n## Consistent estimation of the long-run covariance matrix\nThe estimation of ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) requires some assumptions on the covariances between the terms of the sequence ![\\[eq61\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==).\n\n### Assumption 6 - No serial correlation\nIn order to find a simpler expression for ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), we make the following assumption.\n\n**Assumption 6**: the sequence ![\\[eq62\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is [serially uncorrelated](https:\/\/www.statlect.com\/fundamentals-of-statistics\/autocorrelation), that is,![\\[eq63\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)and [weakly stationary](https:\/\/www.statlect.com\/glossary\/covariance-stationary), that is, ![\\[eq64\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)does not depend on ![i](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==).\n\nRemember that in Assumption 3 (orthogonality) we also ask that![\\[eq65\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\n### More explicit formulae for the long-run covariance\nWe now derive simpler expressions for ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==).\n\nProposition Under Assumptions 3 (orthogonality), the long-run covariance matrix ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) satisfies![\\[eq66\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\nProof\n\nThis is proved as follows:![\\[eq67\\]](https:\/\/www.statlect.com\/images\/OLS-estimator-properties__128.png)\n\nProposition Under Assumptions 3 (orthogonality) and 6 (no serial correlation), the long-run covariance matrix ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) satisfies![\\[eq68\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\nProof\n\nThe proof is as follows:![\\[eq69\\]](https:\/\/www.statlect.com\/images\/OLS-estimator-properties__131.png)\n\n### Proof of consistency under Assumption 6\nThanks to assumption 6, we can also derive an estimator of ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==).\n\nProposition Suppose that Assumptions 1, 2, 3, 4 and 6 are satisfied, and that ![\\[eq70\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is consistently estimated by the sample mean![\\[eq71\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)Then, the long-run covariance matrix ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is consistently estimated by![\\[eq72\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\nProof\n\nWe have![\\[eq73\\]](https:\/\/www.statlect.com\/images\/OLS-estimator-properties__137.png)where in the last step we have applied the Continuous Mapping theorem separately to each entry of the matrices in square brackets, together with the fact that ![\\[eq74\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)To see how this is done, consider, for example, the matrix![\\[eq75\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)Then, the entry at the intersection of its ![k](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\\-th row and ![\\$l\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\\-th column is![\\[eq76\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)and![\\[eq77\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\n### Formula for the covariance matrix of the OLS estimator under Assumption 6\nWhen the assumptions of the previous proposition hold, the asymptotic covariance matrix of the OLS estimator is![\\[eq78\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\nAs a consequence, the covariance of the OLS estimator can be approximated by![\\[eq79\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)which is known as [heteroskedasticity-robust estimator](https:\/\/www.statlect.com\/glossary\/heteroskedasticity-robust-standard-errors).\n\n### Assumption 7 - Conditional homoskedasticity\nA further assumption is often made, which allows us to further simplify the expression for the long-run covariance matrix.\n\n**Assumption 7**: the error terms are [conditionally homoskedastic](https:\/\/www.statlect.com\/glossary\/heteroskedasticity):![\\[eq80\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\n### Proof of consistency under Assumption 7\nThis assumption has the following implication.\n\nProposition Suppose that Assumptions 1, 2, 3, 4, 5, 6 and 7 are satisfied. Then, the long-run covariance matrix ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is consistently estimated by![\\[eq81\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\nProof\n\nFirst of all, we have that![\\[eq82\\]](https:\/\/www.statlect.com\/images\/OLS-estimator-properties__149.png)But we know that, by Assumption 1, ![\\[eq83\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is consistently estimated by![\\[eq84\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)and by Assumptions 1, 2, 3 and 5, ![\\[eq85\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is consistently estimated by![\\[eq86\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)Therefore, by the Continuous Mapping theorem, the long-run covariance matrix ![V](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is consistently estimated by ![\\[eq87\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\n### Formula for the covariance matrix of the OLS estimator under Assumptions 6 and 7\nWhen the assumptions of the previous proposition hold, the asymptotic covariance matrix of the OLS estimator is![\\[eq88\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\n\nAs a consequence, the covariance of the OLS estimator can be approximated by![\\[eq89\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)which is the same estimator derived in the [normal linear regression model](https:\/\/www.statlect.com\/fundamentals-of-statistics\/normal-linear-regression-model).\n\n### Weaker assumptions\nThe assumptions above can be made even weaker (for example, by relaxing the hypothesis that ![\\[eq90\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is uncorrelated with ![\\[eq91\\]](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)), at the cost of facing more difficulties in estimating the long-run covariance matrix.\n\nFor a review of the methods that can be used to estimate ![\\$widehat{V}\\$](data:image\/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), see, for example, Den and Levin (1996).\n\n## Hypothesis testing\nThe lecture entitled [Linear regression - Hypothesis testing](https:\/\/www.statlect.com\/fundamentals-of-statistics\/linear-regression-hypothesis-testing) discusses how to carry out [hypothesis tests](https:\/\/www.statlect.com\/fundamentals-of-statistics\/hypothesis-testing) on the coefficients of a linear regression model in the cases discussed above, that is, when the OLS estimator is asymptotically normal and a consistent estimator of the asymptotic covariance matrix is available.\n\n## References\nHaan, Wouter J. Den, and Andrew T. Levin (1996). \"Inferences from parametric and non-parametric covariance matrix estimation procedures.\" Technical Working Paper Series, NBER.\n\n## How to cite\nPlease cite as:\n\nTaboga, Marco (2021). \"Properties of the OLS estimator\", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https:\/\/www.statlect.com\/fundamentals-of-statistics\/OLS-estimator-properties.\n\nThe books\n\nMost of the learning materials found on this website are now available in a traditional textbook format.\n\n[Probability and statistics](https:\/\/www.statlect.com\/about\/book)[Matrix algebra](https:\/\/www.statlect.com\/about\/linear-algebra-book)\n\nFeatured pages\n\n- [Poisson distribution](https:\/\/www.statlect.com\/probability-distributions\/Poisson-distribution)\n- [Uniform distribution](https:\/\/www.statlect.com\/probability-distributions\/uniform-distribution)\n- [Multinomial distribution](https:\/\/www.statlect.com\/probability-distributions\/multinomial-distribution)\n- [Beta function](https:\/\/www.statlect.com\/mathematical-tools\/beta-function)\n- [Student t distribution](https:\/\/www.statlect.com\/probability-distributions\/student-t-distribution)\n- [Wishart distribution](https:\/\/www.statlect.com\/probability-distributions\/wishart-distribution)\n\nExplore\n\n- [Chi-square distribution](https:\/\/www.statlect.com\/probability-distributions\/chi-square-distribution)\n- [Bayes rule](https:\/\/www.statlect.com\/fundamentals-of-probability\/Bayes-rule)\n- [Exponential distribution](https:\/\/www.statlect.com\/probability-distributions\/exponential-distribution)\n\nMain sections\n\n- [Mathematical tools](https:\/\/www.statlect.com\/mathematical-tools\/)\n- [Fundamentals of probability](https:\/\/www.statlect.com\/fundamentals-of-probability\/)\n- [Probability distributions](https:\/\/www.statlect.com\/probability-distributions\/)\n- [Asymptotic theory](https:\/\/www.statlect.com\/asymptotic-theory\/)\n- [Fundamentals of statistics](https:\/\/www.statlect.com\/fundamentals-of-statistics\/)\n- [Glossary](https:\/\/www.statlect.com\/glossary\/)\n\nAbout\n\n- [About Statlect](https:\/\/www.statlect.com\/about\/)\n- [Contacts](https:\/\/www.statlect.com\/about\/contacts)\n- [Cookies, privacy and terms of use](https:\/\/www.statlect.com\/about\/cookies-privacy-policy-and-terms-of-use)\n\nGlossary entries\n\n- [Factorial](https:\/\/www.statlect.com\/glossary\/factorial)\n- [Alternative hypothesis](https:\/\/www.statlect.com\/glossary\/alternative-hypothesis)\n- [Almost sure](https:\/\/www.statlect.com\/glossary\/almost-sure)\n- [Type II error](https:\/\/www.statlect.com\/glossary\/Type-II-error)\n- [Discrete random variable](https:\/\/www.statlect.com\/glossary\/discrete-random-variable)\n- [Integrable variable](https:\/\/www.statlect.com\/glossary\/integrable-random-variable)\n\nShare\n\n- To enhance your privacy,\n- we removed the social buttons,\n- but **don't forget to share**.","attrs_readable_markdown":null,"meta_canonical":null}

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

📄

INDEXABLE

✅

CRAWLED

11 days ago

🤖

ROBOTS ALLOWED

Page Info Filters

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	PASS	`download_stamp > now() - 6 MONTH`	0.4 months ago
History drop	PASS	`isNull(history_drop_reason)`	No drop reason
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	PASS	`meta_canonical IS NULL OR = '' OR = src_unparsed`	Not set

Page Details

Property	Value
URL	https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties
Last Crawled	2026-04-01 02:46:51 (11 days ago)
First Indexed	2016-05-05 07:46:26 (9 years ago)
HTTP Status Code	200
Meta Title	Properties of the OLS estimator \| Consistency, asymptotic normality
Meta Description	Learn what conditions are needed to prove the consistency and asymptotic normality of the OLS estimator.
Meta Canonical	null
Boilerpipe Text	In this lecture we discuss under which assumptions the OLS (Ordinary Least Squares) estimator has desirable statistical properties such as consistency and asymptotic normality. Table of contents The regression model Matrix notation The estimator Writing the estimator in terms of sample means Consistency of the OLS estimator Assumption 1 - Convergence of sample means to population means Assumption 2 - Full rank Assumption 3 - Orthogonality Proof of consistency Asymptotic normality of the OLS estimator Assumption 4 - CLT condition Proof of asymptotic normality Consistent estimation of the variance of the error terms Assumption 5 - Regularity of error terms Proof of consistency Consistent estimation of the asymptotic covariance matrix Proof of consistency Consistent estimation of the long-run covariance matrix Assumption 6 - No serial correlation More explicit formulae for the long-run covariance Proof of consistency under Assumption 6 Formula for the covariance matrix of the OLS estimator under Assumption 6 Assumption 7 - Conditional homoskedasticity Proof of consistency under Assumption 7 Formula for the covariance matrix of the OLS estimator under Assumptions 6 and 7 Weaker assumptions Hypothesis testing References The regression model Consider the linear regression model where: the outputs are denoted by ; the associated vectors of inputs are denoted by ; the vector of regression coefficients is denoted by ; are unobservable error terms. Matrix notation We assume to observe a sample of realizations, so that the vector of all outputs is an vector, the design matrix is an matrix, and the vector of error terms is an vector. The estimator The OLS estimator is the vector of regression coefficients that minimizes the sum of squared residuals: As proved in the lecture on Linear regression , if the design matrix has full rank, then the OLS estimator is computed as follows: Writing the estimator in terms of sample means The OLS estimator can be written as where is the sample mean of the matrix and is the sample mean of the matrix . Consistency of the OLS estimator In this section we are going to propose a set of conditions that are sufficient for the consistency of the OLS estimator, that is, for the convergence in probability of to the true value . Assumption 1 - Convergence of sample means to population means The first assumption we make is that the sample means in the OLS formula converge to their population counterparts, which is formalized as follows. Assumption 1 (convergence) : both the sequence and the sequence satisfy sets of conditions that are sufficient for the convergence in probability of their sample means to the population means and , which do not depend on . For example, the sequences and could be assumed to satisfy the conditions of Chebyshev's Weak Law of Large Numbers for correlated sequences , which are quite mild (basically, it is only required that the sequences are covariance stationary and that their auto-covariances are zero on average). Assumption 2 - Full rank The second assumption we make is a rank assumption (sometimes also called identification assumption). Assumption 2 (rank) : the square matrix has full rank (as a consequence, it is invertible ). Assumption 3 - Orthogonality The third assumption we make is that the regressors are orthogonal to the error terms . Assumption 3 (orthogonality) : For each , and are orthogonal, that is, Proof of consistency It is then straightforward to prove the following proposition. Proposition If Assumptions 1, 2 and 3 are satisfied, then the OLS estimator is a consistent estimator of . Proof Asymptotic normality of the OLS estimator We now introduce a new assumption, and we use it to prove the asymptotic normality of the OLS estimator. Assumption 4 - CLT condition The assumption is as follows. Assumption 4 (Central Limit Theorem) : the sequence satisfies a set of conditions that are sufficient to guarantee that a Central Limit Theorem applies to its sample mean For a review of some of the conditions that can be imposed on a sequence to guarantee that a Central Limit Theorem applies to its sample mean, you can go to the lecture on the Central Limit Theorem . In any case, remember that if a Central Limit Theorem applies to , then, as tends to infinity, converges in distribution to a multivariate normal distribution with mean equal to and covariance matrix equal to Proof of asymptotic normality With Assumption 4 in place, we are now able to prove the asymptotic normality of the OLS estimator. Proposition If Assumptions 1, 2, 3 and 4 are satisfied, then the OLS estimator is asymptotically multivariate normal with mean equal to and asymptotic covariance matrix equal to that is, where has been defined above. Proof As in the proof of consistency, the dependence of the estimator on the sample size is made explicit, so that the OLS estimator is denoted by . First of all, we have where, in the last step, we have used the fact that, by Assumption 3, . Note that, by Assumption 1 and the Continuous Mapping theorem, we have Furthermore, by Assumption 4, we have that converges in distribution to a multivariate normal random vector having mean equal to and covariance matrix equal to . Thus, by Slutski's theorem, we have that converges in distribution to a multivariate normal vector with mean equal to and covariance matrix equal to Consistent estimation of the variance of the error terms We now discuss the consistent estimation of the variance of the error terms. Assumption 5 - Regularity of error terms Here is an additional assumption. Assumption 5 : the sequence satisfies a set of conditions that are sufficient for the convergence in probability of its sample mean to the population mean which does not depend on . Proof of consistency If this assumption is satisfied, then the variance of the error terms can be estimated by the sample variance of the residuals where Proposition Under Assumptions 1, 2, 3, and 5, it can be proved that is a consistent estimator of . Proof Consistent estimation of the asymptotic covariance matrix We have proved that the asymptotic covariance matrix of the OLS estimator is where the long-run covariance matrix is defined by Usually, the matrix needs to be estimated because it depends on quantities ( and ) that are not known. Proof of consistency The next proposition characterizes consistent estimators of . Proposition If Assumptions 1, 2, 3, 4 and 5 are satisfied, and a consistent estimator of the long-run covariance matrix is available, then the asymptotic variance of the OLS estimator is consistently estimated by Proof Thus, in order to derive a consistent estimator of the covariance matrix of the OLS estimator, we need to find a consistent estimator of the long-run covariance matrix . How to do this is discussed in the next section. Consistent estimation of the long-run covariance matrix The estimation of requires some assumptions on the covariances between the terms of the sequence . Assumption 6 - No serial correlation In order to find a simpler expression for , we make the following assumption. Assumption 6 : the sequence is serially uncorrelated , that is, and weakly stationary , that is, does not depend on . Remember that in Assumption 3 (orthogonality) we also ask that More explicit formulae for the long-run covariance We now derive simpler expressions for . Proposition Under Assumptions 3 (orthogonality), the long-run covariance matrix satisfies Proof This is proved as follows: Proposition Under Assumptions 3 (orthogonality) and 6 (no serial correlation), the long-run covariance matrix satisfies Proof The proof is as follows: Proof of consistency under Assumption 6 Thanks to assumption 6, we can also derive an estimator of . Proposition Suppose that Assumptions 1, 2, 3, 4 and 6 are satisfied, and that is consistently estimated by the sample mean Then, the long-run covariance matrix is consistently estimated by Proof Formula for the covariance matrix of the OLS estimator under Assumption 6 When the assumptions of the previous proposition hold, the asymptotic covariance matrix of the OLS estimator is As a consequence, the covariance of the OLS estimator can be approximated by which is known as heteroskedasticity-robust estimator . Assumption 7 - Conditional homoskedasticity A further assumption is often made, which allows us to further simplify the expression for the long-run covariance matrix. Assumption 7 : the error terms are conditionally homoskedastic : Proof of consistency under Assumption 7 This assumption has the following implication. Proposition Suppose that Assumptions 1, 2, 3, 4, 5, 6 and 7 are satisfied. Then, the long-run covariance matrix is consistently estimated by Proof Formula for the covariance matrix of the OLS estimator under Assumptions 6 and 7 When the assumptions of the previous proposition hold, the asymptotic covariance matrix of the OLS estimator is As a consequence, the covariance of the OLS estimator can be approximated by which is the same estimator derived in the normal linear regression model . Weaker assumptions The assumptions above can be made even weaker (for example, by relaxing the hypothesis that is uncorrelated with ), at the cost of facing more difficulties in estimating the long-run covariance matrix. For a review of the methods that can be used to estimate , see, for example, Den and Levin (1996). Hypothesis testing The lecture entitled Linear regression - Hypothesis testing discusses how to carry out hypothesis tests on the coefficients of a linear regression model in the cases discussed above, that is, when the OLS estimator is asymptotically normal and a consistent estimator of the asymptotic covariance matrix is available. References Haan, Wouter J. Den, and Andrew T. Levin (1996). "Inferences from parametric and non-parametric covariance matrix estimation procedures." Technical Working Paper Series, NBER. How to cite Please cite as: Taboga, Marco (2021). "Properties of the OLS estimator", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties.
Markdown	![Search for probability and statistics terms on Statlect](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABkAAAAXCAMAAADJPRQhAAACXlBMVEX///8zZpkyZZktY5guY5gxZZk0Z5kyZpkyZZgjWZAtYpchXJQcVo8tY5f//PgiWpE2XIkvY5ctZJkwZZksY5gmXpUlXJM0ZpklXZQkXJOpssJNbpbt5+csYpcvZJcmXZP79vQ6Y5H59fQjWpImXJItXpLX2N1OcpqUp78kV4wXUYuDk6saVI11h6NNbpUdVo8eV4+hrsFEbJf///7i4+d4kK6PrMmWpbgZVI6ToLW0vcoiU4cmVon69/fEx88YT4clWIxAa5oZUotAYo3Pz9Tt7O2vs8AZVY8nVYghWZExZZjt7fAHQn81Z5okXJSBlrAzXIovZJkoU4TEydMZVZAfVIrJ0NubrMAjWZEkWpD78OyuusseV5AeVIsrYJTe5e2ywdMjWpDu8fS2wtImWo/T2+Oqucqrucrx7Ovd2dsSUY20w9MdVIyxu8ppgqMoX5UjV4wQQ3szZprn5uk/ZpItZJgTTIcpYJZTdJpshqYcVo4iWpJ/katBZ5McU4q9xM/28/ImVooWUo5whKHn4uQ6Yo8mX5ZLbZX/+/k5Y5EvZJglXpYZTIJtiambqr6ouMuQorlNcJkTTYcuZJkrYpgoWIn18vIbToUeWJEpX5UjWpEeU4oxZpovV4f59vXg4ucjVIkuZJonVIa7wMtHa5QUTIYnX5X99vMvY5gsYZYfWZEdUIZxiqjR198nWIuGmbEkUYPd3+LO0tlsiatIcJw9aZg8bZ5AaZZUeqOVorfs7O7Cy9gkV4stZJojT4Ho5+n8+ff7/P3/+fYdSn359fNEa5YeS3769/X//vz69/Y1HbsLAAABU0lEQVR4Xm3JY5MrURiF0fN2x+bQtm3btm3btm1c2rbxr25V+mTSM5X96dm1EG02tXY+vo1G6PoMAl0trUx1xBGRUVfAwnBADgkMoQkBxg6ONDAz1wOmm7WHbbi9PrCdnDXiogeKIPdEhDy9vA+PwM9fDQECYMQF4xMSCsywTiySbqI4GqkXE6sb/4RKUVJySirSLC1dnkFVZhZk59AkN4/IL1BVYRGnpJQmZWJOeYWqKgVQVU2TGi7BrVNVfQOrqZkmLa1EG872DrJLA3d6oLcPd78SBocuZVgBI6O4x8YJmJikempaCTOzCG9ufgF0F5eWV1bX1jc2YWtbhGFnV8jcY7DY+/oHOjwWm398guH0jDy/uHHz1m2eVHr33v0HDxHeo8ek7OkzhJ6/ePnq9Zu3796r4cNH8tPnL1R//fYdaSbhy378RFr36/cfpH1//2mF/6EqUMbQOA8lAAAAAElFTkSuQmCC) [StatLect](https://www.statlect.com/) [Index](https://www.statlect.com/) \> [Fundamentals of statistics](https://www.statlect.com/fundamentals-of-statistics/) # Properties of the OLS estimator by [Marco Taboga](https://www.statlect.com/about/#author), PhD In this lecture we discuss under which assumptions the OLS (Ordinary Least Squares) estimator has desirable statistical properties such as consistency and asymptotic normality. ![Infographic providing an overview of the assumptions needed to prove the consistency and asymptotic normality of the OLS estimator.](https://www.statlect.com/images/OLS-properties.png) ![Table of Contents](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAB0AAAAYCAIAAACJPGHrAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAABJSURBVEhLY/z//z8DDQATlKY2GGrmgsL3yJEjUB4Dg42NDZBEFiEeQPRCwGj4QsBQS7+j6QECRtMDBIymBwgYTQ8QMLTMZWAAAEw6JxXCQPIuAAAAAElFTkSuQmCC) Table of contents 1. [The regression model](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid2) 2. [Matrix notation](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid3) 3. [The estimator](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid4) 4. [Writing the estimator in terms of sample means](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid5) 5. [Consistency of the OLS estimator](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid6) 1. [Assumption 1 - Convergence of sample means to population means](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid7) 2. [Assumption 2 - Full rank](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid8) 3. [Assumption 3 - Orthogonality](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid9) 4. [Proof of consistency](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid10) 6. [Asymptotic normality of the OLS estimator](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid11) 1. [Assumption 4 - CLT condition](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid12) 2. [Proof of asymptotic normality](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid13) 7. [Consistent estimation of the variance of the error terms](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid14) 1. [Assumption 5 - Regularity of error terms](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid15) 2. [Proof of consistency](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid16) 8. [Consistent estimation of the asymptotic covariance matrix](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid17) 1. [Proof of consistency](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid18) 9. [Consistent estimation of the long-run covariance matrix](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid19) 1. [Assumption 6 - No serial correlation](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid20) 2. [More explicit formulae for the long-run covariance](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid21) 3. [Proof of consistency under Assumption 6](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid22) 4. [Formula for the covariance matrix of the OLS estimator under Assumption 6](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid23) 5. [Assumption 7 - Conditional homoskedasticity](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid24) 6. [Proof of consistency under Assumption 7](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid25) 7. [Formula for the covariance matrix of the OLS estimator under Assumptions 6 and 7](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid26) 8. [Weaker assumptions](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid27) 10. [Hypothesis testing](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid28) 11. [References](https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties#hid29) ## The regression model Consider the [linear regression](https://www.statlect.com/fundamentals-of-statistics/linear-regression) model![\[eq1\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)where: - the outputs are denoted by ![\$y\_{i}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==); - the associated ![\$1 imes K\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) vectors of inputs are denoted by ![\$x\_{i}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==); - the ![Kx1](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) vector of regression coefficients is denoted by ![\$eta \$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==); - ![\$arepsilon \_{i}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) are unobservable error terms. ## Matrix notation We assume to observe a sample of ![\$N\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) realizations, so that the vector of all outputs ![\[eq2\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)is an ![\$N imes 1\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) vector, the [design matrix](https://www.statlect.com/glossary/design-matrix)![\[eq3\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)is an ![\$N imes K\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) matrix, and the vector of error terms![\[eq4\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)is an ![\$N imes 1\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) vector. ## The estimator The OLS estimator ![\$widehat{eta }\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is the vector of regression coefficients that minimizes the sum of squared residuals:![\[eq5\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) As proved in the lecture on [Linear regression](https://www.statlect.com/fundamentals-of-statistics/linear-regression), if the design matrix ![X](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) has full rank, then the OLS estimator is computed as follows:![\[eq6\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) ## Writing the estimator in terms of sample means The OLS estimator can be written as ![\[eq7\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)where ![\[eq8\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)is the [sample mean](https://www.statlect.com/glossary/sample-mean) of the ![\$K imes K\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) matrix ![\$x\_{i}^{ op }x\_{i}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\[eq9\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)is the sample mean of the ![Kx1](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) matrix ![\$x\_{i}^{ op }y\_{i}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). ## Consistency of the OLS estimator In this section we are going to propose a set of conditions that are sufficient for the [consistency](https://www.statlect.com/glossary/consistent-estimator) of the OLS estimator, that is, for the convergence in probability of ![\$widehat{eta }\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) to the true value ![\$eta \$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). ### Assumption 1 - Convergence of sample means to population means The first assumption we make is that the sample means in the OLS formula converge to their population counterparts, which is formalized as follows. Assumption 1 (convergence): both the sequence ![\[eq10\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and the sequence ![\[eq11\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) satisfy sets of conditions that are sufficient for the [convergence in probability](https://www.statlect.com/asymptotic-theory/convergence-in-probability) of their sample means to the population means ![\[eq12\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\[eq13\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), which do not depend on ![i](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). For example, the sequences ![\[eq14\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\[eq15\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) could be assumed to satisfy the conditions of [Chebyshev's Weak Law of Large Numbers for correlated sequences](https://www.statlect.com/asymptotic-theory/law-of-large-numbers#cheby2), which are quite mild (basically, it is only required that the sequences are [covariance stationary](https://www.statlect.com/glossary/covariance-stationary) and that their auto-covariances are zero on average). ### Assumption 2 - Full rank The second assumption we make is a rank assumption (sometimes also called identification assumption). Assumption 2 (rank): the square matrix ![\[eq16\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) has [full rank](https://www.statlect.com/matrix-algebra/rank-of-a-matrix) (as a consequence, it is [invertible](https://www.statlect.com/matrix-algebra/inverse-matrix)). ### Assumption 3 - Orthogonality The third assumption we make is that the regressors ![\$x\_{i}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) are orthogonal to the error terms ![\$arepsilon \_{i}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). Assumption 3 (orthogonality): For each ![i](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), ![\$x\_{i}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\$arepsilon \_{i}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) are orthogonal, that is,![\[eq17\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) ### Proof of consistency It is then straightforward to prove the following proposition. Proposition If Assumptions 1, 2 and 3 are satisfied, then the OLS estimator ![\$widehat{eta }\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is a consistent estimator of ![\$eta \$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). Proof Let us make explicit the dependence of the estimator on the sample size and denote by ![\[eq18\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) the OLS estimator obtained when the sample size is equal to ![\$N.\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) By Assumption 1 and by the [Continuous Mapping theorem](https://www.statlect.com/asymptotic-theory/continuous-mapping-theorem), we have that the probability limit of ![\[eq19\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is ![\[eq20\]](https://www.statlect.com/images/OLS-estimator-properties__47.png)Now, if we pre-multiply the regression equation![\[eq21\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)by ![\$x\_{i}^{ op }\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and we take expected values, we get![\[eq22\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)But by Assumption 3, it becomes![\[eq23\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)or![\[eq24\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)which implies that![\[eq25\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) ## Asymptotic normality of the OLS estimator We now introduce a new assumption, and we use it to prove the asymptotic normality of the OLS estimator. ### Assumption 4 - CLT condition The assumption is as follows. Assumption 4 (Central Limit Theorem): the sequence ![\[eq26\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) satisfies a set of conditions that are sufficient to guarantee that a Central Limit Theorem applies to its sample mean![\[eq27\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) For a review of some of the conditions that can be imposed on a sequence to guarantee that a Central Limit Theorem applies to its sample mean, you can go to the lecture on the [Central Limit Theorem](https://www.statlect.com/asymptotic-theory/central-limit-theorem). In any case, remember that if a Central Limit Theorem applies to ![\[eq26\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), then, as ![\$N\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) tends to infinity,![\[eq29\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) [converges in distribution](https://www.statlect.com/asymptotic-theory/convergence-in-distribution) to a [multivariate normal distribution](https://www.statlect.com/probability-distributions/multivariate-normal-distribution) with mean equal to ![0](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and covariance matrix equal to![\[eq30\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) ### Proof of asymptotic normality With Assumption 4 in place, we are now able to prove the asymptotic normality of the OLS estimator. Proposition If Assumptions 1, 2, 3 and 4 are satisfied, then the OLS estimator ![\$widehat{eta }\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is asymptotically multivariate normal with mean equal to ![\$eta \$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and asymptotic covariance matrix equal to![\[eq31\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)that is,![\[eq32\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)where ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) has been defined above. Proof As in the proof of consistency, the dependence of the estimator on the sample size is made explicit, so that the OLS estimator is denoted by ![\[eq33\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). First of all, we have ![\[eq34\]](https://www.statlect.com/images/OLS-estimator-properties__67.png)where, in the last step, we have used the fact that, by Assumption 3, ![\[eq35\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). Note that, by Assumption 1 and the Continuous Mapping theorem, we have![\[eq36\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)Furthermore, by Assumption 4, we have that![\[eq37\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)converges in distribution to a multivariate normal random vector having mean equal to ![0](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and covariance matrix equal to ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). Thus, by Slutski's theorem, we have that![\[eq38\]](https://www.statlect.com/images/OLS-estimator-properties__73.png)converges in distribution to a multivariate normal vector with mean equal to ![0](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and covariance matrix equal to ![\[eq39\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) ## Consistent estimation of the variance of the error terms We now discuss the consistent estimation of the variance of the error terms. ### Assumption 5 - Regularity of error terms Here is an additional assumption. Assumption 5: the sequence ![\[eq40\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) satisfies a set of conditions that are sufficient for the convergence in probability of its sample mean![\[eq41\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)to the population mean ![\[eq42\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)which does not depend on ![i](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). ### Proof of consistency If this assumption is satisfied, then the variance of the error terms ![sigma^2](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) can be estimated by the sample variance of the residuals![\[eq43\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)where ![\[eq44\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) Proposition Under Assumptions 1, 2, 3, and 5, it can be proved that ![\[eq45\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is a consistent estimator of ![sigma^2](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). Proof Let us make explicit the dependence of the estimators on the sample size and denote by ![\[eq46\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\[eq47\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) the estimators obtained when the sample size is equal to ![\$N.\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) By Assumption 1 and by the [Continuous Mapping theorem](https://www.statlect.com/asymptotic-theory/continuous-mapping-theorem), we have that the probability limit of ![\[eq47\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is ![\[eq49\]](https://www.statlect.com/images/OLS-estimator-properties__89.png)where: in steps ![\$ rame{A}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\$ rame{C}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) we have used the Continuous Mapping Theorem; in step ![\$ rame{B}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) we have used Assumption 5; in step ![\$ rame{D}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) we have used the fact that ![\[eq50\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)because ![\[eq51\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is a consistent estimator of ![\$eta \$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), as proved above. ## Consistent estimation of the asymptotic covariance matrix We have proved that the asymptotic covariance matrix of the OLS estimator is![\[eq52\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)where the long-run covariance matrix ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is defined by![\[eq53\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) Usually, the matrix ![\[eq54\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) needs to be estimated because it depends on quantities (![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and ![\[eq16\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)) that are not known. ### Proof of consistency The next proposition characterizes consistent estimators of ![\[eq56\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). Proposition If Assumptions 1, 2, 3, 4 and 5 are satisfied, and a consistent estimator ![\$widehat{V}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) of the long-run covariance matrix ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is available, then the asymptotic variance of the OLS estimator is consistently estimated by![\[eq57\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) Proof This is proved as follows![\[eq58\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)where: in step ![\$ box{A}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) we have used the Continuous Mapping theorem; in step ![\$ box{B}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) we have used the hypothesis that ![\$widehat{V}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is a consistent estimator of the long-run covariance matrix ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) and the fact that, by Assumption 1, the sample mean of the matrix ![\$x\_{i}^{ op }x\_{i}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is a consistent estimator of ![\[eq59\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), that is![\$QTR{rm}{,}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)![\[eq60\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) Thus, in order to derive a consistent estimator of the covariance matrix of the OLS estimator, we need to find a consistent estimator of the long-run covariance matrix ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). How to do this is discussed in the next section. ## Consistent estimation of the long-run covariance matrix The estimation of ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) requires some assumptions on the covariances between the terms of the sequence ![\[eq61\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). ### Assumption 6 - No serial correlation In order to find a simpler expression for ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), we make the following assumption. Assumption 6: the sequence ![\[eq62\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is [serially uncorrelated](https://www.statlect.com/fundamentals-of-statistics/autocorrelation), that is,![\[eq63\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)and [weakly stationary](https://www.statlect.com/glossary/covariance-stationary), that is, ![\[eq64\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)does not depend on ![i](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). Remember that in Assumption 3 (orthogonality) we also ask that![\[eq65\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) ### More explicit formulae for the long-run covariance We now derive simpler expressions for ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). Proposition Under Assumptions 3 (orthogonality), the long-run covariance matrix ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) satisfies![\[eq66\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) Proof This is proved as follows:![\[eq67\]](https://www.statlect.com/images/OLS-estimator-properties__128.png) Proposition Under Assumptions 3 (orthogonality) and 6 (no serial correlation), the long-run covariance matrix ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) satisfies![\[eq68\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) Proof The proof is as follows:![\[eq69\]](https://www.statlect.com/images/OLS-estimator-properties__131.png) ### Proof of consistency under Assumption 6 Thanks to assumption 6, we can also derive an estimator of ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==). Proposition Suppose that Assumptions 1, 2, 3, 4 and 6 are satisfied, and that ![\[eq70\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is consistently estimated by the sample mean![\[eq71\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)Then, the long-run covariance matrix ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is consistently estimated by![\[eq72\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) Proof We have![\[eq73\]](https://www.statlect.com/images/OLS-estimator-properties__137.png)where in the last step we have applied the Continuous Mapping theorem separately to each entry of the matrices in square brackets, together with the fact that ![\[eq74\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)To see how this is done, consider, for example, the matrix![\[eq75\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)Then, the entry at the intersection of its ![k](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\-th row and ![\$l\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)\-th column is![\[eq76\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)and![\[eq77\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) ### Formula for the covariance matrix of the OLS estimator under Assumption 6 When the assumptions of the previous proposition hold, the asymptotic covariance matrix of the OLS estimator is![\[eq78\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) As a consequence, the covariance of the OLS estimator can be approximated by![\[eq79\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)which is known as [heteroskedasticity-robust estimator](https://www.statlect.com/glossary/heteroskedasticity-robust-standard-errors). ### Assumption 7 - Conditional homoskedasticity A further assumption is often made, which allows us to further simplify the expression for the long-run covariance matrix. Assumption 7: the error terms are [conditionally homoskedastic](https://www.statlect.com/glossary/heteroskedasticity):![\[eq80\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) ### Proof of consistency under Assumption 7 This assumption has the following implication. Proposition Suppose that Assumptions 1, 2, 3, 4, 5, 6 and 7 are satisfied. Then, the long-run covariance matrix ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is consistently estimated by![\[eq81\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) Proof First of all, we have that![\[eq82\]](https://www.statlect.com/images/OLS-estimator-properties__149.png)But we know that, by Assumption 1, ![\[eq83\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is consistently estimated by![\[eq84\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)and by Assumptions 1, 2, 3 and 5, ![\[eq85\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is consistently estimated by![\[eq86\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)Therefore, by the Continuous Mapping theorem, the long-run covariance matrix ![V](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is consistently estimated by ![\[eq87\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) ### Formula for the covariance matrix of the OLS estimator under Assumptions 6 and 7 When the assumptions of the previous proposition hold, the asymptotic covariance matrix of the OLS estimator is![\[eq88\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) As a consequence, the covariance of the OLS estimator can be approximated by![\[eq89\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)which is the same estimator derived in the [normal linear regression model](https://www.statlect.com/fundamentals-of-statistics/normal-linear-regression-model). ### Weaker assumptions The assumptions above can be made even weaker (for example, by relaxing the hypothesis that ![\[eq90\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==) is uncorrelated with ![\[eq91\]](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)), at the cost of facing more difficulties in estimating the long-run covariance matrix. For a review of the methods that can be used to estimate ![\$widehat{V}\$](data:image/gif;base64,R0lGODlhAQABAIAAANvf7wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==), see, for example, Den and Levin (1996). ## Hypothesis testing The lecture entitled [Linear regression - Hypothesis testing](https://www.statlect.com/fundamentals-of-statistics/linear-regression-hypothesis-testing) discusses how to carry out [hypothesis tests](https://www.statlect.com/fundamentals-of-statistics/hypothesis-testing) on the coefficients of a linear regression model in the cases discussed above, that is, when the OLS estimator is asymptotically normal and a consistent estimator of the asymptotic covariance matrix is available. ## References Haan, Wouter J. Den, and Andrew T. Levin (1996). "Inferences from parametric and non-parametric covariance matrix estimation procedures." Technical Working Paper Series, NBER. ## How to cite Please cite as: Taboga, Marco (2021). "Properties of the OLS estimator", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-statistics/OLS-estimator-properties. The books Most of the learning materials found on this website are now available in a traditional textbook format. [Probability and statistics](https://www.statlect.com/about/book)[Matrix algebra](https://www.statlect.com/about/linear-algebra-book) Featured pages - [Poisson distribution](https://www.statlect.com/probability-distributions/Poisson-distribution) - [Uniform distribution](https://www.statlect.com/probability-distributions/uniform-distribution) - [Multinomial distribution](https://www.statlect.com/probability-distributions/multinomial-distribution) - [Beta function](https://www.statlect.com/mathematical-tools/beta-function) - [Student t distribution](https://www.statlect.com/probability-distributions/student-t-distribution) - [Wishart distribution](https://www.statlect.com/probability-distributions/wishart-distribution) Explore - [Chi-square distribution](https://www.statlect.com/probability-distributions/chi-square-distribution) - [Bayes rule](https://www.statlect.com/fundamentals-of-probability/Bayes-rule) - [Exponential distribution](https://www.statlect.com/probability-distributions/exponential-distribution) Main sections - [Mathematical tools](https://www.statlect.com/mathematical-tools/) - [Fundamentals of probability](https://www.statlect.com/fundamentals-of-probability/) - [Probability distributions](https://www.statlect.com/probability-distributions/) - [Asymptotic theory](https://www.statlect.com/asymptotic-theory/) - [Fundamentals of statistics](https://www.statlect.com/fundamentals-of-statistics/) - [Glossary](https://www.statlect.com/glossary/) About - [About Statlect](https://www.statlect.com/about/) - [Contacts](https://www.statlect.com/about/contacts) - [Cookies, privacy and terms of use](https://www.statlect.com/about/cookies-privacy-policy-and-terms-of-use) Glossary entries - [Factorial](https://www.statlect.com/glossary/factorial) - [Alternative hypothesis](https://www.statlect.com/glossary/alternative-hypothesis) - [Almost sure](https://www.statlect.com/glossary/almost-sure) - [Type II error](https://www.statlect.com/glossary/Type-II-error) - [Discrete random variable](https://www.statlect.com/glossary/discrete-random-variable) - [Integrable variable](https://www.statlect.com/glossary/integrable-random-variable) Share - To enhance your privacy, - we removed the social buttons, - but don't forget to share.
Readable Markdown	null
Shard	126 (laksa)
Root Hash	3586688910177265926
Unparsed URL	com,statlect!www,/fundamentals-of-statistics/OLS-estimator-properties s443