Probability and Measure Theory SECOND EDITION ROBERT B. ASH with contributions from Catherine Doléans-Dade OF vee £999 A Harcourt Science and Technology Company San Diego San Francisco New York Boston London Toronto Sydney TokyoThis book is printed on acid-free paper. © Copyright © 2000 by Academic Press All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. ACADEMIC PRESS A Harcourt Science and Technology Company 525 B Street, 1900, San Diego, California 92101-4495, USA http://www.apnet.com ‘Academic Press 24-28 Oval Road, London NW1 7DX, UK hitp://www.hbuk.co.uk/ap/ Harcourt/Academic Press 200 Wheeler Road, Burlington, Massachusetts 01803 http://www. harcourt.ap.com Library of Congress Catalog Card Number: 99-65669 International Standard Book Number: 0-12-065202-1 Printed in the United States of America 99 00 O01 02 03 PW 987654321Contents Preface Summary of Notation 1 Fundamentals of Measure and Integration Theory Introduction Fields, o-Fields, and Measures Extension of Measures Lebesgue-Stieltjes Measures and Distribution Functions Measurable Functions and Integration Basic Integration Theorems Comparison of Lebesgue and Riemann Integrals 2 Further Results in Measure and Integration Theory Introduction Radon—Nikodym Theorem and Related Results Applications to Real Analysis L? Spaces Convergence of Sequences of Measurable Functions Product Measures and Fubini’s Theorem Measures on Infinite Product Spaces Weak Convergence of Measures References 3. Introduction to Functional Analysis Introduction Basic Properties of Hilbert Spaces Linear Operators on Normed Linear Spaces Basic Theorems of Functional Analysis References vii ix 127 127 130 141 152 1654 Basic Concepts of Probability 4.12 Introduction Discrete Probability Spaces Independence Bernouli Trials Conditional Probability Random Variables Random Vectors Independent Random Variables Some Examples from Basic Probability Expectation Infinite Sequences of Random Variables References 5 Conditional Probability and Expectation 5.1 5.2 5.3 54 5.5 5.6 Introduction Applications The General Concept of Conditional Probability and Expectation Conditional Expectation Given a o-Field Properties of Conditional Expectation Regular Conditional Probabilities 6 Strong Laws of Large Numbers and Martingale Theory 6.1 6.2 6.3 6.4 6.5 6.6 Introduction Convergence Theorems Martingales Martingale Convergence Theorems Uniform Integrability Uniform Integrability and Martingale Theory Optional Sampling Theorems Applications of Martingale Theory Applications to Markov Chains References 7 The Central Limit Theorem 7A 7.2 73 Introduction The Fundamental Weak Compactness Theorem Convergence to a Normal Distribution CONTENTS 166 166 167 167 170 171 173 176 178 181 188 196 200 201 201 202 204 215 220 228 235 235 239 248 257 262 266 270 277 285 288 290 290 307CONTENTS, Stable Distributions Infinitely Divisible Distributions Uniform Convergence in the Central Limit Theorem The Skorokhod Construction and Other Convergence Theorems The k-Dimensional Central Limit Theorem References 8 Ergodic Theory 8.1 8.2 8.3 8.4 Introduction Ergodicity and Mixing The Pointwise Ergodic Theorem Applications to Markov Chains The Shannon—McMillan Theorem Entropy of a Transformation Bernoulli Shifts References 9 Brownian Motion and Stochastic Integrals 9.1 Stochastic Processes 9.2 Brownian Motion 9.3. Nowhere Differentiability and Quadratic Variation of Paths 9.4 Law of the Iterated Logarithm 9.5 The Markov Property 9.6 Martingales 9.7 It6 Integrals 9.8 — It6’s Differentiation Formula 9.9 References Appendices 1. The Symmetric Random Walk in R* 2. Semicontinuous Functions 3. Completion of the Proof of Theorem 7.3.2 4. Proof of the Convergence of Types Theorem 7.3.4 S. The Multivariate Normal Distribution Bibliography Solutions to Problems Index 317 320 329 332 336 344 345 350 356 368 374 386 394 397 438 443 447 449 454 456 512Preface It is a pleasure to accept the invitation of Harcourt/Academic Press to publish a second edition. The first edition has been used mainly in graduate courses in measure and probability, offered by departments of mathematics and statistics and frequently taken by engineers. We have prepared the present text with this audience in mind, and the title has been changed from Real Analysis and Probability to Probability and Measure Theory to reflect the revisions we have made. Chapters 1 and 2 develop the fundamentals of measure and integration the- ory. Included are several results that are crucial in constructing the foundations of probability: the Radon—Nikodym theorem, the product measure theorem, the Kolmogorov extension theorem and the theory of weak convergence of measures. We remain convinced that it is best to assemble a complete set of measure-theoretic tools before going into probability, rather than try to de- velop both areas simultaneously. The gain in efficiency far outweighs any temporary loss in motivation. Those who wish to reach probability as quickly as possible may omit Chapter 3, which gives a brief introduction to functional analysis, and Section 2.3, which gives some applications to real analysis. In addition, instructors may wish to summarize or sketch some of the intricate constructions in Sections 1.3, 1.4, and 2.7, The study of probability begins with Chapter 4, which offers a summary of an undergraduate probability course from a measure-theoretic point of view, Chapter 5 is concerned with the general concept of conditional probability and expectation. The approach to problems that involve conditioning, given events of probability zero, is the gateway to many areas of probability theory. Chapter 6 deals with strong laws of large numbers, first from the classical viewpoint, and then via martingale theory. Basic properties and applications of martingale sequences are developed systematically. Chapter 7 considers the central limit problem, emphasizing the fundamental role of Prokhorov’s weak compactness theorem. The last two sections of this chapter cover some material (not in the first edition) of special interest to statisticians: Slutsky’s theorem, the Skorokhod construction, convergence of transformed sequences and a k-dimensional central limit theorem. viiviii PREFACE Chapters 8 and 9 have been added in the second edition, and should be of interest to the entire prospective audience: mathematicians, statisticians, and engineers. Chapter 8 covers ergodic theory, which is developed far enough so that connections with information theory are clearly visible. The Shan- non—McMillan theorem is proved and the isomorphism problem for Bernoulli shifts is discussed. Chapter 9 treats the one-dimensional Brownian motion in detail, and then introduces stochastic integrals and the It6 differentiation formula. To make room for the new material, the appendix on general topology and the old Chapter 4 on the interplay between measure theory and topology have been removed, along with the section on topological vector spaces in Chapter 3. We assume that the reader has had a course in basic analysis and is familiar with metric spaces, but not with general topology. All the necessary background appears in Real Variables With Basic Metric Space Topology by Robert B. Ash, IEEE Press, 1993. (The few exercises that require additional background are marked with an asterisk.) It is theoretically possible to read the text without any prior exposure to probability, picking up the necessary equipment in Chapter 4. But we expect that in practice, almost aif readers will have taken a standard undergradu- ate probability course. We believe that discrete time, discrete state Markov chains, and random walks are best covered in a second undergraduate prob- ability course, without measure theory. But instructors and students usually find this area appealing, and we discuss the symmetric random walk on R* in Appendix 1. Problems are given at the end of each section. Fairly detailed solutions are given to many problems, and instructors may obtain solutions to those prob- lems in Chapters 1-8 not worked out in the text by writing to the publisher. Catherine Doleans—Dade wrote Chapter 9, and offered valuable advice and criticism for the other chapters. Mel Gardner kindly allowed some material from Topics in Stochastic Processes by Ash and Gardner to be used in Chap- ter 8. We appreciate the encouragement and support provided by the staff at Harcourt/Academic Press. Robert B. Ash Catherine Doleans—Dade Urbana, Illinois, 1999Summary of Notation We indicate here the notational conventions to be used throughout the book. The numbering system is standard; for example, 2.7.4 means Chapter 2, Section 7, Part 4. In the appendices, the letter A is used; thus A2.3 means Part 3 of Appendix 2. The symbol (1 is used to mark the end of a proof. 1 Sers If A and B are subsets of a set Q, A U B will denote the union of A and B, and AM B the intersection of A and B. The union and intersection of a family of sets A; are denoted by UA; and (}, Aj. The complement of A (relative to ) is denoted by A°. ‘The statement “B is a subset of A” is denoted by B C A; the inclusion need not be proper, that is, we have A C A for any set A. We also write B C A as A> B, to be read “A is an overset (or superset) of B.” The notation A — B will always mean, unless otherwise specified, the set of points that belong to A but not to B. It is referred to as the difference between A and B; a proper difference is a set A— B, where B CA. The symmetric difference between A and B is by definition the union of A-—B and B —A,; it is denoted by A AB. If Ay © Ap C--- and U2, An = A, we say that the A, form an increasing sequence of sets (increasing to A) and write A, t A. Similarly, if Aj D Az D +++ and (2, A, =A, we say that the A, form a decreasing sequence of sets (decreasing to A) and write A, | A. The word “includes” will always imply a subset relation, and the word “contains” a membership relation. Thus if # and @ are collections of sets, “@& includes 7 means that 7 C #. Equivalently, we may say that # contains all sets in #, in other words, each A € is also a member of #. A countable set is one that is either finite or countably infinite. The empty set is the set with no members. The sets A,, i € I, are disjoint if A; NA; = for all i # j.x SUMMARY OF NOTATION 2 Reat NuMBERS The set of real numbers will be denoted by R, and R” will denote n- dimensional Euclidean space. In R, the interval (a, b] is defined as {x € R: a a}; other types of intervals are defined similarly. If a = (a|,...,@n) and b= (bj,...,b,) are points in R",a0, D-co= 00 o= {2 if beR, b<0, aeR CO = is not defined), CO 0-co=~w-0=0. The rules are convenient when developing the properties of the abstract Lebesgue integral, but it should be emphasized that R is not a field under these operations. Unless otherwise specified, positive means (strictly) greater than zero, and nonnegative means greater than or equal to zero. The set of complex numbers is denoted by C, and the set of n-tuples of complex numbers by C”. 3 Functions If f is a function from Q to Q’ (written as f: the preimage of B under f is given by f—1(B)={we€Q: f(w) € B). It follows from the definition that f~'(U; Bi) =U; /-'Bi), f-'(N Bi) =i f7'Bi), f-"A — B) = f(A) — f-"(B); hence f-'(A°) = [FAN If # is a class of sets, f~'(#) means the collection of sets f~'(B), Be #. If f: R > R, f is increasing iff x < y implies f(x) < f(y); decreasing iff x f(y). Thus, “increasing” and “decreasing” do not have the strict connotation. If fy: 2—> R,n =1,2,..., the f, are said to form an increasing sequence iff fn(w) < fn4i(@) for all n and w; a decreasing sequence is defined similarly. Q> 2’) and BCL’,SUMMARY OF NOTATION xi If f and g are functions from & to R, statements such as f < g are always interpreted as holding pointwise, that is, f(w) < g(w) for all o € . Similarly, if fj: Q— R for each i ¢ J, sup, f; is the function whose value at w is sup{fj(w): i € I}. If f\, f2,..- form an increasing sequence of functions with limit f [that is, lim, soc fn(@) = f(@) for all a], we write f, t f. (Similarly, fy | f is used for a decreasing sequence.) Sometimes, a set such as {w € Q: f(@) < g(w)} is abbreviated as {f < g}; similarly, the preimage {w € Q: f(c) € B) is written as {f € B). If A CQ, the indicator of A is the function defined by I4(w) = 1 if o EA and by /4(w) = 0 if w ¢ A. The phrase “characteristic function” is often used in the literature, but we shall not adopt this term here. If f is a function of two variables x and y, the symbol f(x, -) is used for the mapping y > f(x, y) with x fixed. The composition of two functions X: Q — Q' and f: Q! + Q” is denoted by f eX or f(X). If f: 25 R, the positive and negative parts of f are defined by f+ = max(f,0) and f~ = max(—f, 0), that is, re = {1 if f@)>0, if f(@)<09, - —f@) if f@)<0, FoO= {o if fw) >0. 4 Toroocy A metric space is a set Q with a function d (called a metric) from Q x Q to the nonnegative reals, satisfying d(x, y) > 0, d(x, y) =0 iff x = y, d(x, y) =d(y,x), and d(x, z) < d(x, y) + d(y, 2). If d(a, y) can be 0 for x # y, but d satisfies the remaining properties, d is called a pseudometric (the term semimetric is also used in the literature). A ball (or open ball) in a metric or pseudometric space is a set of the form B(x, r) = {y € Q: d(x, y) < r} where x, the center of the ball, is a point of Q, and r, the radius, is a positive real number. A closed ball is a set of the form B(x, r) = {y € Q: d(x, y) a for all a € C. Then S has a maximal element, that is, an element m such that for each a € S it is not possible to have m < a and m # a. Zorn’s lemma is actually an axiom of set theory, equivalent to the axiom of choice.CHAPTER | FUNDAMENTALS OF MEASURE AND INTEGRATION THEORY In this chapter we give a self-contained presentation of the basic concepts of the theory of measure and integration. The principles discussed here and in Chapter 2 will serve as background for the study of probability as well as harmonic analysis, linear space theory, and other areas of mathematics. 1.1 Inrropuction It will be convenient to start with a little practice in the algebra of sets. This will serve as a refresher and also as a way of collecting a few results that will often be useful. Let Ai, Az, ... be subsets of a set 2. If Ay C Ap C-+- and UX, An =A, we say that the A, form an increasing sequence of sets with limit A, or that the A, increase to A; we write A, t A. If Ar D A> D ++ and (2, A, =A, we say that the A, form a decreasing sequence of sets with limit A, or that the A, decrease to A; we write A, | A. The De Morgan laws, namely, (U,, An)° = My AS: (My An)° =U, AS, im- ply that (1) if A, tA, then AS | A’; if A, | A, then AS t A°. It is sometimes useful to write a union of sets as a disjoint union. This may be done as follows: Let Aj, Az, ... be subsets of 2. For each n we have (2) Ujiy Ai = A1 U (AS Ag) U (AE .AS,..A3) Us UAE ++ AS_ | An). Furthermore, 8) Uni An = UR ALN +145) 1 An). In (2) and (3), the sets on the right are disjoint. If the A,, form an increasing sequence, the formulas become2 1 FUNDAMENTALS OF MEASURE AND INTEGRATION THEORY (4) Unt Ai = Ar U 2 = A) UU An = Ant) and (5) Une An = Une An — Ant) (take Ao as the empty set). The results (1)—(5) are proved using only the definitions of union, intersec- tion, and complementation; see Problem 1. The following set operation will be of particular interest. If Aj, Az, ... are subsets of , we define (6) limsup, An = Wy Ura, Ae- Thus w € limsup, A, iff for every n, w € Ay for some k > n, in other words, (7) w € limsup, An iff @ € A, for infinitely many n. Also define (8) liminf, A, = U2; Mo, Ae Thus o € liminf, A, iff for some n, w € A, for all k > n, in other words, (9) w€liminf, A, iff @€A, eventually, that is, for all but finitely many n. We shall call limsup, A, the upper limit of the sequence of sets A,, and liminf,, A,, the lower limit. The terminology is, of course, suggested by the analogous concepts for sequences of real numbers lim sup xp = inf sup xz, n n ken liminfx, = sup inf x,. n n ken See Problem 4 for a further development of the analogy. The following facts may be verified (Problem 5): (10) (limsup, A,,)° = liminf, AS (11) (liming, A,)° = lim sup, AS (12) liminf, A, C lim sup, A, (13) If An tA or Ay A, then liminf, A, = lim sup, Ay = A. In general, if liminf, A, im sup,, An = A, then A is said to be the limit of the sequence Aj, Ao, . e write A = lim, An. Problems 1. Establish formulas (1)—(5). 2. Define sets of real numbers as follows. Let A, = (—1/n, 1] if n is odd, and A, = (—1, 1/n) if n is even. Find limsup, A, and liminf, An. 3. Let 2 = R?, A, the interior of the circle with center at ((—1)"/n, 0) and radius 1. Find lim sup, A, and lim inf, An.1.2. FIELDS, o-FIELDS, AND MEASURES 3 4. Let {x,} be a sequence of real numbers, and let A, = (—00, x,). What is the connection between tim sup, ,,%n and limsup, An (similarly for lim inf )? 5. Establish formulas (10)—(13). 6. Let A= (a,b) and B = (c,d) be disjoint open intervals of R, and let C, =A ifn is odd, C, = B if n is even. Find lim sup, C, and lim inf, C,. 1.2 Fits, o-Fievps, AND MEASURES Length, area, and volume, as well as probability, are instances of the mea- sure concept that we are going to discuss. A measure is a se? function, that is, an assignment of a number j2(A) to each set A in a certain class. Some structure must be imposed on the class of sets on which yz is defined, and probability considerations provide a good motivation for the type of structure required. If Q is a set whose points correspond to the possible outcomes of a random experiment, certain subsets of Q will be called “events” and assigned a probability, Intuitively, A is an event if the question “Does w belong to A?” has a definite yes or no answer after the experiment is performed (and the outcome corresponds to the point € 82). Now if we can answer the question “Is w € A?” we can certainly answer the question “Is w € A°?,” and if, for each i = 1,...,m, we can decide whether or not w belongs to A;, then we can determine whether or not @ belongs to Ul"; A; (and similarly for (\%_, Ai). Thus it is natural to require that the class of events be closed under comple- mentation, finite union, and finite intersection; furthermore, as the answer to the question “Is @ € Q7” is always “yes,” the entire space 2 should be an event. Closure under countable union and intersection is difficult to justify physically, and perhaps the most convincing reason for requiring it is that a richer mathematical theory is obtained. Specifically, we are able to assert that the limit of a sequence of events is an event; see 1.2.1. 1.2.1 Definitions. Let ¥ be a collection of subsets of a set 2. Then F is called a field (the term algebra is also used) iff 2 € ¥ and F is closed under complementation and finite union, that is, (a) Qe F. (b) IfAeF, then Ac e ¥. (©) If Ai, A2,...44n € F, then Ul, Ai € F. It follows that F is closed under finite intersection. For if Aj,...,An €F, then YE =|Uai] e7. isl If (©) is replaced by closure under countable union, that is,4 1 FUNDAMENTALS OF MEASURE AND INTEGRATION THEORY (@) fA), Ap, ... € F, then UX, A; € F, F is called a o-field (the term o-algebra is also used). Just as above, .¥ is also closed under countable intersection. If F is a field, a countable union of sets in .¥ can be expressed as the limit of an increasing sequence of sets in .7, and conversely. To see this, note that if A=U%, An, then U', A; t A; conversely, if A, tA, then A = U%, An. This shows. that a o- field is a field that is closed under limits of i increasing sequences. 1.2.2 Examples. The largest o-field of subsets of a fixed set Q is the col- lection of all subsets of 2. The smallest o-field consists of the two sets % and Q. Let A be a nonempty proper subset of &, and let .F = {B, 2, A, A°}. Then F is the smallest o-field containing A. For if ¥ is a o-field and A € Y, then by definition of a o-field, 2, J, and A® belong to ¥, hence ¥ Cc ¥. But F is a o-field, for if we form complements or unions of sets in 7, we invariably obtain sets in.¥. Thus .¥ is a o-field that is included in any o-field containing A, and the result follows. If Aj,...,An are arbitrary subsets of Q, the smallest o-field containing Aj,...,An may be described explicitly; see Problem 8. If 7 is a class of sets, the smallest o-field containing the sets of . will be written as o(.%), and sometimes called the minimal o-field over .7. We also call o(7) the o-field generated by /, and currently this is probably the most common terminology. Let Q be the set R of real numbers. Let ¥ consist of all finite disjoint unions of right-semiclosed intervals. (A right-semiclosed interval is a set of the form (a, b] = {x: a 0 for all A €.¥, pis called a measure on F, a probability measure if (2) = 1. Note that countable additivity actually implies finite additivity. For if (A) = +00 for all A € F, or if w(A) = —00 for all A € F, the result is immediate; therefore assume j1(A) finite for some A € . By considering the sequence A,B, O, ..., we find that .() = 0, and finite additivity is now established by considering the sequence Aj, ...,An,9,,..., where Aj,...,An are disjoint sets in 7. Although the set function given by 4(A) = +00 for all A € F satisfies the definition of a measure, and similarly (A) = —oo for all A € F defines a countably additive set function, we shall from now on exclude these cases. Thus by the above discussion, we always have (0) = 0. If A €¥ and (A°) = 0, we can frequently ignore A‘; we say that y is concentrated on A. 1.2.4 Examples. Let Q be any set, and let ¥ consist of all subsets of Q. Define (A) as the number of points of A. Thus if A has n members, n=0,1,2,..., then (A) =n; if A is an infinite set, (A) = 00. The set function jz is a measure on ¥, called counting measure on Q. A closely related measure is defined as follows. Let Q = {x1, x2, ...} be a finite or countably infinite set, and let p;, p2,...be nonnegative numbers. Take ¥ as all subsets of , and define H(A) = > pi. med Thus if A = {xi,, iy ..-}, then (A) = pi, + piz +--+. The set function pz is a measure on ¥ and y{x;} = p;,i = 1,2,.... A probability measure will be obtained iff >; p; = 1; if all pj = 1, then y is counting measure. Now if A is a subset of R, we try to arrive at a definition of the length of A. If A is an interval (open, closed, or semiclosed) with endpoints a and }, it is reasonable to take the length of A to be (A) = b — a. If A is a complicated set, we may not have any intuition about its length, but we shall see in Section 1.4 that the requirements that jz(a, b] = b —a for all a,b € R, a < b, and that w be a measure, determine jz on a large class of sets. Specifically, j is determined on the collection of Borel sets of R, denoted by .#(R) and defined as the smallest o-field of subsets of R containing all intervals (a, b], a,b € R. Note that .4(R) is guaranteed to exist; it may be described (admittedly in a rather ethereal way) as the intersection of all o-fields containing the intervals1.2 FIELDS, o-FIELDS, AND MEASURES 7 (a, b]. Also, if a o-field contains, say, all open intervals, it must contain all intervals (a, b], and conversely. For cS co 1 1 (a,bl=(} (ao+2) and (a,b)= [J (ao-<]. n=l n=I Thus .4(R) is the smallest o-field containing all open intervals. Similarly we may replace the intervals (a, b] by other classes of intervals, for instance, all closed intervals, all intervals [a, b), a,b € R, all intervals (a, 00), a € R, all intervals [a, 00), a € R, all intervals (—oo, b), b € R, all intervals (—oo, b],b € R. Since a o-field that contains all intervals of a given type contains all inter- vals of any other type, .@(R) may be described as the smallest o-field that contains the class of all intervals of R. Similarly, .@(R) is the smallest o-field containing all open sets of R. (To see this, recall that an open set is a count- able union of open intervals.) Since a set is open iff its complement is closed, A(R) is the smallest o-field containing all closed sets of R. Finally, if o is the field of finite disjoint unions of right-semiclosed intervals (see 1.2.2), then -B(R) is the smallest o-field containing the sets of o. Intuitively, we may think of generating the Borel sets by starting with the intervals and forming complements and countable unions and intersections in all possible ways. This idea is made precise in Problem 11. The class of Borel sets of R, denoted by .4(R), is defined as the smallest o-field of subsets of R containing all intervals (a, b],a,b € R. The above discussion concerning the replacement of the right-semiclosed intervals by other classes of sets applies equally well to R. If E € .@(R),.B(E) will denote {B € .@(R): BC E}; this coincides with {ANE: A € .A(R)} (see 1.2.2). We now begin to develop some properties of set functions. 1.2.5 Theorem. Let y be a finitely additive set function on the field 7. (a) w@) =0. (bd) (AUB) + (ANB) = pA) + w(B) for all A, Be F. (c) IfA, Be F and BCA, then w(A) = 4(B) + uA — B) (hefce (A — B) = u(A)— (B) if (B) is finite, and p(B) < (A) if “(A — B) > 0).8 1 FUNDAMENTALS OF MEASURE AND INTEGRATION THEORY (d) If » is nonnegative, If y is a measure, oo oo w| An} sun) n=l n=l for all Ay, A2,... € ¥ such that YU", A, € F. Proor. (a) Pick A € ¥ such that (A) is finite; then (A) = H(A UB) = uA) + nO). (b) By finite additivity, BA) = W(ANB) + w(A-B), H(B) = (ANB) + (BA). Add the above equations to obtain H(A) + W(B) = WAN B) + [w(4 — B) + w(B - A) + WAN B)] = WAN B) + (AUB). (c) We may write A= BU (A —B), hence (A) = 1(B) + (A — B). (a) We have UAi = 41 U AG Ag) U (AG AS 1.43) UU (AE MAS) An) isl [see Section 1.1, formula (2)]. The sets on the right are disjoint and M(ACN---MAS_)MAn) M(A) asn > oo. (b) If Aj, A2,...€4%,An J A, and (A1) is finite [hence u(A,) is fi- nite for all n since (A;) = w(A,) + “(Ai —A,)], then (A,) > uA) as n> co. The same results hold if # is only assumed to be a field, if we add the hypothesis that the limit sets A belong to ¥. [If A ¢ F and yu > 0, 1.2.5(¢) implies that j.(A,,) increases to a limit in part (a), and decreases to a limit in part (b), but we cannot identify the limit with 44(A).] Proor. (a) If “(A,) = 00 for some n, then (A) = (An) + L(A — An) = 00+ (A —A,) = oo. Replacing A by A; we find that (Ax) = 00 for all k > n, and we are finished. In the same way we eliminate the case in which “(A,) = —0o for some n. Thus we may assume that all (A,,) are finite. Since the A, form an increasing sequence, we may use identity (5) of Section 1.1: ASA, U (Ap —A1)U ++ U (Ay — Anat) Ue Therefore, by 1.2.5(c), M(A) = HEA1) + w(A2) = HEAL) + + MA) = WAn—1) +27 = Jim #(A,). (b) If A, | A, then Ay —A, t A; —A, hence w(A; — An) > “(Ai — A) by (a). The result now follows from 1.2.5(c). O10 1 FUNDAMENTALS OF MEASURE AND INTEGRATION THEORY We shall frequently encounter situations in which finite additivity of a partic- ular set function is easily established, but countable additivity is more difficult. It is useful to have the result that finite additivity plus continuity implies countable additivity. 1.2.8 Theorem. Let y be a finitely additive set function on the field ¥. (a) Assume that yz is continuous from below at each A € F, that is, if Aj, A2,-.. €F, A=Up An € F, and A, ¢ A, then 4(A,) — (A). It fol- lows that jz is countably additive on F. (b) Assume that j1 is continuous from above at the empty set, that is, if Ay, A2,...,€F and A, | G, then (A,) > 0. It follows that 2 is countably additive on F. Proor. (a) Let Aj, A2,...be disjoint sets in % whose union A belongs to F. Wf By =U, Ai then B, t A, hence (Bn) > 44(A) by hypothesis, But M(B) = 7%, w(Ai) by finite additivity, hence (A) = limy+oo 02", w(Ai), the desired result. (b) Let Aj, Az, ...be disjoint sets in.F whose union A belongs to ¥, and let By = Ul, Ai. By 1.2.5(c), (A) = K(B,) + W(A — B,); but A— By J B, so by hypothesis, 4(A — B,) > 0. Thus (B,,) > (A), and the result follows as in (a). O If 4; and jz2 are measures on the o-field %, then 4 = j4; — [42 is countably additive on , assuming either j1; or /1 is finite-valued. We shall see later (in 2.1.3) that any countably additive set function on a o-field can be expressed as the difference of two measures. For examples of finitely additive set functions that are not countably addi- tive, see Problems 1, 3, and 4. Problems 1. Let Q be a countably infinite set, and let ¥ consist of all subsets of Q. Define (A) = 0 if A is finite, 4(A) = 00 if A is infinite. (a) Show that y is finitely additive but not countably additive. (b) Show that Q is the limit of an increasing sequence of sets A,, with “w(An) = 0 for all n, but (2) = 00. 2. Let w be counting measure on Q, where Q is an infinite set. Show that there is a sequence of sets A, | & with lim,.o. U(An) # 0. 3. Let 2 be a countably infinite set, and let ¥ be the field consisting of all finite subsets of Q and their complements. If A is finite, set 4(A) = 0, and if A® is finite, set w(A) = I. (a) Show that 12 is finitely additive but not countably additive on F.1.2 FIELDS, o-FIELDS, AND MEASURES ll (b) Show that Q is the limit of an increasing sequence of sets A, € .¥ with 4(A,) =0 for all n, but u(Q) = 1. 4. Let F be the field of finite disjoint unions of right-semiclosed intervals of R, and define the set function 4 on ¥ as follows. L(—00, a] = a, aeéR, u(a,bl=b-a, a,beR, ax 0 write, inductively, B= (Utes B 0. 1.3. Extension or MEasures In 1.2.4, we discussed the concept of length of a subset of R. The problem was to extend the set function given on intervals by j(a, b] = b — a toa larger class of sets. If % is the field of finite disjoint unions of right-semiclosed intervals, there is no problem extending jz to Zo: if Aj,...,An are disjoint right-semiclosed intervals, we set .(U_, Ai) = D7, w(A;). The resulting set function on p is finitely additive, but countable additivity is not clear at this point. Even if we can prove countable additivity on 4%, we still have the problem of extending ju to the minimal o-field over .%, namely, the Borel sets. We are going to consider a generalization of the above problem. Instead of working only with length, we shall examine set functions given by j1(a, 6] = F(b) — F(a) where F is an increasing right-continuous function from R to R. The extension technique to be developed is not restricted to set func- tions defined on subsets of R; we shall prove a general result concerning the extension of a measure from a field % to the minimal o-field over %o. It will be convenient to consider finite measures at first, and nothing is lost if we normalize and work with probability measures. 1.3.1 Lemma. Let Yo be a field of subsets of a set Q, and let P be a probability measure on .%o. Suppose that the sets Aj, Az, ... belong to % and1.3. EXTENSION OF MEASURES 13 increase to a limit A, and that the sets A)’, A,’,... belong to .% and increase to A’. (A and A’ need not belong to 9.) If A C 4’, then lim P(Am) < lim, P(A,'). Thus if A,, and A,’ both increase to the same limit A, then slim P(An) = lim P(A,’). Proor. If m is fixed, Am N Ap’ t Am OMA’ = Am as n — 00, hence P(Am On’) > PlAm) by 1.2.7(a). But P(Am M Aj’) < P(A,’) by 1.2.5(c), hence P(An) = lim P(Am Mn’) < lim P(An’)- Let m > oo to finish the proof. 0 We are now ready for the first extension of P to a larger class of sets. 1.3.2 Lemma. Let P be a probability measure on the field %. Let ¥ be the collection of all limits of increasing sequences of sets in .%o, that is, Ae F iff there are sets A, €.%, n = 1,2,..., such that A, ¢ A. (Note that ¥ can also be described as the collection of all countable unions of sets in .%; see 1.2.1.) Define » on ¥ as follows. If A, €.%, n =1,2,..-,An tA (EF), set BA) = lim,..0 P(An); is well defined by 1.3.1, and wp = P on Yo. Then: (a) BEF and pG)=0; Qe ¥ and w(Q) = 1; 0< p{A) < 1 for all Ac, (b) If G,, G2 € ¥, then G; UG), G; NG, € ¥ and MG, U G2) + W(G, N Ga) = WG) + (G2). ©) If G,, G2 € ¥ and G; C Go, then u(Gi) < u(G2). d) IfG,€¥,n=1,2,..., and G, t G, then G € ¥ and p(G,) > UG). Proor. (a) This is clear since 4 = P on %o and P is a probability measure. (b) Let Ani € Fo, Ani t Gis Anz € Fo, Anz t Gz. We have P(An UAn2) + P(Ani QAn2) = P(Ani) + P(An2) by 1.2.5(b); let n > 00 to complete the argument. (c) This follows from 1.3.1.14 1 FUNDAMENTALS OF MEASURE AND INTEGRATION THEORY (d) Since G is a countable union of sets in %, Ge ¥. Now for each n we can find sets Anm €%0, m = 1,2,..., with Anm t Gn as m—> oo. The situation may be represented schematically as follows: Au Ai Aim et Gi An An os Aamo TG Am Any os Anm ot Ge Let Dm = Aim U Am U-+--UAmm (the Dj, form an increasing sequence). The key step in the proof is the observation that Anm C Dm C Gm for n 00 to conclude that D,, + G, hence P(Dm) > 4(G) by definition of 4. Now iet m > oo in (2) to obtain (Gy) < liMp+co P(Dm) < liMy.90 (Gm); then let n > 00 to conclude that limp ..¢ (Gn) = littm +00 P(Dx) = 4(G). 0 We now extend yz to the class of all subsets of 2; however, the extension will not be countably additive on all subsets, but only on a smaller o-field. The construction depends on properties (a)—(d) of 1.3.2, and not on the fact that 2 was derived from a probability measure on a field. We express this explicitly as follows: 1.3.3 Lemma. Let ¥ be a class of subsets of a set 2, a nonnegative real-valued set function on ¥ such that ¥ and yp satisfy the four conditions (a)-(d) of 1.3.2. Define, for each A C Q, w*(A) = inf(u(G): GE Y, GDA}. Then: (a) wt =pon ¥,0< (A) <1 forall A CQ. (b) w(AUB)+ p(ANB) < (A) + 4*(B); in particular, .*(A) + UA’) > L*(Q) + WB) = U2) + WG) = 1 by 1.3.2).