A Monte Carlo method is a computational / numerical method that uses random numbers to compute / estimate a quantity of interest. The quantity of interests may be the mean of a random variable , functions of several means , distributions of random variables or high dimensional integrals.
Basically, Monte Carlo methods may be grouped into two types:
The direct/simple/classical Monte Carlo methods involves generating identical and independent sequences of random samples. And the other, in a sense, involves generating a sequence of random samples, which are not independent, and is the Markov Chain Monte Carlo methods.
Monte Carlo Integration
Let f(x) be a function of x and suppose we are interested in computation of the integral:
I= \int_0^1 f(x) dx
We can write the integral as,
I=\int_0^1 f(x) p(x) dx =E(f(X))\\
\textit{; where p(x) is the pdf of a r.v. X $\sim$ Unif(0,1)} Now suppose that x_1,x_2,\dots,x_n are independent random samples drawn from Uniform(0,1), then by the law of large number we have,
\frac{1}{n} \sum_{i=1}^n f(x_i) \rightarrow E(f(X)) Thus an estimator of I may be:
\hat{I}=\frac{1}{n} \sum_{i=1}^n f(x_i)On a more general note, if a < b < \infty then,
I= \int_a^b f(x) dx \\
\textit{Taking $y=\frac{x-a}{b-a}$}\\
I= \int_0^1 (b-a) f \left(\frac{a+(b-a)y}{b} \right) dy \\
= \int_0^1 h(y) dy \quad dy \\ \text{;where } h(y)=f \left(\frac{a+(b-a)y}{b} \right) \\
= E\left[ h(Y) | Y \sim Unif(0,1) \right]And when b=\infty ,
I= \int_a^\infty f(x) dx \\
\text{Taking $y=\frac{1}{x+1}$} \\
I= - \int_1^0 f\left( \frac{1}{y} -1 \right) \frac{dy}{y^2} \\
= \int_0^1 h(y) dy \\
\text{; where } h(y) = f\left( \frac{1}{y} -1 \right)/y^2 \\
=E(h(Y) | Y \sim Unif(0,1))Some advantages of using Monte Carlo Methods:
- Estimates could be obtained without any hardcore assumptions.
- The rate of convergence of a Monte Carlo estimator may not be in par with that obtained by other methods. However, whenever one is dealing with high dimensional integrals, Monte Carlo methods always provides estimators that ultimately converges to the exact value.
Leave a Reply