A Monte Carlo method is a computational / numerical method that uses random numbers to compute / estimate a quantity of interest. The quantity of interests may be the mean of a random variable , functions of several means , distributions of random variables or high dimensional integrals.
Basically, Monte Carlo methods may be grouped into two types:
The direct/simple/classical Monte Carlo methods involves generating identical and independent sequences of random samples. And the other, in a sense, involves generating a sequence of random samples, which are not independent, and is the Markov Chain Monte Carlo methods.
Monte Carlo Integration
Let f(x) be a function of x and suppose we are interested in computation of the integral:
I= \int_0^1 f(x) dx
We can write the integral as,
I=\int_0^1 f(x) p(x) dx =E(f(X))\\
\textit{; where p(x) is the pdf of a r.v. X $\sim$ Unif(0,1)} Now suppose that x_1,x_2,\dots,x_n are independent random samples drawn from Unit(0,1), then by the law of large number we have,
\frac{1}{n} \sum_{i=1}^n f(x_i) \rightarrow E(f(X)) Thus an estimator of I may be:
\hat{I}=\frac{1}{n} \sum_{i=1}^n f(x_i)On a more general note, if a < b < \infty then,
I= \int_a^b f(x) dx \\
\textit{Taking $y=\frac{x-a}{b-a}$}\\
I= \int_0^1 (b-a) f \left(\frac{a+(b-a)y}{b} \right) dy \\
= \int_0^1 h(y) dy \quad dy \\ \text{;where } h(y)=f \left(\frac{a+(b-a)y}{b} \right) \\
= E\left[ h(Y) | Y \sim Unif(0,1) \right]And when b=\infty ,
I= \int_a^\infty f(x) dx \\
\text{Taking $y=\frac{1}{x+1}$} \\
I= - \int_1^0 f\left( \frac{1}{y} -1 \right) \frac{dy}{y^2} \\
= \int_0^1 h(y) dy \\
\text{; where } h(y) = f\left( \frac{1}{y} -1 \right)/y^2 \\
=E(h(Y) | Y \sim Unif(0,1))