%%%%%%%%%%%%%%%%%%%%
%I) PRELIMINARIES
%%%%%%%%%%%%%%%%%%%%
\documentclass[11pt,reqno]{amsart}
% Packages
%%%%%%%%%%%%%%%%%%
\usepackage{graphicx}%preferred package for inclusion of graphics
\usepackage{comment}
\usepackage{setspace}
\usepackage{enumerate}%for easy choice of enumerator symbol
\usepackage{tabularx}%for tables width user-defined width
\usepackage{ctable} %for toprule, midrule etc. in tables
\usepackage{multirow} %for more flexibility with tables
\usepackage{textcomp}%for cent-symbol
\usepackage[colorlinks=true, urlcolor=blue]{hyperref}%for inserting hyperlinks
\usepackage{caption} %flexibility with tables
\usepackage{subcaption} %flexibility with tables
\usepackage{url}%to get harvard to work, insert immediately before \title{...}
\usepackage[dcucite]{harvard} %bibliography style, dcu gives commas before year, semicolon between references, and "and" between authors
\usepackage{amssymb} %for the more esotheric math expressions, such as \approxeq
\usepackage{lineno}%for line numbers
\usepackage{lscape}%for inserting landscape format pages
\usepackage{float} % for more flexibility with tables
\usepackage{appendix}%allows for turning appendices on and off
\usepackage{epstopdf} %to allow import of .eps graphics
% Page formatting
%%%%%%%%%%%%%%%%%%%%
\pagestyle{plain} %puts page number center bottom
\setlength{\topmargin}{0in}
\setlength{\textheight}{8.5in}
\setlength{\oddsidemargin}{.0in}
\setlength{\evensidemargin}{.0in}
\setlength{\textwidth}{6.5in}
\setlength{\footskip}{.5in}
% Customized commands
%%%%%%%%%%%%%%%%%%%%%
% Math
\newcommand{\mlt}[1]{\mathbf{#1}} %matrix bold for Latin symbols
\newcommand{\mgr}[1]{\boldsymbol{#1}}%matrix bold for Greek symbols
\newcommand{\kl}{\left(}
\newcommand{\kr}{\right)}
\newcommand{\kll}{\left\{}
\newcommand{\krr}{\right\}}
\newcommand{\kmu}{\mgr{\mu}}
\newcommand{\kpsi}{\mgr{\psi}}
\newcommand{\kphi}{\mgr{\phi}}
\newcommand{\kgam}{\mgr{\gamma}}
\newcommand{\ktheta}{\mgr{\theta}}
\newcommand{\kbeta}{\mgr{\beta}}
\newcommand{\kdelta}{\mgr{\delta}}
\newcommand{\kt}{^{\prime}}
\newcommand{\kdel}{\partial}
\newcommand{\kdot}{\kl . \kr}
\newcommand{\keps}{\epsilon}
\newcommand{\kx}{\mlt{x}}
\newcommand{\kX}{\mlt{X}}
\newcommand{\kV}{\mlt{V}}
\newcommand{\ky}{\mlt{y}}
\newcommand{\kb}{\mlt{b}}
\newcommand{\ki}{\mlt{i}}
\newcommand{\klam}{\lambda}
\newcommand{\kp}{\mlt{p}}
\newcommand{\kprob}{\text{prob}}
\newcommand{\kz}{\mlt{z}}
\newcommand{\ksig}{\sigma^2}
\newcommand{\kSig}{\mgr{\Sigma}}
\newcommand{\klog}{\text{log}}
\newcommand{\kols}{\kl \kX\kt\kX\kr^{-1}\kX\kt\ky}
\newcommand{\kSSE}{\kl \ky-\kX\kb\kr\kt\kl\ky-\kX\kb\kr}
\newcommand{\ksp}{\vspace{0.1in}}
%Special font within regular document
\newcommand{\chp}[1]{\textbf{\textsl{#1}}}
\newcommand{\km}[1]{\textsf{\small{#1}}} %special font for my own comments
\newcommand{\mlab}{\textbf{\texttt{Matlab }}}
\newcommand{\ktt}[1]{\textbf{\texttt{#1}}}
%Tables
\newcolumntype{C}{>{\centering\arraybackslash}X} %for centered columns within tabularx,instead of justified (the default)
%Others
\newcommand{\kpm}{PM_{2.5}}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%
\begin{document}
%%%%%%%%%%%%%%%%%%%%%%%%
\citationmode{abbr} %use only "et al" citations
% the following add space before and after equations
\abovedisplayskip=0.7cm
\abovedisplayshortskip=-0.3cm
\belowdisplayskip=0.7cm
\belowdisplayshortskip=0.4cm
%III) TOP MATTER INFORMATION
\title{Final Exam}
\author{AAEC 6564 \\ Instructor: Klaus Moeltner}
\maketitle %this comes at the end of the top matter to set it.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{General Instructions}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Please type everything in LaTeX (including all Math) and hand in a pdf file. For the Matlab part, answer questions in LaTeX, and attach your script, log file, and any graphs to your main pdf file.\\
IMPORTANT: \emph{Independent work} is required for this assignments - no group collaboration, please.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Question 1: Approximating a truncated normal via Accept-Reject Sampling}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Consider the normal density with \textbf{mean 1} and \textbf{variance 1}, truncated to the $[0,4]$ region. I will refer to it henceforth as the \emph{target density}.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\vspace{0.1in}
\begin{flushleft}
PART A
\end{flushleft}
\vspace{0.1in}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
\begin{enumerate}
\item
Using the formulas given in the module 14 lecture notes, compute the expectation, variance, and standard deviation of the truncated distribution (feel free to use Matlab to do the actual computing). Please be precise to the fourth decimal.\\
\item
Compute the integrating factor $c_f$ for this target density, and show the form of its kernel (call it $\tilde{f}\kl \theta\kr$).\\
\item
You would like to use acceptance sampling to approximate this target distribution. Consider the $U[0,4]$ uniform as a source density. Show its integrating constant $c_s$ and its kernel $\tilde{s}\kl \theta \kr$. \\
\item Compute the importance multiplier $\tilde{M}=\underset{0\leq\theta\leq4}{max}\frac{\tilde{f}\kl\theta\kr}{\tilde{s}\kl\theta\kr}$ and the analytical acceptance rate.\\
\item Using script\emph{ mod14\_AR2} as an example, write a Matlab script for an acceptance sampler with 10,000 accepted draws. Make sure to set your random seeds for \emph{rand} and \emph{randn} to ``37'' at the beginning of your script (as usual).\\
Capture the mean and standard deviation of the kept draws and write them into an Excel table, along with the true (analytical) values for the target density. Also add your results for the analytical and empirical acceptance rates.\\
\end{enumerate}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\vspace{0.1in}
\begin{flushleft}
PART B
\end{flushleft}
\vspace{0.1in}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
\begin{enumerate}
\item Stay in the \emph{same script}, do \emph{NOT re-set the random seed}.
\item Now consider a new source density, that is an \textbf{UN}-truncated normal with (mean = mean of truncated target), and (variance =\textbf{ 2 * } variance of truncated target). Compute the integrating constant for this source and show its kernel.\\
\item Compute the importance multiplier and the analytical acceptance rate.\\
\item Implement an acceptance sampler with 10,000 accepted draws.\\
\emph{(Hint: Make sure to discard all draws from the source that fall outside the [0,4] interval.)}\\
Capture the mean and standard deviation of the kept draws and add them into your Excel table, along with the analytical and empirical acceptance rates. \\
\item
Plot the true target density and the two approximations in the same graph. Comment on the relative performance of the approximations. Use the following code to obtain the draws from the true density for plotting:
\begin{verbatim}
pd=makedist('Normal',1,1);
t=truncate(pd,0,4);
ygrid = linspace(0,4,R)';
ytrue = pdf(t,ygrid);
\end{verbatim}
\end{enumerate}
\clearpage
\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Question 2: Approximating a truncated normal via a 3-Component Mixture Model}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Now suppose you want to approximate the same target density as before with a mixture-of-normals distribution, using a Gibbs Sampler.
\begin{enumerate}
\item Start a \emph{new script}, and \emph{re-set the seeds to 37}. Take 10,000 draws from the truncated normal, using the following code:\\
\begin{verbatim}
n=10000; %sample size
pd=makedist('Normal',1,1);
t=truncate(pd,0,4);
y=random(t,n,1);
\end{verbatim}
\vspace{0.1in}
\item Using the \emph{mod10\_2CMMMix} script as an example, write a Gibbs /sampler for a THREE-component mixture-of-normals model. You will have to adjust the Gibbs Sampler function \emph{gs\_2cmm} to accommodate the third component (call the modified function \emph{gs\_3cmm}). Use the following starting values, priors, and tuners:\\
\begin{verbatim}
% general elements
r1=10000; % burn-ins
r2=10000; % keepers
R=r1+r2;
% generic OLS
bols=inv(X'*X)*X'*y;
res=y-X*bols;
s2=(res'*res)/(n-k);
% prior for betas
mu0=0;
V0=eye(k)*10;
betadraw=bols;
% prior for sig2's
sig2draw=s2;
v0=1/2;
tau0=1/2;
%prior for cp
acp0=[2;2;2];
%draw starting vector of cp
cpdraw=[1/3;1/3;1/3];
%draw starting vector of indicators
Zdraw=mnldraws(repmat(cpdraw',n,1));
%n by 3 - my own function to draw from the MNL - much faster than Matlab's
\end{verbatim}
\vspace{0.1in}
\item In the \emph{same script}, and \emph{without re-setting the random number seed}, generate mixed prediction vector $yp$ as in script \emph{mod10\_2CMMplots} and capture its mean and standard deviation. Add these values to your Excel table from the previous question. \\
\item Plot $yp$ against the true distribution (generate the true draws as you did in the previous question). Comment on the plot and compare it to that from the previous question. How does the 3CMM compare to the A-R approaches from Q1 in fitting the target density?
\end{enumerate}
\clearpage
\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Question 3: Approximating a truncated normal via Metropolis-Hastings Sampling}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Of course, you can also use MH to approximate a free-standing density (not just conditional posteriors, as we have done so far). All you need is the kernel of the otherwise unknown target density, plus truncation bounds, if any. Here we will consider a simple RWC version of the MH to approximate the truncated normal
\begin{enumerate}
\item Start a \emph{new script}, and \emph{re-set the seeds to 37}. Take again 10,000 draws from the truncated normal, using the following code:\\
\begin{verbatim}
n=10000; %sample size
pd=makedist('Normal',1,1);
t=truncate(pd,0,4);
y=random(t,n,1);
\end{verbatim}
\vspace{0.1in}
\item Loop over R=100,000 iterations, with starting draw taken at random from the target density. Set the tuner std for the MH to 1:
\begin{verbatim}
R=100000;
ydraw=random(t,1,1);
ystd=1;
\end{verbatim}
\item At each iteration, draw a candidate \emph{ycan} from the simple normal with mean=\emph{yold}, and std = \emph{ystd}. \\
\item After drawing \emph{ycan}, add an ``if'' condition to tell Matlab to only proceed if \emph{ycan} is within the truncation bounds of the target ([0,4]);\\
\item Derive the logged acceptance probability and decide if you keep the new draw or stick with the old (note: (i) there are no priors involved here, just the kernel of the target density), (ii) if \emph{ycan} falls outside the truncation bounds, you should automatically stick with \emph{yold}.)\\
\item While collecting your draws of y, also keep track of your acceptance count (= how often you accept the candidate).\\
\item Capture the mean and std of your draws of \emph{y}, as well as the acceptance rate (= acceptance count/ R) and add them to your Excel table from the previous question(s). Which approach provided the best approximation to the true moments? Please make sure to attach the Excel table to your output.\\
\item Plot your drawn \emph{y}'s against the true distribution (generate the true draws as you did in the previous question). Comment on the plot and compare it to that from the previous questions. How does the MH approach compare to the previous ones in approximating the true target density?
\end{enumerate}
\end{document}