In this part of the lab, you will implement the statistics collection
needed for reestimating a Gaussian distribution as well as the
actual reestimation algorithm. This will involve filling in
the functions `UpdateCountsRefR`

and
`ReestimateS`

in `Lab2_AM.C`.

More specifically, at the beginning of each iteration through
the training data, all of the Gaussian counts will be initialized
to zero. Then, for each utterance, all of
your calls to `graph.add_count()` during FB
will now pass these posterior counts to
`UpdateCountsRefR`

to let you update Gaussian statistics.
At the end of each iteration through the training data,
the routine `ReestimateS`

will be called for each
Gaussian to let you reestimate these parameters from the counts
you collected. As far as where to find the update equations,
you can look at equations (9.50) and (9.51) on p. 152 of Holmes
or equations (8.55) and (8.56) on p. 396 of HAH. Hint: remember
to use the *new* mean when reestimating the variance. Hint:
remember that we are using diagonal-covariance Gaussians, so you
only need to reestimate covariances along the diagonal of the
covariance matrix.

Your code will be compiled into the training program **TrainLab2**.
To compile this program with your code, type

To run this trainer on some utterances representing isolated digits, run

This script just collects counts for training HMM observation probabilities
from the same mini-training set as before, reestimates Gaussian
parameters, and outputs them to the file

`p3a.oprobs`. The “correct”
output can be found in the file

`p3a.oprobs` in

`~stanchen/e6884/lab2/`; only look at the counts after
the line “

`<gaussians>`”.
Again, it's OK if your output doesn't match exactly, but it should be quite
close.

Once you think you have this part working, run the script

This script reestimates observation probabilities on the training set
(while leaving transition probabilities unchanged),
performing twenty iterations of the forward-backward algorithm
and outputting the average logprob/frame (as computed in the
forward algorithm) at each iteration. If your implementation
of Gaussian reestimation is correct, this logprob should always be increasing
and should look like it's converging. This script will
output trained Gaussian parameters to the file

`p3b.oprobs`.
(Debugging hint: things should still converge
if you update only means and not variances; by commenting out
your variance update, you can see if your mean update looks like
it's working.)

Decode the same test data as in the last part with this trained observation
model (and *untrained* transition model) by running the script

By comparing the word-error rate found here with that found
in the corresponding run in the last part, we can see the
relative importance of transition and observation probabilities.
(BTW, these error rates will be very poor since the
training set is very small; there are some digits that
it does not contain an instance of.)

For further evidence, run the script:

This script starts with the observation model in

`p3b.oprobs`
and trains both the observation and transition probabilities
on the given training set for five iterations, creating the
files

`p3d.oprobs` and

`p3d.tprobs`. It then
decodes the same test set as before with these new models.