The sampling process results in an error called the Monte Carlo error. So this error is a measure of a sampler, which stems from the fact that not all samples that we have drawn are independent. This error is defined by dividing our trace into n different blocks, and then we compute the mean of these blocks. We want to calculate the error as the standard deviation of these means, or the square root of the number of blocks, and it's given by this equation shown here. So we've talked about divergence in course two, and we've seen that happening as an undesirable artifact in our sampling process. Divergences happen in regions of high curvature or high gradient, in our sampling manifold. One PyMC3 detects a divergence. It abandons that chain, and as a result, the samples that are reported to have been diverging are close to the space of high curvature, but not necessarily right on it. In some cases, PyMC3 can indicate falsely that some samples are divergences; this is due to the heuristics that we use to identify these divergences. Concentration of samples, in a region, is an indication that this may not be divergence. We visualized this for the centered model and the non-centered model, with pair plots of the variables. Below you can see how the centered model has difficulties sampling at the edge of the funnel shape, two dimensional manifold, formed by the pairplot. So we can generate pairplots by calling, the plot underscored pair method from within others , and you pass the trace object to it along with the variables of interest. Here we're setting divergences equals true, so we can actually visualize where the divergences are. So this is what it looks like for the centered model. So you have a plots of y against both parameters, Mu and Sigma, here on the bottom, and a plots of the parameter Sigma versus Mu on top. So you can see there are quite a bit of divergences, and they're all over the place for the centered model. We do the same for the non-centered model, and we can see that there are far fewer divergences here compared to the centered model. When you have multi-dimensional data, it can also use a parallel coordinates plot to look at the relationship between the variables, and to also look for divergences. If we notice tight-knit lines around the region; this is usually an indication of difficulties sampling, and hence, it might indicate divergences. So you can call the plot parallel method for within others to produce these parallel plots. So here we're doing that for the centered model and the non-centered model. So this behavior can be observed in the centered model around zero. While the non-centered model has a sparser cluster of lines, around zero here. Sparser clusters can mean indication of false-positive, where divergences are reported. Apart from reformulating the problem, two ways in which we can avoid the issue of divergences are, increase the tuning samples, and increase the value to target-accept.
