We present a method for estimating Markov dynamic models with unobserved state variables which can be serially correlated over time. We focus on the case where all the model variables have discrete support. Our estimator is simple to compute because it is noniterative, and involves only elementary matrix manipulations. Our estimation method is nonparametric, in that no parametric assumptions on the distributions of the unobserved state variables or the laws of motions of the state variables are required. Monte Carlo simulations show that the estimator performs well in practice, and we illustrate its use with a dataset of doctors prescription of pharmaceutical drugs.