Prediction and Planning in Dynamical Systems with Underlying Markov Decision Processes