This thesis studies the problem of estimating the necessary information about the environment in order to make a robotic arm being able to learn autonomously simple control tasks. The state is estimated from a stream of images using a deep neural network. The network is divided into two models that estimate the state in parallel. The first one is an autoencoder like network that maps images to the states. The second one is a simple prediction model that infers the state from the previous one. On top of that, a model is proposed to estimate the uncertainty for each feature of the state estimated from the images. The two estimated states are joined together with this uncertainty measure to give a single robust state of the environment. To assess the performances of the models, a number of experiments are conducted in a simulation environment. Results show that a representation learned with this model is generally more robust to visual occlusion in the images. Finally, the estimated state is used to learn a control policy for the robotic arm to perform a simple task using reinforcement learning.