This paper investigates an algorithm for fault tolerant control of uncertain robot manipulator with only joint position measurement using neural network and second-order sliding mode observer. First, a neural network (NN) observer is designed to estimate the modeling uncertainties. Based on the obtained uncertainty estimation, a second-order sliding mode observer is then designed for two purposes: 1) Providing the velocity estimation, 2) providing the fault information that is used for fault detection, isolation and identification. Finally, a fault tolerant control scheme is proposed for compensating the effect of uncertainties and faults based on the fault estimation information. Computer simulation results on a PUMA560 industrial robot are shown to verify the effectiveness of the proposed strategy.