This paper addresses intention and trajectory prediction of exo-vehicles in an urban driving environment. Urban environments pose challenging scenarios for self-driving cars, specifically pertaining to traffic light detection, negotiating paths at the intersections and sometimes even overtaking illegally parked cars in narrow streets. This complex task of autonomously driving while considering anomalous situations make urban driving conditions unique when compared to highway driving. In order to overcome these challenges, we propose to use road contextual information to predict driving intentions and trajectories of surrounding vehicles. The intention prediction is obtained using a recurrent neural network and the trajectory is predicted using a polynomial model fitting of the past lateral and longitudinal components of the vehicle poses and road contextual information. The integrated process of intention and trajectory prediction is performed in real-time by deploying and testing on a self-driving car in a real urban environment.