Autonomous vehicle technology will revolutionize the way how we live, move, work and entertain ourselves.  The autonomous vehicles will have the direct impact on our society and the way we live.  The autonomy means "autos" (self) and "nomos (rule). The definition of autonomy is the ability of an individual to make a rational, uninfluenced decision. The autonomous vehicle must follow the principles of autonomy like humans, making fair decisions at all situations. To navigate to a given destination based on passenger provided instructions, autonomous vehicle must observe, understand, model, infer, and predict behavior of the surrounds like, weather, animals, debris, follow the traffic rules and signals,  and the humans inside and outside the car and finally make a split-second decision. The social, ethical, emotional, psychological, economical, legal, and geographical effects are enormous.  These aspects must be considered for the development of the autonomous vehicle. This paper would examine the challenges in developing autonomous vehicle.