Review of Environment Perception for Intelligent Vehicles

Overview of environment perception for intelligent vehicles supposes to the state-of-the-art algorithms and modeling methods are given, with a summary of their pros and cons. A special attention is paid to methods for lane and road detection, traffic sign recognition, vehicle tracking, behavior analysis, and scene understanding. Integrated lane and vehicle tracking for driver assistance system that improves on the performance of both lane tracking and vehicle tracking modules. Without specific hardware and software optimizations, the fully implemented system runs at near-real-time speeds of 11 frames per second. On-road vision-based vehicle detection, tracking, and behavior understanding. Vision based vehicle detection in the context of sensor-based on-road surround analysis. We detail advances in vehicle detection, discussing monocular, stereo vision, and active sensor–vision fusion for on-road vehicle detection. The traffic sign detection detailing detection systems for traffic sign recognition (TSR) for driver assistance. Inherently in traffic sign detection to the various stages: segmentation, feature extraction, and final sign detection.


INTRODUCTION
Research and development on environmental perception, advanced sensing, and intelligent driver assistance systems aim at saving human lives. A wealth of research has been dedicated to the development of driver assistance systems and intelligent vehicles for safety enhancement. For the purposes of safety, comfortability, and saving energy, the field of intelligent vehicles has become a major research and development topic in the world.
Many government agencies, academics, and industries invest great amount of resources on intelligent vehicles, such as Carnegie Mellon University, Stanford University, Cornell University, University of Pennsylvania, Oshkosh Truck Corporation, Peking University, Google, Baidu, and Audi. Furthermore, many challenges have been held to test the capability of intelligent vehicles in a real world environment, such as DARPA Grand Challenge, Future challenge, and European Land-Robot Trial.
Intelligent vehicles are also called autonomous vehicles, driverless vehicles, or self-driving vehicles. An intelligent vehicle enables a vehicle to operate autonomously by perceiving the environment and implementing a responsive action. It comprises four fundamental technologies: environment perception and modeling, localization and map building, path planning and decision-making, and motion control as shown in Figure.   One main requirement to intelligent vehicles is that they need to be able to perceive and understand their surroundings in real time. It also faces the challenge of processing large amount of data from multiple sensors, such as camera, radio detection and ranging (Radar), and light detection and ranging (LiDAR). For intelligent vehicles, data are usually collected by multiple sensors, such as camera, Radar, LiDAR, and infrared sensors. [1] After preprocessing, various features of objects from the environment, such as roads, lanes, traffic signs, pedestrians and vehicles, are extracted. Both static and moving objects from the environment are being detected and tracked. Some inference can also be performed, such as vehicle behavior and scene understanding.
The sensors in intelligent vehicles can be divided into internal and external sensors. The information of an ego vehicle can be obtained by internal sensors, such as engine temperature, oil pressure, battery and fuel levels. External sensors measure objects of the ego vehicle's surroundings, such as lanes, roads, other vehicles, and pedestrians. External sensors includes Radar, LiDAR, and Vision. Radar is an object detection system. Using the signal of radio waves, it can be used to determine the range, angle, or velocity of objects. Radar is consistent in different illumination and weather conditions. However, measurements are usually noisy and need to be filtered extensively.
LiDAR has been applied extensively to detect obstacle in intelligent vehicles. It utilizes laser light to detect the distance to objects in a similar fashion as Radar system. Compared with Radar, LiDAR provides a much wider field of-view and cleaner measurements. However, LiDAR is more sensitive to precipitation. Vision sensors are suitable for intelligent vehicle. Compared with Radar and LiDAR, the raw measurement of vision sensor is the light intensity. Vision sensor can be grouped as camera, lowlight level night vision, infrared night vision, and stereo vision. [2] In the Internet of vehicles, these sensors can communicate with other vehicles and road infrastructure. Each year, some 1.2 million people die worldwide as a result of traffic accidents. Research into sensing systems for vehicle safety promises safer journeys by maintaining an awareness of the on-road environment for driver assistance. [3] In monocular vehicle detection studies tend to quantify recall of the true positive vehicles and false alarms. Stereo-vision studies tend to focus on estimation accuracy for the motion parameters of a tracked vehicle, including position, velocity, and yaw rate. Even though this paper is mostly concerned with using TSR for driver assistance, TSR has various well-defined applications summarized here: [4] 1) Highway maintenance: Check the presence and condition of signs along major roads. 2) Sign inventory: Similar to the preceding task, create an Inventory of signs in city environments.
3) Driver-assistance systems: Assist the driver by informing of current restrictions, limits, and warnings. 4) Intelligent autonomous vehicles: Any autonomous car that is to drive on public roads must have a means of obtaining the current traffic regulations. This can be done through TSR.

II. VISION-BASED VEHICLE DETECTION, TRACKING, AND BEHAVIOR ANALYSIS
A Computer vision technique is necessary to detect the vehicles in images and video. In which computer vision standpoint, on-road vehicle detection presents raid challenges. The on-road environment is semi-structured, allowing for only weak assumptions to be made about the scene structure. Object detection from a moving platform requires the system to detect, recognize, and localize the object in video, often without reliance on background modeling.
Cameras feature lower costs than active sensors and are already commonly used for tracking lanes, allowing for system integration, shared hardware, and low costs. While active sensors identify objects, vision definitively recognizes objects as vehicles. [2] Vision integrates nicely with active sensor suites, allowing sensors like lidar to provide physical measurements, whereas vision classifies objects as vehicle or non vehicle, The visual domain is also highly intuitive for humans, making vision-based systems attractive for on-road interactivity and driver assistance. The drawbacks to vision-based vehicle detection include sensitivity to light and weather conditions and increased computational cost.
Vehicles on the road are typically in motion, introducing effects of ego and relative motion. There is variability in the size, shape, and color of vehicles encountered on the road. The on-road environment also features variations in illumination, background, and scene complexity. Complex shadowing, man-made structures, and ubiquitous visual clutter can introduce erroneous detections. Vehicles are also encountered in a variety of orientations, including preceding, oncoming, and cross traffic. Furthermore, a vehicle detection system needs to operate at real-time speeds in order to provide the human or autonomous driver with advanced notice of critical situations.

2.1Camera Placement
A Vehicle detection system needs to operate at real-time speeds in order to provide the human or autonomous driver with advanced notice of critical situations. [2] Vehicle detection using a single camera aims to detect vehicles in a variety of locations with respect to the ego vehicle. The vast majority of monocular vehicle detection studies position the camera looking forward, to detect preceding and oncoming vehicles. However, various novel camera placements have yielded valuable insight and safety-critical applications and also main characteristics includes to  In stereo-vision, focus on estimation accuracy for the motion parameters of a tracked vehicle, including position, velocity, and yaw rate.
[2]  In monocular vehicle detection, commonly used to quantify recall of true positive vehicles and false alarms. [2]  It was noted that monocular vision can detect objects that stereo-vision approaches typically miss, such as disambiguation of two objects that lie close together in 3-D space  This problem was addressed by detecting in the monocular plane but localizing in 3-D using stereo vision.

III. INTEGRATED LANE AND VEHICLE DETECTION, LOCALIZATION, AND TRACKING
Annually, between 1% and 3% of the world's gross domestic product is spent on the medical costs, property damage, and other costs associated with automotive accidents. Research into sensing systems for vehicle Safety promises safer journeys by maintaining an awareness of the on-road environment for driver assistance. [3] In computer vision for on-road safety have involved monitoring the interior of the vehicle &the exterior or both. In that mainly focus on monitoring the exterior of the vehicle. Monitoring the exterior can consist of estimating lanes pedestrians vehicles, or traffic signs. Taking a human-centered approach is integral for providing driver assistance using the visual modality allows the driver to validate the system's output and to infer context. However, specifically tests the system in dense traffic, which is known to be a difficult scenario for vision-based driver assistance systems. The fully implemented integrated lane and vehicle tracking system currently runs at 11 frames per second, using a frame resolution of 704 × 480. [2] Table 1 shows the mean absolute error and standard deviation of error over the entire 5000-frame data set, for the lane tracker alone, and for the integrated lane and vehicle tracking system, further a significant increase in robustness to the dynamic on-road conditions presented by dense traffic.

IV. VISION-BASEDTRAFFIC-SIGN-DETECTION AND ANALYSIS FOR INTELLIGENT DRIVER ASSISTANCE SYSTEMS
A survey of traffic sign detection for driver assistance. It is crucial to not only consider the car's surrounding and external environment when designing an assist system but also consider the internal environment and take the driver into account (e.g., seeing speed limit signs). [4] In this paper, our emphasis is on framing the TSR problem in the context of human-centered driver assistance systems. Traffic sign recognition (TSR) research needs to take into account the visual system of the driver. [4], This can include factors such as visual saliency of signs, driver focus of attention, and cognitive load.

2) KUL Belgium Traffic Signs Data set (KUL Data set).
3) Swedish Traffic Signs Data set (STS Data set). 4) RUG Traffic Sign Image Database (RUG Data set).

5) Stereo polis Database.
The absence of usage of public database may not explain in entirety why very few comparative studies of methods exist. Another reason is that TSR systems are long complex chains of various methods, where it is not always possible to swap individual modules.

Result Comparison and Analysis
The comparison of intelligent transport vehicle technologies implemented so far is given in table 2.

V. CONCLUSION
In this study can concluded that the development of environment perception and modeling technology is one of the key aspects for intelligent vehicles. In that provide an overview of the state of the arts of environment perception and modeling technology. First, the pros and cons of vehicular sensors of the applications are to be reviewed. Next, popular modeling methods and algorithms of lane and road detection, traffic sign recognition, vehicle tracking and behavior analysis, and scene understanding are reviewed. Current challenges for environment perception and modeling technology are due to the complex outdoor environments and the need of efficient methods for their perception in real time. The changeable lighting and weather conditions, and the complex backgrounds, especially the presence of occluding objects still represent significant challenges to intelligent vehicles. 1)The fully implemented system runs at near-real-time speeds of 11 frames per second. 1)The STS Data set and the KUL Data set. 2)They are both very large, although not as large as the GTSRB, and they contain full images.
3)This means that they can both be used for detection purposes.
1)At the moment, the problem in TSR is the lack of use of standardized sign image databases.
2)This makes comparisons between contributions very hard. 1) Main application but carries on focusing on speed limit signs. 2). Detection of speed limits is highly relevant for an autonomous vehicle, but as it turns out, humans are already very good at seeing speed limit signs themselves.