The main objective of this project is to find lane lines in video using basic computer vision. The lines on the road show us where the lanes are and act as a constant reference for where to steer the vehicle. Naturally, one of the first things we would like to do in developing a self-driving car is to automatically detect lane lines using an algorithm.In this project I have created a pipeline to detect lane lines in images and videos using Python and OpenCV.
Because this is my first project I will also explain how to set up your computer and the tools required. Be warned that this post might be long, its however worth it.
- Jupyter Notebook
I have derrived this project from the Udacity’s Starters Kit (below). The kit equips you with all you need to get started.
The purpose of this project is to provide unified software dependency support for students enrolled in Term 1 of the Udacity Self-Driving Car Engineer Nanodegree.
Python 3 is used for the entirety of term 1.
There are two ways to get up and running:
Supported Sytems: Linux (CPU), Mac (CPU), Windows (CPU)
|More straight-forward to use||AWS or GPU support is not built in (have to do this yourself)|
|More community support||Implementation is local and OS specific|
|More heavily adopted|
- Setup python development environment with Miniconda and Jupyter Notebook
- Understand Canny Edge Detector and Hugh Transform
- Understand Numpy and OpenCV
Installing and Configuring Miniconda(Conda)
Conda is an open source package management system and environment management system that runs on Windows, macOS and Linux. Conda quickly installs, runs and updates packages and their dependencies. Conda easily creates, saves, loads and switches between environments on your local computer. It was created for Python programs, but it can package and distribute software for any language.
Download and install miniconda then go to downloads directory on your computer and run this script
This is the environment where we will be working from.
conda create -n lanedetection
Now lets activate our conda environment
source activate lanedetection
After finishing your project detactivate it using this command. Well don’t deactivate it now until we are done.(But you can try if you want)
source deativate lanedetection
Configure Your New Environment
Now while inside the environment, your prompt should look like the following:
Good, now lets proceed to install the required packages
conda install -c https://conda.anaconda.org lmenpo opencv3
This will install 29 packages 6,604 files(16.4MB) so be patient, also install the folowing:
conda install -c asmevrer pango
Then run …
pip install jupyter mousepy matplotlib
If you encounter and error, make sure that your linux has the folowing packages installed. Deactivate your conda environment and install them
sudo pacman -S libselinux libpng12
Finally to start the project, start the Jupyter Notebook
(lanedetection)[wilfred@githuka] jupyter notebook
In a summary the following will be our pipeline:
- Import Initial Images
- Create Helper Functions
- Load Test Images
- Convert Images to Grayscale
- Apply Gaussian Smoothing
- Apply Canny Transform
- Apply Region of interest
- Apply Hough Transform
- Merge Original Image with Lines
- Video Test
This is the final image that we hope to accomplish at the end of this project.
The first step to working with our images is to convert them to grayscale. This is very importanant especially since we will be using Canny Edge Detector of OpenCV. What we are basically doing is collapsing the 3 channels of RGB into a singe channel with a pixel range of [0,255] using this code:
gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
Solving the Yellow~White Lane Markings Problem.
Before diving straight into the Canny Edge Detector, I have realised that Yellow and White in grayscale are almost similar. Infact when you look at the above images, its certainly difficult to know which is white or yellow line. So we will convert it onto a HSV (Hue Value Saturation) HSV is the tech that enabled monochrome tvs to be able to receive colour signals. We will then apply a mask to the RGB image and return the pixels that we want.
lower_yellow = np.array([20, 100, 100], dtype = “uint8”) upper_yellow = np.array([30, 255, 255], dtype=”uint8") mask_yellow = cv2.inRange(img_hsv, lower_yellow, upper_yellow) mask_white = cv2.inRange(gray_image, 200, 255) mask_yw = cv2.bitwise_or(mask_white, mask_yellow) mask_yw_image = cv2.bitwise_and(gray_image, mask_yw)
We will apply a quick gaussian_blur to help us later in our Canny Edge Detector works.
kernel_size = 5 gauss_gray = gaussian_blur(mask_yw_image,kernel_size)
So far so good :-)
A Small Distraction
If you are fascinated by gray_image in openCV like me, here is a small python program that used openCV to apply graysc1aling to an image.
import cv2 image = cv2.imread('sampleimage.png') gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) cv2.imwrite ('gray_image.png', gray_image) cv2.imshow ('gray_image', gray_image) cv2.waitKey (0) cv2.destroyAllWindows()
Canny Edge Detection
Simply put the Canny edge detector is an edge detection operator that uses a multi-stage algorithm to detect a wide range of edges in images. It was developed by John F. Canny in 1986. Canny also produced a computational theory of edge detection explaining why the technique works. John Canny himself recommended a low to high threshold ratio of 1:2 or 1:3 for our canny as it computes.
low_threshold = 50 high_threshold = 150 canny_edges = canny(gauss_gray,low_threshold,high_threshold)
Image output with a Canny Edge Detector applied.
Region of Interest
The region of interest is where te car will be focussing on. The last thing we want is for our car’s attention to be distracted by every thing in the horizon. Everything outside the ROI will be blacked out to zero. Here is the snipet of code that does all the heavylifting…
def maskAction(img): ysize = img.shape xsize = img.shape region = np.array([ [0, ysize], [xsize/2,(ysize/2)+ 10], [xsize,ysize] ], np.int32) return region_of_interest(img, [region]) testImagesMasked = doSaveAndDisplay(testImagesCanny, 'test_images_region', testImageNames, maskAction)
Image output with a ROI defined. This is what the car will concentrate on.
The Hough transform is a technique which can be used to isolate features of a particular shape within an image.Hough transform is most commonly used for the detection of regular curves such as lines, circles, ellipses, etc.
The big take away is that in XY space lines are lines and points are points, but in Hough space lines correspond to points in XY space and points correspond to lines in XY space. This is what our pipeline will look like:
- Pixels are considered points in XY space.
python hough_lines()transforms these points into lines inside of Hough space.
- Wherever these lines intersect, there is a point of intersection in Hough space.
- The point of intersection corresponds to a line in XY space. Thanks @galen.ballew
Image output with Hough Transform applied
Merging Original Image with Lines
Now we plug in our original image with the Lines, this is what I got…
How about Video?
We can also try it out on video, have a look below..
This project has actually introduced me into the deeper powers of python and computer vision. I still have an urge to use my own local images in this project. Thats for next time.
All the files for this project can be found in the repo