Introduction to Computer Vision with the OpenCV Library on Linux
https://linuxconfig.org/introduction-to-computer-vision-with-opencv-on-linux
Introduction to Computer Vision with the OpenCV Library on Linux
1. Abstract
Conventions used in this article:
- $ - execution on the command line by a non-privileged user
- # - execution on the command line by a superuser
- the actual command to be executed on the command line or code of program to be compiled
- OUTPUT: output produced on the command line by command execution
- NOTE: general notes and additional information
2. Introduction
In simple words a Computer Vision is a scientific field which attempts to provide a sight to the machine. This scientific field has expanded rapidly in recent years. Among researchers this growth is because of many improvements of vision algorithms and among the computer vision hobbyists this is due to the cheaper hardware components and processing power. OpenCV library plays a great role in the Computer Vision field as it helps greatly to reduce cost and preparation time of computer vision research environment needed by university students, hobbyists and professionals. OpenCV also provides a simple to use functions to get the work done in a simple, effective and elegant manner. OpenCV was started by Intel, and later it was transformed to an open source project now available on SourceForge.net. OpenCV library has multi-platform availability, and it is partially written in C++ and C language. Despite the fact that this library is available on many Linux distributions from its relevant package repositories, in this article we will attempt to install and use OpenCV library compiled from a source code downloaded from SourceForge.net web site. The reasons for compiling a source code may include:
- new version 2.0.0 recently released and more features available
- some bugs fixed which affected Linux OpenCV 1.0.0 versions ( such as cvGetCaptureProperty() etc. )
- more support is available for OpenCV 2.0.0 version than for former 1.0.0 version
This article will start with installation of OpenCV on Debian 5.0 ( Lenny ). Later a reader will be guided through a number of examples on how to use OpenCV to display an image, play a video and use camera to capture the video input stream.
3. Installation
The following section will describe an installation process of OpenCV library by building a binaries from a source code available from SourceForge.net. The installation procedure demonstrated here was tested on Debian 5.0 ( Lenny ) and Ubuntu 9.10 ( Karmic Koala ). The actual installation procedure should be similar or exactly same for most Linux distributions apart of the first step where package dependencies are installed from relevant Debian and Ubuntu distribution repositories. On RPM linux system you sould consult your Red Hat package management ( RPM ) tool for alternatives to OpenCV prerequisites described in the next section.
3.1. Prerequisites
First, what needs to be done is the installation of required prerequisites required by OpenCV library. The list of dependencies can be slightly modified according to your needs:
- libavformat-dev - development files for libavformat the ffmpeg file format library
- libgtk2.0-dev - development files for the GTK+ graphical user interface library
- pkg-config - manage compile and link flags for libraries
- libswscale-dev - development files for libswscale the ffmpeg video scaling library
- cmake - A cross-platform, open-source make system used for compilation of source code
- bzip2 - high-quality block-sorting file compressor used to extract OpenCV source file
The following command will automatically fetch and install all required packages and its dependencies:
# apt-get install libavformat-dev libgtk2.0-dev pkg-config cmake libswscale-dev bzip2
3.2. Obtaining OpenCV source code
Current version of OpenCV library at the time of writing is a version 2.0.0. You can download an OpenCV source code by pointing your web browser to OpenCV-SourceForge.net or use the wget command to acquire a source code directly on the command line:
$ wget http://downloads.sourceforge.net/project/opencvlibrary/opencv-unix/2.0/OpenCV-2.0.0.tar.bz2
3.3. Extract OpenCV source code
Whether you used web browser or wget utility to download source code of OpenCV library you should end up with OpenCV-2.0.0.tar.bz2 tarball in your current working directory. The next step is to extract source files with the tar command. The following command will extract all files into OpenCV-2.0.0 directory:
$ tar xvjf OpenCV-2.0.0.tar.bz2
New OpenCV-2.0.0 directory ( approx. 67MB ) should be now available in your current working directory and will contain all necessary source files for a compilation.
3.4. Compilation and installation of OpenCV binaries
To compile OpenCV source code, we are going to use an open-source make system cmake. The following cmake configuration compile flags are going to be set:
- CMAKE_BUILD_TYPE=RELEASE : cmake will bulid a release project
- CMAKE_INSTALL_PREFIX=/usr/local : directory to be used as a installation destination
- BUILD_PYTHON_SUPPORT : enable python support
NOTE: cmake utility by default does not provide a way to uninstall your project from a system. If you have a need to uninstall OpencCV from you system you should make appropriate changes before you proceed with compilation.
Navigate to OpenCV-2.0.0 directory containing a source code:
$ cd OpenCV-2.0.0/
Create and navigate to a new directory to be used by cmake. I this case, the directory name is same as project type, "release":
$ mkdir release; cd release
Use cmake to create a configuration files with configuration flags described above:
NOTE: CMAKE_INSTALL_PREFIX flag can be set to any desired installation path
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D BUILD_PYTHON_SUPPORT=ON ..
After execution of the cmake command the installation summary will by displayed and will look similar to the one below.
OUTPUT:
-- General configuration for opencv 2.0.0 =====================================
--
-- Compiler:
-- C++ flags (Release): -Wall -pthread -ffunction-sections -O3 -DNDEBUG -fomit-frame-pointer -O3 -ffast-math -mmmx -DNDEBUG
-- C++ flags (Debug): -Wall -pthread -ffunction-sections -g -O0 -DDEBUG -D_DEBUG
-- Linker flags (Release):
-- Linker flags (Debug):
--
-- GUI:
-- GTK+ 2.x: 1
-- GThread: 1
--
-- Image I/O:
-- JPEG: TRUE
-- PNG: TRUE
-- TIFF: FALSE
-- JASPER: FALSE
--
-- Video I/O:
-- DC1394 1.x: 0
-- DC1394 2.x: 0
-- FFMPEG: 1
-- codec: 1
-- format: 1
-- util: 1
-- swscale: 1
-- gentoo-style: 1
-- GStreamer: 0
-- UniCap:
-- V4L/V4L2: 1/1
-- Xine: 0
--
-- Interfaces:
-- Old Python: 0
-- Python: ON
-- Use IPP: NO
-- Build Documentation 0
--
-- Install path: /usr/local
--
-- cvconfig.h is in: /home/sandbox/OpenCV-2.0.0/release
-- -----------------------------------------------------------------
--
-- Configuring done
-- Generating done
-- Build files have been written to: /home/sandbox/OpenCV-2.0.0/release
When the execution of cmake command did not produce any errors, then we are ready to compile a source code.:
NOTE: There will be a number of warning messages shown on your terminal during a build process. These warning messages can be ignored, unless they do affect your preferred OpenCV environment settings!
$ make
If no errors were displayed on the terminal and the progress dialog reached [100%] during the build process, we are ready to install OpenCV libraries. The installation is optional as long as your environmental variable LD_LIBRARY_PATH is linked to an appropriate OpenCV built directory. If you wish to install OpenCV into /usr/local as set by cmake flags above, execute a following command:
# make install
Export correct path to LD_LIBRARY_PATH environment variable and use ldconfig to dynamically link to an OpenCV library:
$ export LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH
# ldconfig
If you do not wish to install OpenCV library you should just export a correct path to OpenCV library build directory to let your system know where the library is located. Suppose that your new release directory is located at ~/OpenCV-2.0.0/release then your export path will look like this:
$ export LD_LIBRARY_PATH=~/OpenCV-2.0.0/release/:$LD_LIBRARY_PATH
# ldconfig
This completes an installation procedure of OpenCV library. For additional information in regard to the OpenCV installation visit OpenCV install guide.
4. OpenCV examples
Without prolonging a discussion about what Computer vision is and how it is related to OpenCV we will now move right to some examples on how to write, compile and execute simple programs using OpenCV library. If you are interested in a more intense introduction to Computer Vision and OpenCV I recommend a book: "Learning OpenCV: Computer Vision with the OpenCV Library by Gary Bradski and Adrian Kaehler".
4.1. An Image conversion
Let's start with something really simple and that is 7 lines of code to convert image between following image types:
- Windows bitmaps - BMP, DIB
- JPEG files - JPEG, JPG, JPE
- Portable Network Graphics - PNG
- Portable image format - PBM, PGM, PPM
- Sun rasters - SR, RAS
- TIFF files - TIFF, TIF
The following program will accept two command line arguments, source image and destination image. Source image will be stored as an image type specified by destination image file extension. Save the following code in a file called image-conversion.c :
#include "highgui.h"
int main( int argc, char** argv ) {
IplImage* img = cvLoadImage( argv[1]);
cvSaveImage( argv[2] , img);
cvReleaseImage( &img );
return0;
}
Source code of our new program is ready and here comes the compilation part. Assuming that you have saved your first OpenCV program as image-conversion.c you can compile your program with the following command:
$ g++ `pkg-config opencv --cflags --libs` image-conversion.c -o image-conversion
After successful compilation a new executable binary file named image-conversion is created in your current working directory. Before we test this new program, we need some sample image:
$ wget -O image.png http://www.linuxconfig.org/templates/rhuk_milkyway/images/mw_joomla_logo.png
wget downloaded and saved an image image.png into your current directory, and we can now attempt to convert this image to any image type listed above. The following command will convert image type PNG to JPG. Assuming that program compilation did not produce any errors and your binary file is saved as image-conversion you can convert between two image types with following command:
$ ./image-conversion image.png image.jpg
To confirm that image was converted, a file command can be used to display a file type for a given file as an argument:
$ file image.*
OUTPUT:
image.jpg: JPEG image data, JFIF standard 1.01
image.png: PNG image, 270 x 105, 8-bit/color RGBA, non-interlaced
When you look at the compilation command once more you can observe that a pkg-config utility had been used to retrieve a location of an OpenCV library with the use of --cflags option as well as to get all dependencies using --libs option. Therefore, an alternative command to the one above without pkg-config utility can be constructed to look something like this:
g++ -I/usr/local/include/opencv -L/usr/local/lib \
-lcxcore -lcv -lhighgui -lcvaux -lml image-conversion.c -o image-conversion
However, in both cases the compilation command will create unwanted library dependencies:
$ ldd image-conversion | grep local
OUTPUT:
libcxcore.so.2.0 => /usr/local/lib/libcxcore.so.2.0 (0xb7ccc000)
libcv.so.2.0 => /usr/local/lib/libcv.so.2.0 (0xb7a7a000)
libhighgui.so.2.0 => /usr/local/lib/libhighgui.so.2.0 (0xb7a3f000)
libcvaux.so.2.0 => /usr/local/lib/libcvaux.so.2.0 (0xb793b000)
libml.so.2.0 => /usr/local/lib/libml.so.2.0 (0xb78d8000)
Our program is dependent on OpenCv's highgui.h library and therefore including -lcvaux -lml -lcxcore and -lcv dependencies into a compilation command is not necessary. A shortened version of compilation command will look like this:
$ g++ -I/usr/local/include/opencv -lhighgui image-conversion.c -o image-conversion
Consequently, a program library dependency had been reduced:
$ ldd image-conversion | grep local
OUTPUT:
libhighgui.so.2.0 => /usr/local/lib/libhighgui.so.2.0 (0xb7f61000)
libcxcore.so.2.0 => /usr/local/lib/libcxcore.so.2.0 (0xb7a75000)
libcv.so.2.0 => /usr/local/lib/libcv.so.2.0 (0xb7823000)
From now on, it is up to you how you compile following examples in this article. Keep in mind that the first compile command including pkg-config will be able to compile all examples. However, it may produce a binary with excessive dependencies.
4.2. Display an Image
At this point, we have been able to convert an image type and confirm its meta description by file command. It is time to display an image on the screen and visually confirm that it was converted correctly. The following example program will display an image on the screen:
#include "highgui.h"
int main( int argc, char** argv ) {
// cvLoadImage determines an image type and creates datastructure with appropriate size
IplImage* img = cvLoadImage( argv[1]);
// create a window. Window name is determined by a supplied argument
cvNamedWindow( argv[1], CV_WINDOW_AUTOSIZE );
// Display an image inside and window. Window name is determined by a supplied argument
cvShowImage( argv[1], img );
// wait indefinitely for keystroke
cvWaitKey(0);
// release pointer to an object
cvReleaseImage( &img );
// Destroy a window
cvDestroyWindow( argv[1] );
}
NOTE: Return to an image conversion section above, if you need help on how to compile this OpenCV program.
Execution of this display-image program with an image.jpg produced in preceding section will display this image on the screen:
$ display-image image.jpg
OUTPUT:
4.3. Gaussian smooth
You can also attempt to create a simple image transformation using the gaussian smooth method. Add a following line into your display-image code before a cvShowImage function call:
...
cvNamedWindow( argv[1], CV_WINDOW_AUTOSIZE );
cvSmooth( img, img, CV_GAUSSIAN, 9, 9 );
cvShowImage( argv[1], img );
...
and add as a first line to your program ' #include "cv.h" ' directive.
This will incorporate a gaussian smooth method centered on each pixel with 9 x 9 area into the output image. After compilation and execution a following output will be presented:
OUTPUT:
4.4. Play video
This section includes a program code which will create a simple video player using OpenCV library. Sample video, tree.avi can be found in your OpenCV-2.0.0 directory where you have extracted its source files ( OpenCV-2.0.0/samples/c/tree.avi ) :
#include "cv.h"
#include "highgui.h"
// initialize global variables
int g_slider_position = 0; // trackbar position
CvCapture* g_capture = NULL; // structure to create a video input
// routine to be called when user moves a trackbar slider
void onTrackbarSlide(int pos) {
cvSetCaptureProperty(
g_capture,
CV_CAP_PROP_POS_FRAMES,
pos
);
}
int main( int argc, char** argv ) {
// create a window with appropriate size. Windows name is determined by file name
// supplied as an argument
cvNamedWindow( argv[1], CV_WINDOW_AUTOSIZE );
// open video
g_capture = cvCreateFileCapture( argv[1] );
// set read position in units of frames and retrieve total number of frames
int frames = (int) cvGetCaptureProperty(
g_capture,
CV_CAP_PROP_FRAME_COUNT
);
// do not create treackbar if video does not include an information
// about number of frames
if( frames!=0 ) {
cvCreateTrackbar(
"Position",
argv[1],
&g_slider_position,
frames,
onTrackbarSlide
);
}
// display video frame by frame
IplImage* frame;
while(1) {
frame = cvQueryFrame( g_capture );
if( !frame ) break;
cvShowImage( argv[1], frame );
// set trackbar to a current frame position
cvSetTrackbarPos("Position", argv[1], g_slider_position);
g_slider_position++;
char c = cvWaitKey(33);
// quit if ESC is pressed
if( c == 27 ) break;
}
// free memory
cvReleaseCapture( &g_capture );
cvDestroyWindow( argv[1] );
return(0);
}
NOTE: Return to an image conversion section above, if you need help on how to compile this OpenCV program.
Execute your new OpenCV program and as an argument supply a video file:
$ ./video-player ~/OpenCV-2.0.0/samples/c/tree.avi
OUTPUT:
4.5. Input from a video camera
The aim of this section is to provide some simple tips on how to configure a camera on a linux system and how to confirm that your video camera is recognized by your system correctly. When your camera is ready, you will be presented with a simple program which is capable to display a video using a video camera as an input.
For this article I have used a Logitech, Inc. QuickCam Pro 9000 camera. Installation of this camera on Debian 5.0 or Ubuntu 9.10 ( Karmic Koala ) system was simple Plug & Play procedure. Here are some hints on how to confirm that your camera had been recognized by your system:
NOTE: your output will be different !
$ lsusb
OUTPUT:
Bus 002 Device 003: ID 046d:0990 Logitech, Inc. QuickCam Pro 9000
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 045e:00d1 Microsoft Corp. Optical Mouse with Tilt Wheel
Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
lsusb command reveals a camera type plugged into your system. Output of lsusb command does not necessary means that your camera is now ready to use. Let's see if some modules are associated with video:
$ lsmod | grep video
OUTPUT:
uvcvideo 45800 0
compat_ioctl32 1312 1 uvcvideo
videodev 27520 1 uvcvideo
v4l1_compat 12260 2 uvcvideo,videodev
usbcore 118192 7 snd_usb_audio,snd_usb_lib,uvcvideo,usbhid,ehci_hcd,ohci_hcd
This looks very promising. My camera is using uvcvideo module. If you do not see any ouptut or you see only output not related to your camera device you may need to recompile your kernel or install an appropriate module.
Now we need to find a device file corresponding with your camera. To do that we use xawtv utility:
NOTE: if xawtv command is not avaialable you need to install xawtv package.
$ xawtv -hwscan
OUTPUT:
This is xawtv-3.95.dfsg.1, running on Linux/i686 (2.6.26-2-686)
looking for available devices
port 65-96
type : Xvideo, image scaler
name : NV Video Blitter
/dev/video0: OK [ -device /dev/video0 ]
type : v4l2
name : UVC Camera (046d:0990)
flags: capture
The device file assciated with my camera is /dev/video0 . You may also see an error on your terminal saying: open /dev/video0: Permission denied. To fix this problem you need to make your self a part of a group "video". Now test your camera with a following command:
$ xawtv -c /dev/video0
If you had some issues in some of the previous steps, here are some links, which may assist you to troubleshoot your problem:
- Linux OpenCV Camera Compatibility
- Linux Webcam HOWTO
- Supported cameras using Spca5xx drivers
- Supported cameras using uvcvideo drivers
To use a camera with the OpenCV library is simple as writing a program to play video. Copy a previously created source code of your video player program and change line:
CvCapture* capture = cvCreatefileCapture(argv[1]);
to:
CvCapture* capture = cvCreateCameraCapture(0);
So the whole code will look similar to the one below:
#include "highgui.h"
int main( int argc, char** argv ) {
cvNamedWindow( "Example2", CV_WINDOW_AUTOSIZE );
CvCapture* capture = cvCreateCameraCapture(0) ;
IplImage* frame;
while(1) {
frame = cvQueryFrame( capture );
if( !frame ) break;
cvShowImage( "Example2", frame );
char c = cvWaitKey(33);
if( c == 27 ) break;
}
cvReleaseCapture( &capture );
cvDestroyWindow( "Example2" );
}
Notice that a function cvCreateCameraCapture() did not take any specific device file or argument. In this case OpenCV will start using the first available camera in your system. Compile and execute this program and if everything up to this point went well you should see yourself on your screen.
NOTE: Return to an image conversion section above, if you need help on how to compile this OpenCV program.
4.6. Write avi file from a camera
The last example will attempt to read an input from a camera and write it to a file. In the meantime, the program will also display a window with a camera input video stream. The Video input is saved to a file supplied as an argument on the command line. The codec used is specified by FOURCC ( Four Character Code ) MJPG which in this case is Motion JPEG. This sample program is very basic and there is a plenty of room for improvement:
#include <cv.h>
#include <highgui.h>
main( int argc, char* argv[] ) {
CvCapture* capture = NULL;
capture = cvCreateCameraCapture( 0 );
IplImage *frames = cvQueryFrame(capture);
// get a frame size to be used by writer structure
CvSize size = cvSize (
(int)cvGetCaptureProperty( capture, CV_CAP_PROP_FRAME_WIDTH),
(int)cvGetCaptureProperty( capture, CV_CAP_PROP_FRAME_HEIGHT)
);
// declare writer structure
// use FOURCC ( Four Character Code ) MJPG, the motion jpeg codec
// output file is specified by first argument
CvVideoWriter *writer = cvCreateVideoWriter(
argv[1],
CV_FOURCC('M','J','P','G'),
30, // set fps
size
);
//Create a new window
cvNamedWindow( "Recording ...press ESC to stop !", CV_WINDOW_AUTOSIZE );
// show capture in the window and record to a file
// record until user hits ESC key
while(1) {
frames = cvQueryFrame( capture );
if( !frames ) break;
cvShowImage( "Recording ...press ESC to stop !", frames );
cvWriteFrame( writer, frames );
char c = cvWaitKey(33);
if( c == 27 ) break;
}
cvReleaseVideoWriter( &writer );
cvReleaseCapture ( &capture );
cvDestroyWindow ( "Recording ...press ESC to stop !");
return0;
}
Assuming that you saved and compiled this programm as "save-camera-input" you can strat recording a video to a video-file.avi with this command:
NOTE: Return to an image conversion section above, if you need help on how to compile this OpenCV program.
$ ./save-camera-input video-file.avi
5. Conclusion
This article should give a you a good start to OpenCV library from an installation perspective. Examples presented do not have much to do with Computer Vision itself, but rather they provide a good testing ground for your OpenCV installation. Even from these simple OpenCV examples it is also clear that OpenCV is a highly civilized library since with just couple lines of OpenCV code you can achieve great results. Your comment on this article is highly appreciated as it may have a great impact on article quality. There is more comming about OpenCV, so stay tuned by subcribing to linuxconfig.org RSS feed ( top left corner ).
