Carridademo Program User Documentation

Revision: 1.2
Date: 2018-05-25
Contact: support@carrida-technologies.com
Copyright: 2017-2019 Carrida Technologies GmbH, Ettlingen, Germany
Author: VC Support

Home page

Table of Contents

1   Introduction

Carrida is a software library for Automated License Plate Recognition (ALPR) in images. The purpose of Carrida is to detect and read vehicle license plates and provide them as text strings to your application. The main function of Carrida is very similar to optical character recognition, but it is tuned and optimized to the special structure and syntax of license plates text. Typical applications of the Carrida engine include:

  • Parking lot management (registering license plates as they enter/exit the parking lot)
  • Vehicle ticketing/management
  • Automatic gate opening
  • Vehicle counting
  • Law enforcement applications
  • Traffic planning (counting the vehicles, mapping the traffic to geographic regions)
  • Traffic flow monitoring

For those applications, Carrida provides a simple programming interface to extract license plate strings from images.

Carrida offers the following features and benefits:

  • Easy to configure
  • Any number of license plates per image can be detected
  • Support for State/Country recognition
  • Reading of license plates with two rows
  • Reading of license plates with stacked characters
  • Robustness against image varying quality (illumination, contrast, noise, rotation)
  • Computation of confidences for detected plates as well as for individual characters
  • Exact position of detected plates as well as all individual characters within the image is delivered
  • Thread safe implementation

A list of abbreviations used throughout this document:

ARM64
Stands for processors with the v8 AARCH 64-bit instruction set.
ARM
Stands for processors with the v7 ARM 32-bit instruction set.
ALPR
Automatic License Plate Reader.
dB
Decibel, in the Carrida documents used to describe a relative sensitivity (gain) value in cameras.
fps
Frames per second.
FOV
Field of view.
HW
Hardware.
LP
License Plate.
mA
Milli-Ampere, one 1/1000:sup:th of an ampere.
ms
Millisecond, one 1/1000:sup:th of a second.
OSD
On-screen display
PC
Personal Computer.
PK
Product Key (for software licensing).
px
Pixel. The smallest unit of resolution in an image. For example, VGA images are made up of 640x480 pixels.
ROI
Region of Interest, typically a marked area in an image.
SDK
Software Development Kit.
SW
Software.
x86
Stands for Processor Architectures with the Intel instruction set. 32-bit hardware.
x86_64
Stands for Processor Architectures with the Intel instruction set. 64-bit hardware.
VC
Vision Components GmbH.
Image symbols used in this document
Symbol Meaning
Note Sign Points to especially important aspects of the manual or the software.
Hint Sign The lightbulb highlights the hints and the ideas that may be helpful.
Warning Sign This warning sign alerts of possible pitfalls to avoid. Please pay careful attention to sections marked with this sign.

2   Getting started with Carridademo

When starting the program for the first time Quick help window will appear, as shown on the image below. It provides useful information about the program and quick tips on how to use it.

./images/Image01.png

Disable Quick help on every startup.

If you wish not to see Quick help each time on start-up uncheck ,Show on start-up’.

Illustrated on the image below is the graphical user interface for Carridademo. The Menu bar (1) contains general sub-menus: File, Tools, Window and Help, detailed in Chapter 3.

./images/Image02.png

Carridademo user interface

Source bar (2) is used for defining input sources.

Input can be selected as:

  • folder with images,
  • video file,
  • live stream and
  • USB camera(s).

Specifics about different source types are elaborated in Chapter 4.

The Log Window (3) that appears at start-up contains the logging history during the time the program is running. When closed it can re-open it in Windows sub-menu located on the Menu bar (1).

./images/Image03.png

Carridademo user interface

On the right side of the program window the Region file is located. This is the area where region files and Make and Model classifiers are selected.

On the information icon (marked with the letter i on the image), users can see which countries are included in the current region package.


Program settings are located on the right side of the GUI. They are divided in three sub-categories Basic, Stream and Advanced settings, detailed in Chapter 5.

On the Results card (5) last result image can be viewed by clicking on ,View image’. The ,History’ button will open a new window with an extended list of results (image below).

./images/Image05.png

List of results images.

Highlighted on the image above is the ,Open output folder’ button. It opens the folder with result images defined during installation.

Clicking on any of the entries on the list will open the image on the OSD, and information about the reading will display on the right side beside the picture.

Hint

Hint Sign The results in History are limited to the amount set in File-Settings-Number of last results to keep in the program which is by default set to 100.


On the image below two green vertical lines are positioned on the lower left side of the OSD (6). These lines are used to illustrate the size of the characters in pixels.

The engine will recognize only those license plates with characters bigger than the min size (left green bar) and smaller than max size (right green bar).

./images/Image04.png

Carridademo user interface

With Min being the Minimal letter height, and Max being the Maximal letter height, both are part of Basic settings (4). Use these bars as a guideline for more appropriate size estimation seeing how both of these parameters can affect processing speed and result reading.


The bottom part of the screen is the Play/Record bar (7). When video or image files are selected this bar will serve for enabling/disabling image processing and playing/pausing the source material.

Shown in the following table are steps how to start reproducing source material, license plate detection and producing results.

Only when in Recording mode the images are being automatically saved in the output folder.

When a stream is suspended, it will be indicated with a red "Paused" sign on the bottom of the screen.


The white/blue bar next to the Play/Pause button indicates the percentage of the stream that is currently processed (image below).

./images/Image06.png

How to read information displayed above the bar:
p7 Average processing time for the last 100 readings.
p8 Percentage of image magnification.
p9 Size of the area in which the ALPR will search for plates.
p10 Position of the mouse on the image in pixels

2.1   Starting image processing

The green rectangle bordering the image (see figure below) represents the region of interest (ROI) for ALPR detection, meaning that the program will search for plates only inside of this rectangular area.

./images/Image08.png

Resizing the region of interest (ROI) for ALPR.

Before any image processing starts make sure the ROI size covers the path on which the plates will move through the screen.

ROI can be re-sized by:

  • clicking on any of the four corners and dragging them to the wanted position on the screen, or
  • clicking on any of the lines and carrying the border to the desired position.

Images size on the OSD can be manipulated. Look at the following steps how to zoom in/out an image:

  • Press and hold ALT and scroll in and out with the mouse.


    65a

  • Press + or - buttons on the keypad.


    65b

  • When zoomed in you can pan the image by pressing ALT and dragging the image with the mouse while pressing left mouse button.


    65c

  • Double-clicking on the image it will resize it to the full screen. To exit full screen view press Escape (Esc) or double-click on the image.


Hint

Hint Sign Use Min and Max sidebars to define the character size. Enlarge the image and use the sidebars as guidelines to set the size more accurately (see picture below).


ii



Minimal and maximal letter height can be modified in Basic settings. When the size is optimized processing speed increases as well as the reading certainty.


After setting the ROI and defining the min and max letter height do the following:

p11 Select an input source: Video file, Image folder, USB camera or Live stream. Two folders with video and image files are provided with the demo program.
p12

Click on ,Play’ to start the stream processing (if it did not start automatically).

Press ,Record’ to start saving results on a hard drive.

To pause image streaming process click on ,Pause’. p4

To start saving results on a hard drive press ,Record’. It will change color to red when in recording mode.

Press the same button to pause/stop recording.

p5
The ,Play/Stop’ button moves the video frame by frame. When using image files this button will move trough the file image by image. p6
p13 Go to History on the Results card to view processed images. Image files are saved in the defined output folder, and a direct link can be found in History window.

After everything is set up properly, license plate processing will start as soon as a vehicle is detected. The image below shows an example of one output produced by the ALPR.


i23 Example of the result image

The first information to be displayed is state/country where the vehicle is registered with its recognition confidence (1). Second to be displayed is the vehicle number plate with its recognition certainty (2).

In the second line (3) information about the car manufacturer and car model is displayed. Next to that percentage of recognition certainty is shown.

Which information displays on the result images can be adjusted under File-Settings.

4   Source bar

Source material can be imported from USB camera(s), Video files, Folders with images or from a Live stream.

./images/Image20.png

Hint

Hint Sign With Carridademo two samples are provided, a video file and an image folder. They are intended for testing the program and different parameter settings.



When selecting any of the sources they will have common general elements, illustrated on the image below.

ag 1 Region of interest (ROI)

The green rectangular border represents region of interest in ALPR. Just inside these borders ALPR will detect plates.

Resize it by clicking and dragging any corner, or click on any of the lines and "carry" the border to a desired position.

2 History History will contain a limited list of results with images. The size can be defined in File-Settings-Number of last results.
Live

Image streaming will be displayed on the Live card in real time without implementing optional processing (image rotation, deinterlacing, etc.).

In situations where ALPR results can be displayed with a delay, for example offline streaming mode, Live image will display the input stream without any delays.

3 Start ALPR detection

On the Play/Record bar pressing ,Play’ will start image streaming.

Press the ,Record’ button for the ALPR detection to start. Processed images will then be saved to the output folder.

Examples and detailed explanations can be found in Chapter 2.1, Start image processing.

4 Refresh and Folder buttons

When using image folders and video files an additional ,Folder’ button will appear. Use it to open new files or folder of the same source.

Use the ,Refresh’ button to reinitialize files/folder in use.

4.1   USB Camera

When selecting USB Camera as the source, image streaming from the camera should appear on the OSD.

To start image processing click on the ,Record’ button (1).

Processed images will then be saved in the specified output folder.

./images/Image22.png

USB camera source.

Suspended input stream will be indicated with a red "Paused" sign on the bottom of the OSD.

Note that in some cases ALPR results will be available with a delay (for example in offline streaming mode, see stream presets further in this document). Live image without any delays can be viewed on the Live card (2) if ,Show live stream’ (3) is enabled.


cam If more than one camera is connected, the additional window may appear as shown on the image. Select a specific camera from the drop down list of connected devices and choose one for the main source.

4.2   Video file and Image folder

Select one of these sources to reproduce single video file or a folder with one or more images.

When Video file is selected the program will only recognize video formats, not images and vice versa. If the selected source format conflict with the data chosen an error message will appear.

To change the path for prior selected source click on the Folder icon (1). And to reload the same folder/video click on the Refresh icon .

./images/Image24.png

On the Play/Record bar (2) image streaming can be started/paused and recording processed images can be activated. When the streaming is suspended, a red "Paused" sign will appear. Further use of the buttons is explained in the table below.

Click on ,Play’ to start the stream processing (if it did not start automatically). sf
To pause the image streaming process click on ,Pause’. sb

To start saving the results on the hard drive press ,Record’. It will change the color to red when in recording mode.

Press the same button to pause/stop recording.

p5
To go through the image files. image by image click on the ,Play/Stop’ button. p6

Only when the Record button is activated, (red) results will be produced to the destination folder.


Basic information about the latest available ALPR result is displayed on the Results card (3). The last result image can be accessed through ,View image’ and the list of results can be found in ,History...’.

4.3   Live stream

For Live stream you need to specify the path to the stream (rtsp or http) in the pop up window.

How to start image processing and view results read in Chapter 2.

5   ALPR parameter settings

Generally all ALPR relevant parameters are stored in the ALPR settings file (1).

./images/Image25.png

The default Carrida intallation includes only so called world region file.

What is a region file?

Region files contain state/country specific information. This information can for example be the size a font of letters and numbers, plate dimensions, format etc. They are divided in regions (Europe, USA, Thailand...) and each region contains different country's.

Using a suitable region file will improve the accuracy of the plate recognition according to the country/state parameter as they support ALPR with features that make recognition easier and more reliable.

The "world" file is set as default. It is set up to read license plates containing Latin letters (A-Z) and Arabic numerals (0-9). It does not contain state recognition or region specific information. For that reason ALPR generates state recognition result as "WORLD" with "0%" recognition certainty.

To increase the reading accuracy and enable state recognition an appropriate region file must be applied.


After selecting an appropriate setting file, its parameters will be loaded and displayed in the corresponding fields (Basic, Stream and Advanced settings). All parameters can be changed "on the fly", meaning every change will have immediate effect. To make changes persistent, click on the ,Save’ icon.

Parameter settings are divided into three groups:

  • Basic
  • Stream
  • Advanced settings.

On the bottom right side of each parameter group, help button (2) is located. When pressing on this button, a window will appear with detailed description of each parameter.

5.1   Basic settings

Basic settings offer the possibility to change the most frequently used parameters. Optimizing the minimal and maximal letter height is important before starting image processing, and is located in the basic settings. Setting the character size accurately will improve both recognition quality and processing time.

./images/Image26.png

Depending on the source material being processed a corresponding mode must be selected. Modes are set to have different processing speed and certainty of recognition. When, for example, a live image stream is set as the source, and processing speed is important, then the fast mode is optimal. When the processing speed is not that important, in case of video files, then best mode is optimal, as it has the best possible reading quality.

More information on basic settings are in the continuation of this Chapter.

5.1.1   Mode

The Carrida engine can operate in several performance modes, which are differ in recognition speed and reading confidence.

You can choose between the following modes:

  Mode Description
mo Fastest Fastest possible processing speed with lower recognition quality than in other modes.
Fast Fast processing speed with slightly better recognition quality than in the fastest mode.
Standard A good compromise between speed and reading performance, recommended being used in most situations.
High Lower speed and reading performance than in standard mode, with increased reading confidence.
Best The best possible reading quality. Can be up to two times slower than in the fast mode. At the same time, reading confidence will be about 3-4% better.

As the best compromise between processing speed and reading accuracy we recommend operating in Normal mode.

When working in an environment where processing time is not critical or (e.g. processing of video files) or when using powerful hardware, use Best mode to reach the best possible reading accuracy.

5.1.2   Minimum plate confidence (%)

The minimum confidence level of license plates to be accepted. License plates with lower confidence will be rejected.

Example:If the plate confidence level is set to 70, the program will only prodce results recognized with the confidence of 70% or higher, all other detections will be ignored.

5.1.3   Minimum/Maximum letter height (in pixels)

Define the minimal and maximal height of characters that ALPR will recognize. The minimum value cannot not be less than 8 pixels, for a maximum value there are no limits.

These are two highly important parameters - setting them accurately will improve both recognition quality and the processing time.

Use the Min and Max sidebars on the bottom left side of the image to control the correctness of the min/max values.

./images/Image12.png

Zoom in (+/-) and use Min and Max bars as guidelines to assess the average size of characters.

Use the zoom buttons to enlarge the image and set min and max height more precise. Optimizing the size will increase processing speed and improve reading accuracy.

5.1.4   Deinterlace

Deinterlacing is a process of converting interlaced video (such as analog signals or 1080i format HDTV signals, etc.) into a non-interlaced form.

If enabled, deinterlacing will be applied to the input image before further processing.

5.1.5   Scale width/height

When these parameters are adjusted, input images will be resized by given ratios in percentage before processing.

New values have to be provided in percentage of the original size (100 = original size).

This procedure can be found useful when license plates are visually distorted due to not ideal camera positions, especially for steep top views.

Scale parameters can also be used to reduce the image resolution for faster processing. For example when the image is bigger than needed it can be reduced in size and the program will not have to process as many pixels.

5.1.6   Rotate/Flip image

Images can be rotated before the processing begins. In case of rotation all further settings will apply on rotated images. Output license plate coordinates and output images will be rotated as well.

Possible rotations:

  • None - no rotation
  • Rotate right - 90 degrees clockwise.
  • Rotate left - 90 degrees counter clockwise.
  • Rotate image 180 degrees (mirror it both horizontally and vertically).
  • Flip image horizontally.
  • Flip image vertically.
./images/Image28.png
./images/Image29.png

5.1.7   Rotation angle

Rotate the angle of input images in one degree steps so cars can appear horzontal in the images. This is not a requirement, but it can slightly improve the recognition quality.

Example:

How the rotation angle tool can help when the camera does not have the ideal angle to the vehicle:


./images/Image30.png

5.2   Stream settings

Stream parameters are related to handling the video streams - i.e., situations where a number plate is visible in more than one successive frame. Mostly, this concerns the way how multiple license plate readings are merged into one reading result per vehicle.

There are several streaming modes available that set individual parameters to some meaningful predefined values.

./images/Image31.png

Stream parameters are set according to the type of source material. Meaning if a video file is selected as the source, streaming mode will automatically be set to offline video. This option can be disabled in File-Settings by un-checking "Automatically adjust stream preset depending on source type" .

5.2.1   Apply preset

In order to make some basic setups easier to handle, the demo program provides several presets, which can be used as a good starting point for further experiments.

The stream presets provide processing settings for various application scenarios, detailed in the table below.

    Usage Output
pr Single images Streaming parameters are configured in such a way that the video handling function is switched off. Each input image is considered not to be related to adjacent images. Every single detected license plate. Recommended for image folders.
Parking mode This mode is designed for access and parking control systems. Each vehicle number plate is required to be detected at least three times. The output result is available immediately afterwards. Unique license plates. Recommended for live stream and USB cameras.
Offline video In this mode results are available with a delay of about 25 frames. The reading quality is the best in this mode as the engine has more frames to analyse before producing results. It is not suited for real time application where any delay is unwanted. The main purpose of this mode is to process video files. Unique license plates. Recommended for video files.
Free flow This mode is designed for freeflow real-time systems. It is configured in such a way that it tries to catch all passing vehicles. The result may become available with a delay up to 15 frames. Unique license plates. Recommended for live stream and USB cameras.

By default ,Automatically adjust stream preset depending on source type’ is enabled (File-Settings). Meaning that every time a new source is selected ALPR will switch the preset mode accordingly (e.g. live stream preset may be changed to freeflow). Each individual parameter can be set explicitly (thus overriding its default value). In that case the automatic adjustment may not be desired and can be disabled in File-Settings.

5.2.2   Minimal number of readings

Refers to the minimal number of required readings per vehicle before its license plate will be produced to output. It can be used to suppress occasional wrong readings.

This parameter can be set in range from 1 to 10 readings.

If set too high, the engine may produce less output images, as the vehicles may pass too quickly to read them the desired number of times.

Example:

Readings for the last seven successive frames: ABC123 x ABC123 x x x ABC123 (x being a frame without reading results).

If Minimal readings = 5, no images will be produces. But if e.g. Minimal readings = 2, output image will be produced as there are more than 2 readings detected.

5.2.3   Maximal matching distance

Describes the maximal Levensthein distance* between two ANPR reading results achieved in two different frames. It is used by the program to make a decision if these readings belong to the same vehicle or not.

Setting this parameter to a higher value increases the tolerance of the system to reading errors. Normally, there is no need to change this parameter.

The Levenshtein distance = Number of character deletions + Insertions + Replacements to transform one string into another. For example, the Levenshtein distance between "HR737AM" and "HR231AN" is 3, since it takes exactly three edits to change one into the other.

5.3   Advanced settings

./images/Image33.png

5.3.1   Maximal plate angle

This parameter allows to change the largest expected license plate angle relative to x-Axis. It is defined in degrees, and can be set up to 45°.

5.3.2   Maximal number of plates in frame

This parameter refers to the maximal number of license plates that the engine will return per image.

Example:./images/Image35.png

On the first image the limitation for ,*,Max plates in frame’* is set to 1 and just one result has been produced.

In case of multiple plates in frame plates with higher confidence will be selected over the ones with lower confidence.

Hint

Hint Sign If set to any, all recognized plates will be returned.

5.3.3   Minimal contrast

Defines the minimal contrast to differentiate the foreground (characters) from background (license plate itself). It is defined in pixels and can be set between 8 and 100. By default, it is set to 12 px in the best mode, and 50 px for other modes.

Hint

Hint Sign Reducing this parameter can increase both reading performance and processing time.

5.3.4   State recognition

Disabling state recognition switches off the state recognition process for region files with built-in state recognition.

Hint

Hint Sign Processing time will be reduced, but the reading performance can degrade slightly.

5.3.5   Filter noise

When checked image noise will be filtered before processing begins.

Usage of this parameter can significantly improve run-time performance for noisy images.

Hint

Hint Sign Setting this parameter to true for images that do not have much noise can increase processing time.

5.3.6   Remove pure letter/number plates

If checked ,Remove pure letters’ the program filters license plates that consist only of letters, and vice versa. If ,Remove pure numbers’ is checked the program will not process plates with only numbers.

Example:

Shown on the first image is a vehicle with a license plate formed of only letters. The second image shows a vehicle with the license plate made of pure numbers.

./images/Image37.png

If you wish not to process images with pure letters check ,Remove pure letters’.

5.3.7   Search inverted

Search inverted refers to searching for plates with dark background and light colored characters.

When none is selected inverted plates will never be detected by the ALPR. If search is selected inverted plates will also be searched for in case when no black-on-white plates are detected. If always is selected inverted plates will always be searched for. Do not use this setup unless necessary, as it will increase the processing time.

Example:./images/Image38.png

If ,Search inverted’ is disabled the program will not detect inverted license plates, as illustrated on the first image. On the second image this parameter is enabled and the ALPR processed the plates and generated output.

Hint

Hint Sign Setting this parameter to true will increase the processing time per image.

5.3.8   Detect make and model

Make and model is a feature of Carridademo for car manufacturer and car model recognition.

For Make and model to operate, appropriate classifiers need to be uploaded. Those clasifiers are .ini files and contain additional data for ALPR to be able to recognize vehicle producers and types. If the proper calssifier is not uploaded make and model feature (under Advanced mode) can not be enabled.

./images/Image39.png

Make and model enabled.

Hint

Hint Sign

To receive make and model classifiers contact the support center via the email displayed in the header.