Beyond ANPR

Revision: 1.0
Date: 2020-12-15
Contact: support@carrida-technologies.com
Copyright: 1996-2020 Carrida Technologies GmbH, Ettlingen, Germany
Author: Carrida Support

Home page

Table of Contents

1   Introduction

Doing ANPR is the core task of the Carrida library. At the same time, we are constantly adding new powerful functionalities that can be utilized either additionally or independently from the ANPR job. This document provides an overview of the following functionalities:

  • Make & Model classification (M&M)
  • Vehicle classification
  • License plate finder

The first two points can be seen as an object classification task and are described in the next section.

2   Object classification

To be able to utilize object classification functionality, there are two prerequisites to be met:

  • The Carrida license must unlock object classification functionality (please note that the standard Carrida license allows only ANPR functionality)
  • the extended region .dat file is needed
  • a separate Carrida installation for vehicle classification functionality is needed

Both M&M and vehicle classification can be performed either additionally or independently of the ANPR job. The desired mode can be set via the configuration ini file:


[MAKE_AND_MODEL]

OperationMode = 1

Alternatively, API set_parameter() function can be used. Here, OperationMode is a combination of desired modes:


enum LPR_operation_mode
{
   lpr_operation_anpr = 1,    ///< detect number plates
   lpr_operation_make_model = 2, ///< detect vehicles brands (e.g. "Opel Astra")
   lpr_operation_vehicle_classification = 4 ///< detect vehicle type (car/truck/bus/...)
};

Any combination of these operation modes can be used - the only restriction is that simultaneous use of Make&Model and vehicle classification is not allowed. The following table gives an overview of all possible setups.

OperationMode Description
0 Nothing to be done (not useful in the practice)
1 ANPR mode (this is also the default configuration) - register all license plates
2 Make&Model mode - detect only vehicle brands and models. Note that only one vehicle per image can be detected when using this mode. This mode performs better if just one vehicle is visible in an image
4 Vehicle classification mode. Detect all vehicles in each image and provide their types (car, truck, etc.)
3 ANPR + M&M mode - register all vehicles license plates, and, for each vehicle, detect make and model
5 ANPR + vehicle classification mode - register all vehicles license plates, and, for each vehicle, detect its type

2.1   Make&Model detection

Whenever Make&Model functionality is used, the License_plate structure provides additional information for the following fields:


struct License_plate
{
       // ...

       std::string make;                       ///< vehicle make if available

       std::string model;                      ///< vehicle model if available

       uint32 make_model_confidence;           ///< vehicle make and model confidence if available

       // ...
}

Currently, M&M detection is supported for the following regions:

  • Europe
  • USA

M&M detection runs completely on a CPU - no graphic card is used.

2.2   Vehicle classification

Vehicle classification functionality is supported starting from the Carrida version 4.5.0. One needs a separate Carrida installation that utilizes the Tensorflow framework. Currently, this is supported for the Windows version only. The processing will run on the NVIDIA graphic card if available, otherwise on a CPU.

If the vehicle classification is switched on, the following fields of the License_plate structure will be additionally filled:


struct License_plate
{
       // ...

       Vehicle_type vehicle_type;              ///< vehicle type (car, truck, etc.)

       uint32 vehicle_type_confidence;         ///< vehicle type confidence value [0..100]

       LPR_rect_int vehicle_position;          ///< vehicle position

       // ...
}

Please note that unlike for M&M, vehicle rectangular position in pixels is additionally provided.

Following vehicle types are supported:

  • car
  • motorcycle
  • van small (below 3.5 tons)
  • van big (above 3.5 tons)
  • truck
  • bus

3   License plate finder

In this mode, only positions of all present license plates as bounding boxes will be returned. The license plate text itself will not be detected. For the plate finder functionality, a standard license is sufficient. One needs merely an appropriate region file.

Plate finder can be utilized in two ways described below.

3.1   Autonomous mode

Here, Carrida API function Carrida::LPR::plate_finder() is used:


std::vector< LPR_plate_position > plates = lpr.plate_finder( image );  // find positions of all license plates in the image

For each input image, this function returns positions of all founded license plates as well as their confidence values. Additionally, inverted flag tells if a license plate contains dark characters on a bright background or vice versa.


struct LPR_plate_position
{
       int32 x;                                    ///< Plate rectangle upper left x coordinate
       int32 y;                                    ///< Plate rectangle upper left y coordinate
       int32 width;                                ///< Plate rectangle width
       int32 height;                               ///< Plate rectangle height
       int32 confidence;                           ///< Plate confidence value [0..100]
       int32 inverted;                             ///< 1 means dark characters on bright background, 2 - bright on dark
};

3.2   ANPR boosting mode

In this mode, license plate finder functionality supports ANPR job in the following way:

  • first, the usual ANPR process takes place
  • then, possible missed plates will be found with the plate finder mode - typically, these are problematic plates (very poor contrast, not sharp, dirty, occluded etc.)
  • for the founded plates, license plate strings will be extracted

ANPR boosting mode can be switched on in the configuration ini file:


[ANPR_ENGINE]

Platefinder = true

Alternatively, the API function set_parameter() can be used.

Please note that the plate finder mode uses computationally heavy algorithms - thus, the processing time will grow. It is not recommended to use in real-time systems when dealing with high frame rates.