Supporting Document Mandatory Technical Document: Evaluation Activities for collaborative PP-Module for Biometric enrolment and verification - for unlocking the device - [BIOSD]

For definitions of standard CC terminology see [CC1]. For definitions of biometrics and the computer, see [BIOPP-Module] and the Base-PP.

[BIOPP-Module] defines the assumptions for the mobile device that is the operational environment of the biometric system. These assumptions are implicitly satisfied if the mobile device is successfully evaluated based on [PP_MD] and the operational guidance does not need to describe the security measures to be followed in order to fulfil the security objectives for the operational environment derived from those assumptions.

[BIOPP-Module] supposes that the biometric system is fully integrated into the mobile device and the preparative procedures are unnecessary for [BIOPP-Module]. Therefore, AGD_PRE.1 is deemed satisfied for [BIOPP-Module].

[BIOPP-Module] is intended to be used with [PP_MD] and reference for the mobile device can be used as the TOE (mobile device + biometric system) reference only if the reference for the mobile device also uniquely identifies the biometric system embedded in the mobile device.

The evaluator shall follow the same EAs defined in [PP_MD] and the BIOSD. As explained in the Chapter 7, Evaluation Activities for PAD testing, details of PAD testing for ATE_IND.1 are defined in [Toolbox].

The evaluator shall follow the same EAs defined in [PP_MD] and the BIOSD. As explained in the Chapter 7, Evaluation Activities for PAD testing, details of PAD testing for ATE_IND.1 are defined in [Toolbox].

There are many types of Presentation Attack Instruments that can be used to test a PAD subsystem. The [BIOPP-Module] specifically defines the artefacts that are to be used as artificial, and not natural. Natural artefacts, such as a dead eye, are not considered in scope for this evaluation. When searching for new artefact species, only artificial species should be considered.

As described in previous section, the [Toolbox] defines test items to create a representative set of artefacts that the evaluator shall use for the testing. During ATE_IND.1 evaluation, the evaluator shall conduct all test items in the [Toolbox] for the selected modalities without any change. The evaluator is not allowed to skip any test items in the [Toolbox] to maintain compatibility between different evaluations.

The developer needs to assign the maximum IAPAR in FIA_MBV_EXT.3. The evaluator shall follow the information in the [Toolbox], for example, the number of artefact presentations, to conduct the testing and confirm that measured IAPARs for all artefacts are equal to or less than the assigned IAPAR through the independent testing.

During the independent testing, the evaluator may find artefacts that are incorrectly matched to the enroled target user. However, the evaluator may not be able to reliably reproduce a successful presentation attack. The evaluator shall assign a fail verdict only if the evaluator can reproduce the artefact of which IAPAR is higher than the maximum value reliably and repeatably.

Those artefacts of which IAPAR is less than the maximum value but show one or more successful imposter presentations will be tested again during the AVA_VAN.1 evaluation.

The developer can select any biometric modalities in FIA_MBV_EXT.1 (corresponding test instructions for each modality are included in the [Toolbox]). If the developer wants to evaluate modalities not currently included in FIA_MBV_EXT.1, the developer and evaluator shall contact the BIO-iTC to work together to add the new modality and extend the [Toolbox]. Upon the BIO-iTC approval of this extension, the evaluator can proceed with PAD evaluation for the new modality.

The EAs presented in this section are derived from ATE_IND.1-3, ATE_IND.1-4 and ATE_IND.1-7 and their verdicts will be associated with those work units.

The [Toolbox] describes a test subset and test documentation that is sufficiently detailed to enable the tests to be reproducible (ATE_IND.1-3 and ATE_IND.1-4). The [Toolbox] also includes information that support the evaluator’s decision (ATE_IND.1-7).

This Section describes EAs for AVA_VAN.1 step by step following the order of AVA_VAN.1 CEM work units.

The EAs presented in this section are derived from AVA_VAN.1-3, AVA_VAN.1-4, AVA_VAN.1-5, AVA_VAN.1-6, AVA_VAN.1-7 and AVA_VAN.1-10 and their verdicts will be associated with those work units.

EAs in the Section 7.3.1.1, “Search for new artefacts” complements evaluator’s action for searching publicly available information and identifying potential vulnerabilities (e.g. new artefact) (AVA_VAN.1-3, AVA_VAN.1-4 and AVA_VAN.1-5).

EAs in Section 7.3.1.3, “Produce test plan” and Section 7.3.1.4, “Conduct the penetration testing” complements evaluator’s action for creating the test plan and conducting the penetration testing for PAD (AVA_VAN.1-6 and AVA_VAN.1-7).

EAs in Section 7.3.1.5, “Determine Pass/Fail of penetration testing” provides specific guidance for pass or failure of the testing (AVA_VAN.1-10).

The developer shall report following general information about the performance testing.

The performance report shall specify a target application modelled in the test, such as biometric verification in an indoor office environment with a habituated crew.

The performance report shall also report influential factors that may influence performance, measures to control such factors and under what factors the performance testing was conducted.

Influential factors can be determined by referring to appropriate documents (e.g. ISO/IEC 19795-3) or referring the product datasheet (e.g. operating temperature). These factors should be consistent with the target application.

The following factors are examples of controlling factors for finger/hand vein verification. The developer shall define these factors properly, for example, based on ISO/IEC 19795-3. Any information that is useful in the context of the used biometric modality shall be considered by the developer to determine the factors.

It is recommended to control all influential factors appropriately because different error rates may be measured under different influential factors.

The selection method of test subjects shall be reported (e.g. gather test subjects from developer’s employees or recruit them from public). It is recommended that the demographics of test subjects follow the target application.

Instructions and training given to the test subjects shall be reported. The same instructions and training shall be given to the all test subjects.

The following information about test subject management shall be reported. Proper management is necessary to avoid human errors that may occur during the testing.

A test protocol for the testing shall be reported. The following items shall be covered.

The developer shall follow the guidance in this Section to define the transaction if the developer selects FAR and FRR in FIA_MBV_EXT.1 or to define the number of samples per each test subject if the developer selects FMR and FNMR in FIA_MBV_EXT.1.

The user may use the biometric verification in a different way.

Suppose the computer provides both a NBAF and a BAF and the user can use either factor to unlock the device. One user may try to unlock the device with the BAF until allowable maximum number of unsuccessful authentication attempts is exceeded. Another user may try to unlock the device with the BAF only three times and switch to the NBAF if all three attempts were failed.

It may also be possible for user to enrol multiple body parts (e.g. index and thumb fingerprint) or single body part for biometric verification.

However, it is not possible to evaluate all these scenarios to measure the performance. The developer shall define one test scenario and describe it in the ST.

For example, if the ST sets the maximum number of unsuccessful authentication attempts for fingerprint verification to five, the developer shall assume that the attacker makes all five fingerprint unlock attempts in succession to try to unlock the computer.

This means that if FAR and FRR are selected, the developer shall define that the genuine and imposter transaction is consists of up to five unlock attempts and only one transaction can be run by each user.

If FMR and FNMR are selected, the developer may follow the same scenario and collect five samples from each test subject. However, FMR/FNMR is a comparison subsystem measure while FAR/FRR is a system level measure, therefore FAR/FRR should be selected in FIA_MBV_EXT.1 if the developer considers the specific test scenario to measure the performance at the system level.

The developer shall also select the most common scenario among users to conduct the performance testing. For example, if the user can enrol multiple fingerprints, the developer should assume that the user enrols index and thumb fingerprint if such enrolment is most common. FAR may increase and FRR may decrease if the user enrols multiple fingerprints however, performance of widely used configuration should be measured.

Only one template can be generated from each body part (e.g. right index fingerprint, left hand vein or face) of test subject and used for the performance testing.

The quality of the template may have a significant impact on the biometric verification performance. This BIOSD assumes that the user is familiar with the computer’s operation and enrols correctly following the AGD guidance provided by the developer. The test subject may make enough practice attempts to become familiar with the device operation before the final enrolment transaction.

The developer shall define the maximum number of samples per test subject to be collected following the guidance provided in Section 9.1.1, “Test scenario for biometric verification”.

Only one transaction can be run by each test subject because the computer locks the biometric verification as required by the Base-PP after the certain number of attempts are failed.

The BIOSD allows full cross-comparison to estimate FAR/FMR because it is commonly agreed that the statistical loss of computing all possible cross-comparisons between test subjects is acceptable.

This BIOSD also allows cross-comparison of attempts/templates of ordered pairs if there is no explicit reason that this cross-comparison hinders the accuracy of the result of performance testing. Cross-comparison of attempts/templates of ordered pairs allows the comparison between user A’s template and user B’s sample and user A’s sample and user B’s template separately. However, if the TOE’s verification algorithm is symmetric and make no distinction between the ordered pairs, this assumption can not be used. The type of verification algorithm used by the TOE is reported in the developer’s performance report Section 8.3.1, “Overview of the performance testing”.

This BIOSD doesn’t allow intra-individual comparison that is a comparison between one body part and another body part of the same test subject (e.g. comparison between right and left iris of the same user).

Table 3, “Quality Assessment Criteria Report Items (standard quality metrics)” shows items that shall be reported in the report. The name or structure of report does not need to follow Table 3, “Quality Assessment Criteria Report Items (standard quality metrics)”. However, all items in Table 3, “Quality Assessment Criteria Report Items (standard quality metrics)” should be included somewhere in the report. If some items are not included in the report, the developer shall provide a rationale for such exclusion to the evaluator.

This Section describes each item in Table 3, “Quality Assessment Criteria Report Items (standard quality metrics)” in detail. Most items are created based on the Section 8.3, “Reporting items description” so the developer may refer them to create the report.

The developer may point to a public standard used for the determining of quality metrics, such as ISO/IEC 29794-4 to provide the necessary information.

Methods to create low-quality samples (e.g. varying temperature / humidity conditions of the finger skin, low physical pressure, too less presentation time or incorrect finger positioning angles for fingerprint verification) shall be described. This information is used for the evaluator to conduct the independent testing for FIA_MBE_EXT.2 and FIA_MBV_EXT.2.

Table 4, “Quality Assessment Criteria Report Items (non-standard quality metrics)” shows items that shall be reported in the report. The name or structure of report does not need to follow Table 4, “Quality Assessment Criteria Report Items (non-standard quality metrics)”. However, all items in Table 4, “Quality Assessment Criteria Report Items (non-standard quality metrics)” should be included somewhere in the report. If some items are not included in the report, the developer shall provide a rationale for such exclusion to the evaluator.

This Section describes each item in Table 4, “Quality Assessment Criteria Report Items (non-standard quality metrics)” in detail. Most of items are created based on Section 8.3, “Reporting items description” so the developer may refer them to create the report.

Same as Section 10.1.3, “Test configuration”

Same as Section 10.1.6, “Methods to create low-quality samples”

As in [CEM], the factors to be considered consist of Elapsed time, Expertise, Knowledge of the TOE, Window of opportunity, and Equipment. But Window of opportunity is divided into two subfactors Window of opportunity (Access to the TOE) and Window of opportunity (Access to biometric characteristics).

Elapsed time is the total amount of time taken by the attacker.

In the identification phase, elapsed time corresponds to the time required to create the attack, and to demonstrate that it can be successfully applied to the TOE (including setting up or building any necessary hardware or software equipment). The demonstration that the attack can be successfully applied needs to consider any difficulties in expanding a result shown in the laboratory to create a useful attack. One of the outputs from identification is, for instance, a script that gives a step-by-step description of how to carry out the attack. This script is assumed to be used in the exploitation part.

In the exploitation phase, elapsed time corresponds to the time necessary to apply the "script" to specific biometric characteristics. For example, for a presentation attack to a fingerprint capture device, it corresponds to the time required to create an artefact from an image of a print (and not the acquisition of this image which is taken into account in the factor Window of opportunity (Access to biometric characteristics)).

Potential difficulties to have an access to the TOE in exploitation environment are taken into account in the factor Window of opportunity (Access to the TOE).

Expertise refers to the level of proficiency required by the attacker and the general knowledge that he possesses, not specific of the system being attacked. The levels are as follows:

Knowledge of the TOE refers to the amount of knowledge of the system required to perform the attack. For instance, format of the acquired samples, size and resolution of acquisition systems, specific format of templates, but also specifications and implementation of countermeasures are knowledge that could be required to set up an attack.

This information could be publicly available at the website of the capture device manufacturer or protected (distributed to stakeholders under non-disclosure agreement or even classified inside the company). The levels are as follows:

Special attention should be paid in this point to possible countermeasures that may be implemented in the system and whether it is necessary or not to have knowledge of their existence in order to be successful in a given attack.

It is assumed that all the knowledge required to perform the attack is gained during the identification phase and "scripted" for the exploitation. Therefore, this factor is not used for the exploitation phase.

Window of opportunity (Access to the TOE) refers to measuring the difficulty to access the TOE either to prepare the attack or to perform it on the target system.

For the identification phase, elements that should be taken into account include the easiness to buy the same biometric equipment (with and without countermeasures).

For exploitation phase, both technical (such as known/unknown tuning) and organizational measures (presence of a guard, ability to physically modify the target, limited number of tries, etc.) should be taken into account.

The number and the level of equipment requested to build the attack is also taken into account in this factor.

This factor is not expressed in terms of time. The levels are as follows:

Window of opportunity (Access to biometric characteristics) refers to measuring the difficulty to access the target biometric characteristics either to prepare the attack or to perform it on the target system

Security evaluations of CC are dedicated to evaluate the intrinsic resistance of a system. Due to the potential number of attack paths (with or without the cooperation of an enroled subject for example) the evaluation does not take into account the way a real biometric characteristic is acquired. For presentation attack detection, the vulnerability analysis is based on the hypothesis that a real "image" is available, and the rating only concerns the creation and the presentation of an artefact.

However, it is important to be able to compare the resistance of various systems, even based on different biometrics. In addition, getting a real "image" to build an artefact is clearly part of an attack and it is of interest, for the final user of the TOE and the pertinence of a certificate to add a factor related to this aspect.

The levels are as follows:

Equipment refers to the type of equipment required to perform the attack. This includes the biometric databases used (if any). The levels are follows:

Standard equipment is an orderable, easy to obtain and simple to operate equipment (e.g., computer, video cameras, mobile phones, "do it yourself" material, and artistic leisure materials).

Specialised equipment refers to fairly expensive equipment, not available in standard markets and which require of some specific formation to be used (e.g., laboratory equipment, advanced printer specific materials and inks, and advanced oscilloscopes).

Bespoke equipment refers to very expensive equipment with difficult and controlled access; for example, research printing systems with specific ink definition and flexible support adaptation. In addition, if more than one specialised equipment is required to perform different parts of the attack, this value should be used. Before using this level, it has to be carefully checked that no service is available (renting, limited time access, etc.). If such service exists, the level has to be moved down to Specialised level.

Table 5, “Calculation of attack potential for general biometric system” identifies the factors discussed in the previous Section and associates numeric values with the total value of each factor.

In order to calculate the attack potential value of the entire attack, the evaluator shall add all the values of all the factors in identification phase and exploitation phase. However, Table 5 is intended as a guide. Evaluator may modify the table with a proper justification.

The "Values" column of Table 6, “Rating of vulnerabilities and TOE resistance” indicates the range of attack potential values (calculated using Table 5, “Calculation of attack potential for general biometric system”) of an attack scenario that results in the SFRs being undermined.

The evaluator shall select one week at maximum because the evalator shall finish the penetration testing within one week.

The evaluator shall select “Easy” because the TOE is a computer that anyone can purchase.

The evaluator shall select “Moderate” because the number of unsuccessful authentication attempts for biometric verification is limited, and biometric verification becomes unusable if the number of failure attempts exceed the limit.