Supporting Document Mandatory Technical Document: Evaluation Activities for collaborative PP-Module for Mobile biometric enrolment and verification - for unlocking the device -

For definitions of standard CC terminology see [CC1]. For definitions of biometrics and mobile device, see [BIOPP-Module] and [MDFPP].

The evaluator shall refer the EA for FIA_MBV_EXT.3 to perform evaluation of this SFR.

The evaluator shall refer the EA for FCS_CKM_EXT.4 in [MDFPP] to perform evaluation of this SFR.

[BIOPP-Module] defines the assumptions for the mobile device that is the operational environment of the biometric system. These assumptions are implicitly satisfied if the mobile device is successfully evaluated based on [MDFPP] and the operational guidance doesn’t need to describe the security measures to be followed in order to fulfil the security objectives for the operational environment derived from those assumptions.

There is additional application note related to EAs for FIA_MBV_EXT.3 in Section 9.3.2, “Additional application notes for AGD Class for FIA_MBV_EXT.3”. The evaluator shall also follow this note depending on the result of the penetration testing for PAD.

[BIOPP-Module] supposes that the biometric system is fully integrated into the mobile device and the preparative procedures are unnecessary for [BIOPP-Module]. Therefore, AGD_PRE.1 deems satisfied for [BIOPP-Module].

[BIOPP-Module] is intended to be used with [MDFPP] and reference for the mobile device can be used as the TOE (mobile device + biometric system) reference only if the reference for the mobile device also uniquely identifies the biometric system embedded in the mobile device.

Same EA should be applied to [BIOPP-Module] except SFR FIA_MBV_EXT.3 (Presentation attack detection for mobile biometric verification). The evaluator shall perform EAs defined in Section 6, “Evaluation Activities for FIA_MBV_EXT.3” for FIA_MBV_EXT.3.

Same EA should be applied to [BIOPP-Module] except SFR FIA_MBV_EXT.3 (Presentation attack detection for mobile biometric verification). The evaluator shall perform EAs defined in Section 6, “Evaluation Activities for FIA_MBV_EXT.3” for FIA_MBV_EXT.3.

As described in previous section, [Toolbox] defines test items to create a representative set of PAIs that the evaluator shall use for the testing. During ATE_IND.1 evaluation, the evaluator shall conduct all test items in [Toolbox] for the selected modalities without any change. The evaluator is not allowed to skip any test items in the [Toolbox] to maintain compatibility between different evaluations.

During the independent testing, the evaluator may find PAIs that are incorrectly matched to the enrolled target user however, the evaluator may not be able to reliably reproduce a successful presentation attack.

[Toolbox] defines the Pass/Fail criteria, maximum attack presentation match rate for PAIs. The evaluator shall follow the [Toolbox] criteria for the number of PAI presentations and confirm that the TOE’s match rate is below the specified criteria during the independent testing. The evaluator shall assign fail verdict to those TOE that doesn’t satisfy the criteria.

The PAIs that pass the criteria but show the higher attack presentation match rate will be tested again during the AVA_VAN.1 evaluation.

[Toolbox] does not necessarily cover all biometric modalities. If the developer wants to evaluate modalities not currently included in [Toolbox], the developer and evaluator shall contact to the BIO-iTC to work together to extend [Toolbox]. Upon the BIO-iTC approval of this extension, the evaluator can proceed with PAD evaluation for new modality.

The EAs presented in this section are derived from ATE_IND.1-3, ATE_IND.1-4 and ATE_IND.1-7 and their verdicts will be associated with those work units.

[Toolbox] describes a test subset and test documentation that is sufficiently detailed to enable the tests to be reproducible (ATE_IND.1-3 and ATE_IND.1-4). [Toolbox] also defines Pass/Fail criteria that support evaluator’s decision (ATE_IND.1-7).

This Section describes EAs for AVA_VAN.1 step by step following the order of AVA_VAN.1 CEM work units.

The EAs presented in this section are derived from AVA_VAN.1-3, AVA_VAN.1-4, AVA_VAN.1-5, AVA_VAN.1-6, AVA_VAN.1-7 and AVA_VAN.1-10 and their verdicts will be associated with those work units.

EAs in the Section 6.3.1.1, “Search for new PAIs” and Section 6.3.1.2, “Identify candidate PAIs for testing” complements evaluator’s action for searching publicly available information and identifying potential vulnerabilities (e.g. new PAI) (AVA_VAN.1-3, AVA_VAN.1-4 and AVA_VAN.1-5).

EAs in Section 6.3.1.3, “Produce test plan” and Section 6.3.1.4, “Conduct the penetration testing” complements evaluator’s action for creating the test plan and conducting the penetration testing for PAD (AVA_VAN.1-6 and AVA_VAN.1-7)

EAs in Section 6.3.1.5, “Determine Pass/Fail of penetration testing” provides specific guidance for pass or failure of the testing (AVA_VAN.1-10).

The developer shall report following general information about the performance testing.

Test document shall specify a target application modelled in the test, such as mobile biometric verification in an indoor office environment with a habituated crew.

Test document shall also report influential factors that may influence performance, measures to control such factors and under what factors the performance testing was conducted.

Influential factors can be determined by referring appropriate documents (e.g. [ISO19795-3]) or referring the product datasheet (e.g. operating temperature). These factors should be consistent with the target application.

The following factors are examples of controlling factors for finger/hand vein verification. The developer shall define these factors properly, for example, based on [ISO19795-3]. Any information that are useful in the context of the used biometric modality shall be considered by the developer to determine the factors.

It’s recommended to control all influential factors appropriately because different error rates may be measured under different influential factors.

Selection method of test subjects shall be reported (e.g. gather test subjects from developer’s employees or recruit them from public). It is recommended that demographics of test subjects follow the target application.

Instructions and training given to the test subjects shall be reported. The same instructions and training shall be given to the all test subjects.

The following information about test subject management shall be reported. Proper management is necessary to avoid human errors that may occur during the testing.

A test protocol for the testing shall be reported. The following items shall be covered.

The developer shall follow the guidance in this Section to define the transaction if the developer selects FAR and FRR in FIA_MBV_EXT.1 or to define the number of samples per each test subject if the developer selects FMR and FNMR in FIA_MBV_EXT.1.

The user may use the mobile biometric verification in a different way.

Suppose the mobile device provides both Password Authentication Factor and BAF and user can use either of factor to unlock the device. One user may try to unlock the device with BAF until allowable maximum number of unsuccessful authentication attempts is exceeded. Another user may try to unlock the device with BAF only three times and switch to the password if all three attempts were failed.

It may also be possible for user to enrol multiple body parts (e.g. index and thumb fingerprint) or single body part for mobile biometric verification.

However, it’s not possible to evaluate all these scenarios to measure the performance but the developer shall refer the ST that claims conformance to [MDFPP] to define the scenario.

For example, if the ST sets the maximum number of unsuccessful authentication attempts for mobile fingerprint verification to five, the developer shall assume that the attacker makes all five fingerprint unlock attempts in succession to try to unlock the mobile device.

This means that if FAR and FRR are selected, the developer shall define that the genuine and imposter transaction is consisted up to five unlock attempts and only one transaction can be run by each user.

If FMR and FNMR are selected, the developer may follow the same scenario and collect five samples from each test subject. However, FMR/FNMR is a comparison subsystem measure while FAR/FRR is a system level measure, therefore FAR/FRR should be selected in the ST if the developer considers the specific test scenario to measure the performance.

The developer shall also select the most common scenario among users to conduct the performance testing. For example, if the user can enrol multiple fingerprints, the developer should assume that the user enrols index and thumb fingerprint if such enrolment is most common. FAR may increase and FRR may decrease if the user enrols multiple fingerprints however, performance of widely used configuration should be measured.

Only one template can be generated from each body part (e.g. right index fingerprint, left hand vein or face) of test subject and used for the performance testing.

Quality of template may have significant impact on the mobile biometric verification performance. This SD assumes that the user is familiar with the mobile devices operation and enrol him/herself correctly following the AGD guidance provided by the developer. The test subject may make enough number of practice attempts to get familiar with the device operation before the final enrolment transaction.

The developer shall define the maximum number of samples per test subject to be collected following the guidance provided in Section 8.1.1, “Test scenario for mobile biometric verification”.

Only one transaction can be run by each test subject because the mobile device locks the mobile biometric verification as required by [MDFPP] after the certain number of attempts are failed.

FMR/FAR shall be estimated following rule of 3 or 30 because these errors are most relevant to the security of the TOE and the trustworthiness of those values shall be evaluated statistically. While the rule of 3 would require that one test subject is only involved in one impostor transaction, it is commonly agreed that the statistical loss of computing all possible cross-comparisons between test subjects is acceptable. This SD allows full cross-comparison to estimate FAR/FMR.

This SD also allows cross-comparison of attempts/templates for ordered pair if there is no explicit reason that this cross-comparison hinders the accuracy of the result of performance testing. Cross-comparison of attempts/templates for ordered pair allows to compare between user A’s template and user B’s sample and user A’s sample and user B’s template separately. However, if the TOE’s verification algorithm is symmetric and make no distinction between the ordered pair, this assumption can’t be used.

This SD doesn’t allow intra-individual comparison that is a comparison between one body part and another body part of the same test subject (e.g. comparison between right and left iris of the same user).

Rule of 3 requires no error occurred for all attempts/transactions and rule of 30 may require too many attempts/transactions if the FNMR/FRR is quite low. Therefore, the developer may calculate FNMR/FRR directly from the result of performance testing without considering the statistical confidence.

As in [CEM], the factors to be considered consist of Elapsed time, Expertise, Knowledge of the TOE, Window of opportunity, and Equipment. But Window of opportunity is divided into two subfactors Window of opportunity (Access to the TOE) and Window of opportunity (Access to biometric characteristics).

Elapsed time is the total amount of time taken by the attacker.

In the identification phase, elapsed time corresponds to the time required to create the attack, and to demonstrate that it can be successfully applied to the TOE (including setting up or building any necessary hardware or software equipment). The demonstration that the attack can be successfully applied needs to consider any difficulties in expanding a result shown in the laboratory to create a useful attack. One of the outputs from identification is, for instance, a script that gives a step-by-step description of how to carry out the attack. This script is assumed to be used in the exploitation part.

In the exploitation phase, elapsed time corresponds to the time necessary to apply the "script" to specific biometric characteristics. For example, for a presentation attack to a fingerprint capture device, it corresponds to the time required to create a PAI from an image of a print (and not the acquisition of this image which is taken into account in the factor Window of opportunity (Access to biometric characteristics)).

Potential difficulties to have an access to the TOE in exploitation environment are taken into account in the factor Window of opportunity (Access to the TOE).

Expertise refers to the level of proficiency required by the attacker and the general knowledge that he possesses, not specific of the system being attacked. The levels are as follows:

Knowledge of the TOE refers to the amount of knowledge of system required to perform the attack. For instance, format of the acquired samples, size and resolution of acquisition systems, specific format of templates, but also specifications and implementation of countermeasures are knowledge that could be required to set up an attack.

This information could be publicly available at the website of the capture device manufacturer or protected (distributed to stakeholders under non-disclosure agreement or even classified inside the company). The levels are as follows:

Special attention should be paid in this point to possible countermeasures that may be implemented in the system and whether it is necessary or not to have knowledge of their existence in order to be successful in a given attack.

It is assumed that all the knowledge required to perform the attack is gained during the identification phase and "scripted" for the exploitation. Therefore, this factor is not used for the exploitation phase.

Window of opportunity (Access to the TOE) refers to measuring the difficulty to access the TOE either to prepare the attack or to perform it on the target system.

For the identification phase, elements that should be taken into account include the easiness to buy the same biometric equipment (with and without countermeasures).

For exploitation phase, both technical (such known/unknown tuning) and organizational measures (presence of a guard, ability to physically modify the target, limited number of tries, etc.) should be taken into account.

The number and the level of equipment requested to build the attack is also taken into account in this factor.

This factor is not expressed in terms of time. The levels are as follows:

Window of opportunity (Access to biometric characteristics) refers to measuring the difficulty to access the target biometric characteristics either to prepare the attack or to perform it on the target system

Security evaluations of [CC] are dedicated to evaluate the intrinsic resistance of a system. Due to the potential number of attack paths (with or without the cooperation of an enrolled subject for example) the evaluation does not take into account the way a real biometric characteristic is acquired. For presentation attack detection, the vulnerability analysis is based on the hypothesis that a real "image" is available, and the rating only concerns the creation and the presentation of a PAI.

However, it is important to be able to compare the resistance of various systems, even based on different biometrics. In addition, getting a real "image" to build a PAI is clearly part of an attack and it is of interest, for the final user of the TOE and the pertinence of a certificate to add a factor related to this aspect.

The levels are as follows:

Equipment refers to the type of equipment required to perform the attack. This includes the biometric databases used (if any). The levels are follows:

Standard equipment is an orderable, easy to obtain and simple to operate equipment (e.g., computer, video cameras, mobile phones, "do it yourself" material, and artistic leisure materials).

Specialised equipment refers to fairly expensive equipment, not available in standard markets and which require of some specific formation to be used (e.g., laboratory equipment, advanced printer specific materials and inks, and advanced oscilloscopes).

Bespoke equipment refers to very expensive equipment with difficult and controlled access; for example, research printing systems with specific ink definition and flexible support adaptation. In addition, if more than one specialised equipment is required to perform different parts of the attack, this value should be used. Before using this level, it has to be carefully checked that no service is available (renting, limited time access, etc.). If such service exists, the level has to be moved down to Specialised level.

Table 3, “Calculation of attack potential for general biometric system” identifies the factors discussed in the previous Section and associates numeric values with the total value of each factor.

In order to calculate the attack potential value of the entire attack, the evaluator shall add all the values of all the factors in identification phase and exploitation phase.

The "Values" column of Table 4, “Rating of vulnerabilities and TOE resistance” indicates the range of attack potential values (calculated using Table 3, “Calculation of attack potential for general biometric system”) of an attack scenario that results in the SFRs being undermined.

The evaluator shall select “Easy” because the TOE is the mobile device that anyone can purchase.

The evaluator shall select “Difficult” because number of unsuccessful authentication attempts for mobile biometric verification is limited, and mobile biometric verification become unusable if the number of failure attempts exceed the limit.

The mobile device limits the number of unsuccessful authentication attempts for mobile biometric verification, as required by [MDFPP]. Therefore, the attacker must succeed the presentation attack at least one time within this limit.

This SD assumes that the attacker actually performs the presentation attack only if the attacker can create the “Reliable PAIs”. “Reliable PAIs” are those PAIs that succeed at least one attack within the allowable number of attempts (i.e. succeed to unlock the mobile device) at more than 80% of probability. This SD selects this probability based on the use case assumed in [BIOPP-Module].

The probability of a successful presentation attack for one attempt p needs to satisfy the following equation to satisfy the above condition.

1-(1-p)ⁿ > 0.8 (n = allowable number of unsuccessful attempts)

The following table shows that example of pairs (maximum p for particular n) that satisfy the above equation.

The evaluator shall set n based on the assignment in FIA_AFL_EXT.1 in the ST that claim conformance to [MDFPP]. If the ST assign 5 to the maximum number of unsuccessful attempts for mobile biometric verification, n should be 5. If the ST states that this number is configurable from 5 to 10, the evaluator shall assume the worst-case scenario and n should be 10.

The evaluator shall assign pass verdict to the TOE only if the evaluator can’t find those PAIs that the probability of successful attack is more than p.

The evaluator shall make at least 3 PAIs from three test subjects following the same creation process and perform at least 10 attempts for each PAI to calculate p (i.e. minimum number of attempts for calculation of p for each PAI is 3 * 10 = 30).

The evaluator should focus on a few PAIs that show highest error rate at the independent testing or hold highest quality for the penetration testing and spend enough time for training before conducting the final testing to measure p for those PAIs.

CEM work unit AGD_OPE.1-1 requires the evaluator to examine the AGD guidance to determine that it describes appropriate warnings for secure use of the TOE.

The evaluator shall examine that appropriate warnings is provided in the AGD guidance if the evaluator can find those PAIs that pass the penetration test however whose p is higher than 7%.

Those PAIs can succeed at least one presentation attack (and succeed to unlock the mobile device) at 25% of probability when allowable number of unsuccessful attempts is 4 (i.e. n = 4).

Example of warnings is that the AGD guidance may warn that the mobile biometric verification is less secure than a password and recommend using a password for security sensitive services.