bob.spear 1.1.5

1. Voice Activity Detection

This step aims to filter out the non speech part. Depending on the configuration file, several routines can be enabled or disabled.

• Energy-based VAD

• 4Hz Modulation energy based VAD

2. Feature Extraction

This step aims to extract features. Depending on the configuration file, several routines can be enabled or disabled.

• LFCC/MFCC feature extraction

• Spectrogram extraction

• Feature normalization

• HTK Feature reader

• SPro Feature reader

3. Universal Background Model Training and Projection

This step aims at computing the universal background model referenced as Projector. The training includes both k-means and ML steps. In the parallel implementation, the E (Estimation) step is split to run on parallel processes. Then, the computation of sufficient statistics in SPEAR is referenced as the projection-ubm step. It aims at projecting the cepstral features using the previously trained Projector.

4. Subspace Training and Projection

This steps aims at estimating the subspaces needed by ISV, JFA and I-Vector. The I-Vector can also be parallelized similarly to the UBM. The projection here is referenced by either projection-isv, projection-jfa, or projection-ivector. Notice that the I-Vector projection process is the extraction of the i-vectors.

5. Conditioning and Compensation

This steps is used by the I-Vector toolchain. It includes Whitening, Length Normalization, LDA and WCCN projection.

6. Model Enrollment

Model enrollment defines the stage, where several (projected or compensated) features of one identity are used to enroll the model for that identity. In the easiest case, the features are simply averaged, and the average feature is used as a model.

7. Scoring

In the final scoring stage, the models are compared to probe features and a similarity score is computed for each pair of model and probe. Some of the models (the so-called T-Norm-Model) and some of the probe features (so-called Z-Norm-probe-features) are split up, so they can be used to normalize the scores later on.

In addition, there are independent scripts for fusion and evaluation.

8. Fusion

The fusion of scores from different systems is done using logistic regression that should be trained normally on the development scores.

9. Evaluation

One way to compute the final result is to use the bin/evaluate.py e.g., by calling:

$ bin/evaluate.py -d PATH/TO/USER/DIRECTORY/scores-dev -e PATH/TO/USER/DIRECTORY/scores-eval -c EER -D DET.pdf -x

This will compute the EER, the minCLLR, CLLR, and draw the DET curve. To better compare different systems using DET curves, a separate script can be used like in this example:

$ ./bin/det.py -s gmm-scores isv-scores ivector-scores -n GMM ISV i-vectors

1. Voxforge dataset

Voxforge is a free database used in free speech recognition engines. We randomly selected a small part of the english corpus (< 1GB). It is used as a toy example for our speaker recognition tool since experiment can be easily run on a local machine, and the results can be obtained in a reasonnable amount of time (< 2h).

Unlike TIMIT and BANCA, this dataset is completely free of charge.

More details about how to download the audio files used in our experiments, and how the data is split into Training, Development and Evaluation set can be found here:

https://pypi-hypernode.com/pypi/xbob.db.voxforge

One example of command line is:

$ ./bin/spkverif_gmm.py -d config/database/voxforge.py -p config/preprocessing/energy.py \
 -f config/features/mfcc_60.py -t config/tools/ubm_gmm/ubm_gmm_256G.py -b ubm_gmm -z \
 --user-directory PATH/TO/USER/DIR --temp-directory PATH/TO/TEMP/DIR

In this example, we used the following configuration:

• Energy-based VAD,

• (19 MFCC features + Energy) + First and second derivatives,

• UBM-GMM Modelling (with 256 Gaussians), the scoring is done using the linear approximation of the LLR.

The performance of the system on DEV and EVAL are:

• DEV: EER = 2.00%

• EVAL: HTER = 1.65%

If you want to run the same experiment on SGE:

$ ./bin/spkverif_gmm.py -d config/database/voxforge.py -p config/preprocessing/energy.py \
 -f config/features/mfcc_60.py -t config/tools/ubm_gmm/ubm_gmm_256G.py -b ubm_gmm -z \
 --user-directory PATH/TO/USER/DIR --temp-directory PATH/TO/TEMP/DIR -g config/grid/grid.py

If you want to run the parallel implementation of the UBM on the SGE:

$ ./bin/para_ubm_spkverif_gmm.py -d config/database/voxforge.py -p config/preprocessing/energy.py \
  -f config/features/mfcc_60.py -t config/tools/ubm_gmm/ubm_gmm_256G.py -b ubm_gmm -z \
  --user-directory PATH/TO/USER/DIR --temp-directory PATH/TO/TEMP/DIR -g config/grid/para_training_sge.py

If you want to run the parallel implementation of the UBM on your local machine:

$ ./bin/para_ubm_spkverif_gmm.py -d config/database/voxforge.py -p config/preprocessing/energy.py \
  -f config/features/mfcc_60.py -t config/tools/ubm_gmm/ubm_gmm_256G.py -b ubm_gmm -z \
  --user-directory PATH/TO/USER/DIR --temp-directory PATH/TO/TEMP/DIR -g config/grid/para_training_local.py

$ bin/jman –local -vv run-scheduler –parallel 6

In this example, the number of nodes is 6.

Another example is to use ISV toolchain instead of UBM-GMM:

$ ./bin/spkverif_isv.py -d config/database/voxforge.py -p config/preprocessing/energy.py \
 -f config/features/mfcc_60.py -t config/tools/isv/isv_256g_u50.py  -z -b isv \
 --user-directory PATH/TO/USER/DIR --temp-directory PATH/TO/TEMP/DIR

• DEV: EER = 1.41%

• EVAL: HTER = 1.56%

One can also try JFA toolchain:

$ ./bin/spkverif_jfa.py -d config/database/voxforge.py -p config/preprocessing/energy.py \
 -f config/features/mfcc_60.py -t config/tools/jfa/jfa_256_v5_u10.py  -z -b jfa \
 --user-directory PATH/TO/USER/DIR --temp-directory PATH/TO/TEMP/DIR

• DEV: EER = 5.65%

• EVAL: HTER = 4.82%

or also IVector toolchain where Whitening, L-Norm, LDA, WCCN are used like in this example where the score computation is done using Cosine distance:

$ ./bin/spkverif_ivector.py -d config/database/voxforge.py -p config/preprocessing/energy.py \
 -f config/features/mfcc_60.py -t config/tools/ivec/ivec_256g_t100_cosine.py -z -b ivector_cosine \
 --user-directory PATH/TO/USER/DIR --temp-directory PATH/TO/TEMP/DIR

• DEV: EER = 15.33%

• EVAL: HTER = 15.78%

The scoring computation can also be done using PLDA:

$ ./bin/spkverif_ivector.py -d config/database/voxforge.py -p config/preprocessing/energy.py \
 -f config/features/mfcc_60.py -t config/tools/ivec/ivec_256g_t100_plda.py -z -b ivector_plda \
 --user-directory PATH/TO/USER/DIR --temp-directory PATH/TO/TEMP/DIR

• DEV: EER = 15.33%

• EVAL: HTER = 16.93%

Note that in the previous examples, our goal is not to optimize the parameters on the DEV set but to provide examples of use.

2. BANCA dataset

BANCA is a simple bimodal database with relatively clean data. The results are already very good with a simple baseline UBM-GMM system. An example of use can be:

$ bin/spkverif_gmm.py -d config/database/banca_audio_G.py -p config/preprocessing/energy.py \
  -f config/features/mfcc_60.py -t config/tools/ubm_gmm/ubm_gmm_256G_regular_scoring.py \
  --user-directory PATH/TO/USER/DIR --temp-directory PATH/TO/TEMP/DIR -z

The configuration in this example is similar to the previous one with the only difference of using the regular LLR instead of its linear approximation.

Here is the performance of this system:

• DEV: EER = 1.66%

• EVAL: EER = 0.69%

3. TIMIT dataset

TIMIT is one of the oldest databases (year 1993) used to evaluate speaker recognition systems. In the following example, the processing is done on the development set, and LFCC features are used:

$ ./bin/spkverif_gmm.py -d config/database/timit.py -p config/preprocessing/energy.py \
  -f config/features/lfcc_60.py -t config/tools/ubm_gmm/ubm_gmm_256G.py \
  --user-directory PATH/TO/USER/DIR --temp-directory PATH/TO/TEMP/DIR -b lfcc -z --groups dev

Here is the performance of the system on the Development set:

• DEV: EER = 2.68%

4. MOBIO dataset

This is a more challenging database. The noise and the short duration of the segments make the task of speaker recognition relatively difficult. The following experiment on male group (Mobile-0) uses the 4Hz modulation energy based VAD, and the ISV (with dimU=50) modelling technique:

$ ./bin/spkverif_isv.py -d config/database/mobio/mobile0-male.py -p config/preprocessing/mod_4hz.py \
 -f config/features/mfcc_60.py -t config/tools/isv/isv_u50.py \
 --user-directory PATH/TO/USER/DIR --temp-directory PATH/TO/TEMP/DIR -z

Here is the performance of this system:

• DEV: EER = 10.40%

• EVAL: EER = 10.36%

To generate the results presented in the ICASSP 2014 paper, please check the script included in the icassp folder of the toolbox. Note that the MOBIO dataset has different protocols, and that are all implemented in xbob.db.mobio. But in this toolbox, we provide separately mobile-0 protocol (into filelist format) for simplicity.

5. NIST SRE 2012

We first invite you to read the paper describing our system submitted to the NIST SRE 2012 Evaluation. The protocols on the development set are the results of a joint work by the I4U group. To reproduce the results, please check this dedicated package:

https://pypi-hypernode.com/pypi/spear.nist_sre12