Indexers Benchmark

In this page we show the performance capabilities of the different Indexers available in Jina Hub.

Results

In the below graphs you can see the performance, in terms of speed and evaluation. Speed is reported for query time, in total seconds. Evaluation is provided as recall, out of the top_k 100. The tables are sorted by time_search.

gist

Table

name

config

time_search

recall

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

10.115

0.857

Rii

{‘subspaces’: 96, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

10.238

0.671

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

10.266

0.878

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

10.280

0.879

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

10.475

0.914

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

10.515

0.858

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

10.663

0.943

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

10.693

0.939

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

10.872

0.561

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

10.890

0.911

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

10.893

0.914

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

11.065

0.561

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

11.068

0.561

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

11.090

0.939

Rii

{‘subspaces’: 192, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

11.244

0.768

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

11.286

0.611

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

11.306

0.942

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

11.324

0.911

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

11.476

0.611

Rii

{‘subspaces’: 192, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

11.494

0.737

Rii

{‘subspaces’: 96, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

11.672

0.649

Faiss

{‘index_key’: ‘IVF128,PQ32’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

11.987

0.499

Rii

{‘subspaces’: 120, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

12.102

0.736

Rii

{‘subspaces’: 96, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

12.108

0.649

Rii

{‘subspaces’: 96, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

12.128

0.697

Rii

{‘subspaces’: 120, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

12.303

0.736

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

12.377

0.583

Faiss

{‘index_key’: ‘IVF128,PQ32’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

12.391

0.510

Rii

{‘subspaces’: 120, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

12.539

0.736

Rii

{‘subspaces’: 120, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

12.566

0.736

Rii

{‘subspaces’: 192, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

13.361

0.802

Rii

{‘subspaces’: 192, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

13.549

0.802

Rii

{‘subspaces’: 192, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

13.746

0.838

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

13.781

0.599

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

13.854

0.611

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

13.854

0.561

Rii

{‘subspaces’: 192, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

13.946

0.838

Rii

{‘subspaces’: 120, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

14.659

0.689

Rii

{‘subspaces’: 96, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

14.734

0.649

Rii

{‘subspaces’: 120, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

15.036

0.689

Rii

{‘subspaces’: 96, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

15.058

0.649

Faiss

{‘index_key’: ‘IVF1024,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

15.182

0.617

Rii

{‘subspaces’: 96, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

15.293

0.697

Rii

{‘subspaces’: 120, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

15.488

0.689

Rii

{‘subspaces’: 120, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

15.681

0.689

Rii

{‘subspaces’: 96, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

15.795

0.697

Faiss

{‘index_key’: ‘IVF512,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

16.567

0.693

Rii

{‘subspaces’: 192, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

16.810

0.802

Faiss

{‘index_key’: ‘HNSW32’, ‘metric’: ‘euclidean’}

17.420

0.601

Rii

{‘subspaces’: 192, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

17.559

0.838

Faiss

{‘index_key’: ‘IVF1024,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

18.035

0.765

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

18.488

0.611

Faiss

{‘index_key’: ‘IVF512,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

21.598

0.832

Faiss

{‘index_key’: ‘IVF256,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

23.483

0.788

Faiss

{‘index_key’: ‘IVF128,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

30.475

0.862

Faiss

{‘index_key’: ‘IVF256,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

31.892

0.911

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

42.589

0.861

Faiss

{‘index_key’: ‘IVF64,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

48.416

0.926

Faiss

{‘index_key’: ‘IVF128,SQ4’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

51.615

0.802

Faiss

{‘index_key’: ‘IVF128,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

51.866

0.959

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

69.125

0.956

Faiss

{‘index_key’: ‘IVF64,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

85.781

0.988

Faiss

{‘index_key’: ‘IVF32,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

85.789

0.981

Faiss

{‘index_key’: ‘IVF128,SQ4’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

86.327

0.862

Faiss

{‘index_key’: ‘IVF32,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

148.602

1.000

Faiss

{‘index_key’: ‘Flat’, ‘metric’: ‘euclidean’}

241.103

1.000

Table

name

config

time_search

recall

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

9.639

0.381

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

10.052

0.458

Rii

{‘subspaces’: 120, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

10.241

0.482

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

10.260

0.786

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

10.271

0.785

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

10.456

0.413

Rii

{‘subspaces’: 96, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

10.460

0.538

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

10.464

0.807

Rii

{‘subspaces’: 96, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

10.478

0.490

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

10.685

0.422

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

10.712

0.807

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

10.862

0.867

Rii

{‘subspaces’: 120, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

10.865

0.576

Rii

{‘subspaces’: 96, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

10.869

0.495

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

10.893

0.874

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

11.052

0.903

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

11.095

0.868

Rii

{‘subspaces’: 120, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

11.211

0.530

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

11.280

0.910

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

11.297

0.874

Rii

{‘subspaces’: 120, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

11.306

0.518

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

11.306

0.910

Rii

{‘subspaces’: 96, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

11.320

0.452

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

11.523

0.903

Rii

{‘subspaces’: 192, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

11.656

0.601

Rii

{‘subspaces’: 192, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

11.891

0.569

Rii

{‘subspaces’: 192, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

12.490

0.641

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

13.155

0.455

Rii

{‘subspaces’: 192, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

13.702

0.681

Faiss

{‘index_key’: ‘IVF128,PQ32’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

14.883

0.408

Rii

{‘subspaces’: 96, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

15.369

0.608

Rii

{‘subspaces’: 120, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

15.470

0.596

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

15.670

0.500

Rii

{‘subspaces’: 96, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

15.958

0.608

Rii

{‘subspaces’: 120, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

15.966

0.654

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

16.701

0.455

Faiss

{‘index_key’: ‘IVF128,PQ32’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

16.733

0.415

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

16.917

0.508

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

17.995

0.508

Rii

{‘subspaces’: 96, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

18.159

0.554

Rii

{‘subspaces’: 96, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

18.497

0.554

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

18.524

0.512

Rii

{‘subspaces’: 192, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

19.309

0.736

Rii

{‘subspaces’: 120, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

19.466

0.654

Rii

{‘subspaces’: 192, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

20.501

0.786

Faiss

{‘index_key’: ‘IVF1024,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

22.263

0.589

Rii

{‘subspaces’: 192, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

22.674

0.736

Rii

{‘subspaces’: 192, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

24.885

0.786

Faiss

{‘index_key’: ‘IVF512,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

27.338

0.676

Faiss

{‘index_key’: ‘IVF1024,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

27.682

0.741

Faiss

{‘index_key’: ‘IVF256,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

39.996

0.779

Faiss

{‘index_key’: ‘IVF512,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

42.739

0.828

Faiss

{‘index_key’: ‘IVF128,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

61.930

0.856

Faiss

{‘index_key’: ‘IVF256,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

65.994

0.910

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

86.940

0.854

Faiss

{‘index_key’: ‘IVF64,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

106.795

0.926

Faiss

{‘index_key’: ‘IVF128,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

108.918

0.956

Faiss

{‘index_key’: ‘IVF128,SQ4’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

116.630

0.778

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

169.056

0.952

Faiss

{‘index_key’: ‘IVF32,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

193.191

0.974

Faiss

{‘index_key’: ‘IVF64,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

205.614

0.989

Faiss

{‘index_key’: ‘IVF128,SQ4’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

218.027

0.836

Faiss

{‘index_key’: ‘IVF32,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

337.916

0.999

Faiss

{‘index_key’: ‘Flat’, ‘metric’: ‘euclidean’}

578.059

1.000

Table

name

config

time_search

recall

Rii

{‘subspaces’: 96, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

10.266

0.324

Rii

{‘subspaces’: 120, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

10.451

0.353

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

10.454

0.726

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

10.475

0.271

Rii

{‘subspaces’: 120, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

10.650

0.388

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

10.681

0.303

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

10.695

0.706

Rii

{‘subspaces’: 96, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

10.870

0.375

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

10.886

0.304

Rii

{‘subspaces’: 96, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

10.905

0.361

Rii

{‘subspaces’: 96, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

11.049

0.415

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

11.055

0.812

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

11.072

0.821

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

11.098

0.725

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

11.309

0.813

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

11.314

0.344

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

11.323

0.707

Rii

{‘subspaces’: 120, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

11.465

0.409

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

11.468

0.864

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

11.473

0.853

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

11.494

0.821

Rii

{‘subspaces’: 120, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

11.510

0.450

Rii

{‘subspaces’: 192, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

11.643

0.424

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

11.933

0.853

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

12.096

0.864

Rii

{‘subspaces’: 192, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

12.266

0.458

Rii

{‘subspaces’: 192, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

13.087

0.511

Rii

{‘subspaces’: 192, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

13.294

0.545

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

16.971

0.367

Rii

{‘subspaces’: 96, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

20.023

0.464

Faiss

{‘index_key’: ‘IVF128,PQ32’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

20.409

0.337

Rii

{‘subspaces’: 96, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

20.663

0.464

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

20.772

0.367

Rii

{‘subspaces’: 96, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

20.828

0.522

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

21.248

0.417

Rii

{‘subspaces’: 120, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

21.335

0.508

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

21.450

0.417

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

22.162

0.429

Faiss

{‘index_key’: ‘IVF128,PQ32’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

22.224

0.342

Rii

{‘subspaces’: 120, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

22.509

0.572

Rii

{‘subspaces’: 96, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

24.615

0.522

Rii

{‘subspaces’: 120, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

25.297

0.508

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

26.422

0.438

Rii

{‘subspaces’: 120, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

26.527

0.572

Rii

{‘subspaces’: 192, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

29.036

0.671

Rii

{‘subspaces’: 192, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

31.064

0.727

Faiss

{‘index_key’: ‘IVF1024,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

32.628

0.573

Rii

{‘subspaces’: 192, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

33.497

0.671

Rii

{‘subspaces’: 192, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

36.443

0.727

Faiss

{‘index_key’: ‘IVF512,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

44.005

0.675

Faiss

{‘index_key’: ‘IVF1024,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

46.694

0.730

Faiss

{‘index_key’: ‘IVF256,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

68.386

0.772

Faiss

{‘index_key’: ‘IVF512,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

70.265

0.828

Faiss

{‘index_key’: ‘IVF128,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

108.011

0.847

Faiss

{‘index_key’: ‘IVF256,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

123.070

0.901

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

156.673

0.845

Faiss

{‘index_key’: ‘IVF128,SQ4’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

201.969

0.745

Faiss

{‘index_key’: ‘IVF128,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

207.494

0.950

Faiss

{‘index_key’: ‘IVF64,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

207.639

0.927

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

306.423

0.945

Faiss

{‘index_key’: ‘IVF64,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

366.850

0.988

Faiss

{‘index_key’: ‘IVF32,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

372.286

0.977

Faiss

{‘index_key’: ‘IVF128,SQ4’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

400.682

0.801

Faiss

{‘index_key’: ‘IVF32,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

723.329

0.999

Faiss

{‘index_key’: ‘Flat’, ‘metric’: ‘euclidean’}

1087.499

1.000

sift

Table

name

config

time_search

recall

Rii

{‘subspaces’: 32, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

17.097

0.851

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

18.920

0.988

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

18.964

0.996

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

19.120

0.993

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

19.130

0.996

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

19.140

0.993

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

19.153

0.987

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

19.354

0.988

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

19.389

0.986

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

19.587

0.996

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

19.742

0.993

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

19.802

0.993

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

20.222

0.996

Rii

{‘subspaces’: 32, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

20.706

0.830

Rii

{‘subspaces’: 32, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

20.730

0.830

Rii

{‘subspaces’: 32, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

21.184

0.861

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

21.759

0.918

Faiss

{‘index_key’: ‘IVF128,PQ32’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

21.931

0.845

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

21.931

0.940

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

21.983

0.940

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

22.063

0.940

Rii

{‘subspaces’: 128, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

22.155

0.972

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

22.212

0.918

Faiss

{‘index_key’: ‘IVF512,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

22.584

0.843

Faiss

{‘index_key’: ‘IVF1024,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

22.835

0.794

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

23.079

0.940

Faiss

{‘index_key’: ‘IVF256,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

23.150

0.896

Faiss

{‘index_key’: ‘IVF512,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

23.151

0.935

Faiss

{‘index_key’: ‘IVF1024,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

23.214

0.894

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

23.707

0.913

Faiss

{‘index_key’: ‘IVF128,PQ32’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

23.921

0.864

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

24.132

0.942

Rii

{‘subspaces’: 128, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

24.177

0.978

Rii

{‘subspaces’: 128, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

24.504

0.988

Faiss

{‘index_key’: ‘IVF128,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

24.761

0.947

Rii

{‘subspaces’: 128, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

24.870

0.978

Faiss

{‘index_key’: ‘IVF256,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

25.208

0.968

Rii

{‘subspaces’: 128, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

25.416

0.988

Rii

{‘subspaces’: 32, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

26.157

0.830

Rii

{‘subspaces’: 32, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

26.456

0.861

Rii

{‘subspaces’: 32, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

26.466

0.830

Rii

{‘subspaces’: 32, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

26.518

0.861

Faiss

{‘index_key’: ‘IVF128,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

27.352

0.990

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

27.546

0.945

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

27.580

0.918

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

27.856

0.918

Faiss

{‘index_key’: ‘IVF64,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

28.396

0.980

Faiss

{‘index_key’: ‘IVF128,SQ4’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

29.578

0.891

Faiss

{‘index_key’: ‘HNSW32’, ‘metric’: ‘euclidean’}

30.087

0.786

Rii

{‘subspaces’: 128, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

30.393

0.978

Rii

{‘subspaces’: 128, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

31.381

0.978

Rii

{‘subspaces’: 128, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

31.918

0.988

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

34.857

0.987

Faiss

{‘index_key’: ‘IVF32,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

36.722

0.996

Faiss

{‘index_key’: ‘IVF64,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

37.934

0.999

Faiss

{‘index_key’: ‘IVF128,SQ4’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

38.875

0.916

Faiss

{‘index_key’: ‘IVF32,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

46.228

1.000

Faiss

{‘index_key’: ‘Flat’, ‘metric’: ‘euclidean’}

70.977

1.000

SimpleIndexer

{‘match_args’: {‘metric’: ‘euclidean’, ‘only_id’: True}}

266.892

1.000

Table

name

config

time_search

recall

Rii

{‘subspaces’: 32, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

44.418

0.754

Rii

{‘subspaces’: 32, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

45.446

0.785

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

46.897

0.957

Rii

{‘subspaces’: 32, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

47.004

0.717

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

47.037

0.960

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

47.297

0.957

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

47.597

0.978

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

47.936

0.979

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

48.202

0.960

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

48.769

0.986

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

48.872

0.979

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

48.937

0.986

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

49.309

0.986

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

50.132

0.849

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

50.298

0.986

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

50.750

0.978

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

51.353

0.862

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

52.172

0.824

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

54.128

0.891

Rii

{‘subspaces’: 128, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

55.139

0.940

Rii

{‘subspaces’: 128, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

57.199

0.885

Rii

{‘subspaces’: 32, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

57.658

0.763

Rii

{‘subspaces’: 128, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

58.818

0.890

Rii

{‘subspaces’: 32, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

59.657

0.763

Rii

{‘subspaces’: 32, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

60.307

0.803

Rii

{‘subspaces’: 128, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

61.121

0.949

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

64.772

0.882

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

67.142

0.912

Faiss

{‘index_key’: ‘IVF1024,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

68.208

0.749

Faiss

{‘index_key’: ‘IVF128,PQ32’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

68.521

0.788

Faiss

{‘index_key’: ‘IVF1024,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

69.216

0.873

Faiss

{‘index_key’: ‘IVF512,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

72.808

0.826

Rii

{‘subspaces’: 32, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

73.895

0.763

Faiss

{‘index_key’: ‘IVF256,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

74.120

0.893

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

75.032

0.887

Rii

{‘subspaces’: 32, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

75.734

0.803

Faiss

{‘index_key’: ‘IVF512,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

76.672

0.930

Faiss

{‘index_key’: ‘IVF128,PQ32’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

78.513

0.805

Rii

{‘subspaces’: 128, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

79.814

0.970

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

80.495

0.914

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

81.776

0.882

Rii

{‘subspaces’: 128, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

82.068

0.970

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

83.783

0.912

Rii

{‘subspaces’: 128, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

83.940

0.982

Faiss

{‘index_key’: ‘IVF128,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

84.118

0.944

Faiss

{‘index_key’: ‘HNSW32’, ‘metric’: ‘euclidean’}

84.434

0.638

Faiss

{‘index_key’: ‘IVF256,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

88.011

0.969

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

103.199

0.941

Rii

{‘subspaces’: 128, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

106.225

0.982

Faiss

{‘index_key’: ‘IVF64,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

110.428

0.982

Faiss

{‘index_key’: ‘IVF128,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

112.318

0.990

Faiss

{‘index_key’: ‘IVF128,SQ4’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

120.859

0.855

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

141.645

0.985

Faiss

{‘index_key’: ‘IVF64,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

146.081

0.999

Faiss

{‘index_key’: ‘IVF32,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

146.651

0.997

Faiss

{‘index_key’: ‘IVF128,SQ4’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

164.157

0.879

Faiss

{‘index_key’: ‘IVF32,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

221.713

1.000

Faiss

{‘index_key’: ‘Flat’, ‘metric’: ‘euclidean’}

361.638

1.000

SimpleIndexer

{‘match_args’: {‘metric’: ‘euclidean’, ‘only_id’: True}}

1324.322

1.000

Table

name

config

time_search

recall

Rii

{‘subspaces’: 32, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

94.363

0.677

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

94.770

0.923

Rii

{‘subspaces’: 32, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

94.798

0.719

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

95.335

0.926

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

95.673

0.926

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

96.769

0.963

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

96.972

0.973

Rii

{‘subspaces’: 32, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

96.997

0.658

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

97.110

0.974

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

97.228

0.963

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

97.376

0.962

Rii

{‘subspaces’: 32, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

97.391

0.620

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

97.518

0.923

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

97.880

0.962

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

100.554

0.724

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

101.271

0.974

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

103.660

0.973

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

105.981

0.744

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

110.398

0.799

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

110.678

0.829

Rii

{‘subspaces’: 128, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 10000}

112.749

0.872

Rii

{‘subspaces’: 128, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 10000}

115.281

0.878

Rii

{‘subspaces’: 128, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 10000}

115.651

0.771

Rii

{‘subspaces’: 128, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 10000}

116.964

0.775

Rii

{‘subspaces’: 32, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

141.953

0.720

Rii

{‘subspaces’: 32, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

149.069

0.768

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

167.034

0.860

Rii

{‘subspaces’: 32, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

176.633

0.768

Faiss

{‘index_key’: ‘IVF128,PQ32’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

177.007

0.748

Rii

{‘subspaces’: 32, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

180.531

0.720

Faiss

{‘index_key’: ‘IVF512,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

181.653

0.832

Faiss

{‘index_key’: ‘IVF1024,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

184.510

0.756

Faiss

{‘index_key’: ‘IVF1024,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

185.060

0.878

Faiss

{‘index_key’: ‘IVF128,PQ32’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

193.135

0.763

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

203.210

0.869

Faiss

{‘index_key’: ‘IVF512,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

207.564

0.932

Faiss

{‘index_key’: ‘IVF256,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

208.289

0.895

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

209.611

0.896

Faiss

{‘index_key’: ‘HNSW32’, ‘metric’: ‘euclidean’}

211.001

0.616

Rii

{‘subspaces’: 64, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

212.089

0.860

Rii

{‘subspaces’: 64, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

215.622

0.896

Rii

{‘subspaces’: 128, ‘codewords’: 128, ‘cluster_center’: 1000, ‘candidates’: 50000}

224.690

0.961

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

227.490

0.894

Rii

{‘subspaces’: 128, ‘codewords’: 128, ‘cluster_center’: 5000, ‘candidates’: 50000}

234.007

0.957

Faiss

{‘index_key’: ‘IVF256,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

250.604

0.968

Faiss

{‘index_key’: ‘IVF128,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

259.265

0.946

Rii

{‘subspaces’: 128, ‘codewords’: 256, ‘cluster_center’: 1000, ‘candidates’: 50000}

281.552

0.977

Rii

{‘subspaces’: 128, ‘codewords’: 256, ‘cluster_center’: 5000, ‘candidates’: 50000}

282.742

0.977

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

322.146

0.941

Faiss

{‘index_key’: ‘IVF128,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

332.204

0.990

Faiss

{‘index_key’: ‘IVF64,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

359.077

0.982

Faiss

{‘index_key’: ‘IVF128,SQ4’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

383.529

0.832

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

468.810

0.983

Faiss

{‘index_key’: ‘IVF32,Flat’, ‘nprobe’: 8, ‘metric’: ‘euclidean’}

504.964

0.996

Faiss

{‘index_key’: ‘IVF64,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

535.492

0.999

Faiss

{‘index_key’: ‘IVF128,SQ4’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

562.634

0.854

Faiss

{‘index_key’: ‘IVF32,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

834.854

1.000

Faiss

{‘index_key’: ‘Flat’, ‘metric’: ‘euclidean’}

1437.081

1.000

SimpleIndexer

{‘match_args’: {‘metric’: ‘euclidean’, ‘only_id’: True}}

4843.714

1.000

sift 1 billion

This has only been run with sub-samples, not with the full dataset.

Table

name

config

time_search

recall

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

95.291

0.971

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

97.240

0.972

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

97.672

0.972

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

100.603

0.992

Hnsw

{‘max_connection’: 64, ‘ef_construction’: 400, ‘ef_query’: 100, ‘metric’: ‘euclidean’}

105.235

0.995

Faiss

{‘index_key’: ‘IVF512,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

1156.892

0.966

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

1359.404

0.936

Faiss

{‘index_key’: ‘IVF256,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

1638.537

0.983

Faiss

{‘index_key’: ‘IVF128,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

2323.912

0.994

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

3543.806

0.990

Faiss

{‘index_key’: ‘IVF64,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

3866.858

0.998

Faiss

{‘index_key’: ‘IVF32,Flat’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

6918.610

1.000

Table

name

config

time_search

recall

Hnsw

{‘max_connection’: 16, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

95.436

0.927

Hnsw

{‘max_connection’: 48, ‘ef_construction’: 200, ‘ef_query’: 50, ‘metric’: ‘euclidean’}

99.769

0.976

Faiss

{‘index_key’: ‘IVF128,PQ64’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

2745.310

0.908

Faiss

{‘index_key’: ‘IVF128,SQ8’, ‘nprobe’: 16, ‘metric’: ‘euclidean’}

6902.018

0.988

Indexers

The purpose is to compare how they each scale, both with the amount of Documents indexed and with the embedding dimensionality. We have restricted our selection to the following:

SimpleIndexer is based on doing exhaustive search across all the Documents. However, FAISS, HNSW and Rii are based on different approximate nearest neighbor approaches. For each of these, we provide several configurations of parameters, in order to ascertain which behaves best.

Datasets

We use three different datasets, each with different number of samples or dimensionality of vectors. They are all based at the IRISA website. The datasets are:

datasetindex vectorsdimensionalityquery vectorstop_k
sift1 million12810k100
gist1 million9601k100
sift1b1 billion12810k1000

Note that out of the 1 billion dataset, we have only run a sub-sample of 10 and 20 million. We have also only used a top_k value of 100. The datasets provide ground-truths for the query vectors.

Methodology

For each combination of configuration and dataset, we index a gradual subsampling of the dataset into the specific Indexer. Indexing batch size is set to 1000. The FAISS Indexer requires training. This is done with a further sub-sampling of the index set. This depended on the configuration parameters for the FAISS model. Then we search with the respective search set, using a batch size of 1, to mimic single query operations.