Approximate Nearest Neighbor library
Project description
N2 - approximate Nearest Neighbor
import numpy as np
from n2 import HnswIndex
N, dim = 10240, 20
samples = np.arange(N * dim).reshape(N, dim)
index = HnswIndex(dim)
for sample in samples:
index.add_data(sample)
index.build(m=5, n_threads=4)
print(index.search_by_id(0, 10))
# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]Install
To install:
sudo pip install n2For more detail, see the installation for instruction on how to build N2 from source.
Introduce
N2 is an approximate nearest neighborhoods algorithm library written in C++ (including Python/Go bindings). N2 provides a much faster search speed than other implementations when modeling large dataset. Also, N2 supports multi-core CPUs for index building.
Background
There are great approximate nearest neighborhoods libraries such as annoy and nmslib, but they did not fully meet the requirments to handle Kakao’s dataset. Therefore, we decided to implement a library that improves usability and performs better based on nmslib. And finally, we release N2 to the world.
Features
Efficient implementations. N2 is a lightweight library which runs faster even with large datasets.
Support multi-core CPUs for index building.
Support a mmap feature by default for handling large index files efficiently.
Support Python/Go bindings.
Performance
If you want to read about detail benchmark explanation. See the benchmark for more experiments.
Index build times
Search speed
Memory usage
Bindings
The following guides explain how to use N2 with basic examples and API.
References
“Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs” by Yu. A. Malkov, D. A. Yashunin (available on arxiv: http://arxiv.org/abs/1603.09320)
License
This software is licensed under the Apache 2 license, quoted below.
Copyright 2017 Kakao Corp. http://www.kakaocorp.com
Licensed under the Apache License, Version 2.0 (the “License”); you may not use this project except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0.
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file n2-0.1.5b0.tar.gz.
File metadata
- Download URL: n2-0.1.5b0.tar.gz
- Upload date:
- Size: 8.7 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/39.2.0 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/3.6.8
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 258057cd83d833855c8f9d3d659b8481b40fcfc1aaf86aaec67e99da2362d832 |
|
| MD5 | 20d975cf3f10144974f83d275c218102 |
|
| BLAKE2b-256 | 66dbf9700108c9ac8057f86d678570ecfcc23b7a614aa61c48805bc14c4a4118 |
