Corrfunc 2.3.1

Pre-requisites

1. make >= 3.80

2. OpenMP capable compiler like icc, gcc>=4.6 or clang >= 3.7. If not available, please disable USE_OMP option option in theory.options and mocks.options. You might need to ask your sys-admin for system-wide installs of the compiler; if you prefer to install your own then conda install gcc (MAC/linux) or (sudo) port install gcc5 (on MAC) should work.

3. gsl >= 2.4. Use either conda install -c conda-forge gsl (MAC/linux) or (sudo) port install gsl (MAC) to install gsl if necessary.

4. python >= 2.7 or python>=3.4 for compiling the C extensions.

5. numpy>=1.7 for compiling the C extensions.

Preferred Install Method

$ git clone https://github.com/manodeep/Corrfunc/
$ make
$ make install
$ python setup.py install (--user)
$ make tests

Assuming you have gcc in your PATH, make and make install should compile and install the C libraries + python extensions within the source directory. If you would like to install the python C extensions in your environment, then python setup.py install (--user) should be sufficient. If you are primarily interested in the python interface, you can condense all of the steps by using python setup.py install CC=yourcompiler (--user) after git clone.

Compilation Notes

• If python and/or numpy are not available, then the C extensions will not be compiled.

• make install simply copies files into the lib/bin/include sub-directories. You do not need root permissions

• Default compiler on MAC is set to clang, if you want to specify a different compiler, you will have to call make CC=yourcompiler, make install CC=yourcompiler, make tests CC=yourcompiler etc. If you want to permanently change the default compiler, then please edit the common.mk file in the base directory.

• If you are directly using python setup.py install CC=yourcompiler (--user), please run a make distclean beforehand (especially if switching compilers)

Alternate Install Method

The python package is directly installable via pip install Corrfunc. However, in that case you will lose the ability to recompile the code according to your needs. Installing via pip is not recommended, please open an install issue on this repo first; doing so helps improve the code-base and saves future users from running into similar install issues.

Installation notes

If compilation went smoothly, please run make tests to ensure the code is working correctly. Depending on the hardware and compilation options, the tests might take more than a few minutes. Note that the tests are exhaustive and not traditional unit tests.

While I have tried to ensure that the package compiles and runs out of the box, cross-platform compatibility turns out to be incredibly hard. If you run into any issues during compilation and you have all of the pre-requisites, please see the FAQ or email the Corrfunc mailing list. Also, feel free to create a new issue with the Installation label.

Clustering Measures on a Cosmological box

All codes that work on cosmological boxes with co-moving positions are located in the theory directory. The various clustering measures are:

1. DD – Measures auto/cross-correlations between two boxes. The boxes do not need to be cubes.

2. xi – Measures 3-d auto-correlation in a cubic cosmological box. Assumes PERIODIC boundary conditions.

3. wp – Measures auto 2-d point projected correlation function in a cubic cosmological box. Assumes PERIODIC boundary conditions.

4. DDrppi – Measures the auto/cross correlation function between two boxes. The boxes do not need to be cubes.

5. DDsmu – Measures the auto/cross correlation function between two boxes. The boxes do not need to be cubes.

6. vpf – Measures the void probability function + counts-in-cells.

Clustering measures on a Mock

All codes that work on mock catalogs (RA, DEC, CZ) are located in the mocks directory. The various clustering measures are:

1. DDrppi_mocks – The standard auto/cross correlation between two data sets. The outputs, DD, DR and RR can be combined using wprp to produce the Landy-Szalay estimator for wp(rp).

2. DDsmu_mocks – The standard auto/cross correlation between two data sets. The outputs, DD, DR and RR can be combined using the python utility convert_3d_counts_to_cf to produce the Landy-Szalay estimator for xi(s, mu).

3. DDtheta_mocks – Computes angular correlation function between two data sets. The outputs from DDtheta_mocks need to be combined with wtheta to get the full omega(theta)

4. vpf_mocks – Computes the void probability function on mocks.

Theory (in theory.options)

1. PERIODIC (ignored in case of wp/xi) – switches periodic boundary conditions on/off. Enabled by default.

2. OUTPUT_RPAVG – switches on output of <rp> in each rp bin. Can be a massive performance hit (~ 2.2x in case of wp). Disabled by default.

3. DOUBLE_PREC – switches on calculations in double precision. Disabled by default (i.e., calculations are performed in single precision by default).

Mocks (in mocks.options)

1. OUTPUT_RPAVG – switches on output of <rp> in each rp bin for DDrppi_mocks. Enabled by default.

2. OUTPUT_THETAAVG – switches on output of in each theta bin. Can be extremely slow (~5x) depending on compiler, and CPU capabilities. Disabled by default.

3. DOUBLE_PREC – switches on calculations in double precision. Disabled by default (i.e., calculations are performed in single precision by default).

4. LINK_IN_DEC – creates binning in declination for DDtheta. Please check that for your desired limits \theta, this binning does not produce incorrect results (due to numerical precision). Generally speaking, if your \thetamax (the max. \theta to consider pairs within) is too small (probaly less than 1 degree), then you should check with and without this option. Errors are typically sub-percent level.

5. LINK_IN_RA – creates binning in RA once binning in DEC has been enabled. Same numerical issues as LINK_IN_DEC

6. FAST_DIVIDE – Disabled by default. Divisions are slow but required DD(r_p,\pi). Enabling this option, replaces the divisions with a reciprocal followed by a Newton-Raphson. The code will run ~20% faster at the expense of some numerical precision. Please check that the loss of precision is not important for your use-case.

7. FAST_ACOS – Relevant only when OUTPUT_THETAAVG is enabled. Disabled by default. An arccos is required to calculate <\theta>. In absence of vectorized arccos (intel compiler, icc provides one via intel Short Vector Math Library), this calculation is extremely slow. However, we can approximate arccos using polynomials (with Remez Algorithm). The approximations are taken from implementations released by Geometric Tools. Depending on the level of accuracy desired, this implementation of fast acos can be tweaked in the file utils/fast_acos.h. An alternate, less accurate implementation is already present in that file. Please check that the loss of precision is not important for your use-case.

8. COMOVING_DIST – Currently there is no support in Corrfunc for different cosmologies. However, for the mocks routines like, DDrppi_mocks and vpf_mocks, cosmology parameters are required to convert between redshift and co-moving distance. Both DDrppi_mocks and vpf_mocks expects to receive a redshift array as input; however, with this option enabled, the redshift array will be assumed to contain already converted co-moving distances. So, if you have redshifts and want to use an arbitrary cosmology, then convert the redshifts into co-moving distances, enable this option, and pass the co-moving distance array into the routines.

Using the command-line interface

Navigate to the correct directory. Make sure that the options, set in either theory.options or mocks.options in the root directory are what you want. If not, edit those two files (and possibly common.mk), and recompile. Then, you can use the command-line executables in each individual subdirectory corresponding to the clustering measure you are interested in. For example, if you want to compute the full 3-D correlation function, \xi(r), then run the executable theory/xi/xi. If you run executables without any arguments, the program will output a message with all the required arguments.

Calling from C

Look under the run_correlations.c and run_correlations_mocks.c to see examples of calling the C API directly. If you run the executables, run_correlations and run_correlations_mocks, the output will also show how to call the command-line interface for the various clustering measures.

Calling from Python

If all went well, the codes can be directly called from python. Please see call_correlation_functions.py and call_correlation_functions_mocks.py for examples on how to use the C extensions directly. Here are a few examples:

from __future__ import print_function
import os.path as path
import numpy as np
import Corrfunc
from Corrfunc.theory import wp

# Setup the problem for wp
boxsize = 500.0
pimax = 40.0
nthreads = 4

# Create a fake data-set.
Npts = 100000
x = np.float32(np.random.random(Npts))
y = np.float32(np.random.random(Npts))
z = np.float32(np.random.random(Npts))
x *= boxsize
y *= boxsize
z *= boxsize

# Setup the bins
rmin = 0.1
rmax = 20.0
nbins = 20

# Create the bins
rbins = np.logspace(np.log10(0.1), np.log10(rmax), nbins + 1)

# Call wp
wp_results = wp(boxsize, pimax, nthreads, rbins, x, y, z, verbose=True, output_rpavg=True)

# Print the results
print("#############################################################################")
print("##       rmin           rmax            rpavg             wp            npairs")
print("#############################################################################")
print(wp_results)