Skip to main content

dips: detrending periodic signals

Project description

Detrending Periodic Signals (dips)
==================================

_dips_ is an algorithm for detrending timeseries of strictly periodic signals. It does not assume any functional form for the signal or the background or the noise; it disentangles the strictly periodic component from everything else. We use it in astronomy for detrending _Kepler_, _K2_ and _TESS_ timeseries of periodic variable stars, eclipsing binary stars, exoplanets etc. The algorithm is described in detail in Prsa et al. (2019), PASP, in review -- the reference will be updated shortly.

This repository contains a python implementation of _dips_.

Pre-requisites
--------------

To run _dips_, you will need:

* python
* numpy
* scipy

Note that you will need python 3.3 or later to enable multiprocessing. All examples below assume that you have python 3.3 or later installed.

Installation
------------

The `dips` program is available from pip. To install, run `pip3 install dips` for a local install, or `sudo pip3 install dips` for a global install.

If you prefer to install `dips` manually, grab the tarball from github, extract it and run `python3 setup.py install` in the top-level `dips` directory (local install), or `sudo python3 setup.py install` in the top-level `dips` directory (global install).

Running _dips_
--------------

The _dips_ program is run from the command line. It takes a filename with the timeseries as input and computes the disentangled synchronous and asynchronous components of the signal as output. The disentangling process is iterative and might take an appreciable amount of time, depending on data length and pdf bin size.

Run _dips_ with:

`dips.py [-h] [-V] [-b BINS] [-t0 ORIGIN] [-P PERIOD] [-l LOGFILE] [-eta TOLERANCE] [-dxk DIFFERENCE] [-xi STEP_SIZE] [-af ATTENUATION] [--allow-upstep] [--cols COLS [COLS ...]] [--disable-mp] [--initial-pdf INITIAL_PDF] [--interim-prefix INTERIM_PREFIX] [--jitter JITTER] [--output-prefix OUTPUT_PREFIX] [--renormalize] [--save-interim SAVE_INTERIM] [--yonly] finput`

The arguments are summarized in the table below.

| Argument | Usage | Type | Default value |
|----------|-------|------|---------------|
| -h, --help | print out the help message and exit | n/a | n/a |
| -V, --version | print dips version and exit | n/a | n/a |
| -b BINS, --bins BINS | assign the number of synchronous pdf bins | int | 200 |
| -t0 ORIGIN, --origin ORIGIN | the zero-point of the timeseries | float | 0.0 |
| -P PERIOD, --period PERIOD | period of the synchronous signal | float | 1.0 |
| -l LOGFILE, --logfile LOGFILE | log file to send output to instead of screen | str | None |
| -eta TOLERANCE, --tolerance TOLERANCE | tolerance for convergence | float | 1e-8 |
| -dxk DIFFERENCE, --difference DIFFERENCE | finite difference size for computing slopes | float | 2e-5 |
| -xi STEP_SIZE, --step-size STEP_SIZE | initial down-step multiplier | float | 1e-3 |
| -af ATTENUATION, --attenuation ATTENUATION | attenuation factor for xi | float | 0.9 |
| --allow-upstep | allow step size to increase during convergence | bool | False |
| --cols COL1 COL2 \[COL3\] | a list of input columns to be parsed, starting from 0 | list of ints | 0 1 |
| --disable-mp | disable multiprocessing (force serial computation) | bool | False |
| --initial-pdf | choice of pdf initialization ('flat', 'mean', 'median', 'random', or external filename) | str | 'median' |
| --interim-prefix | filename prefix for interim results | str | finput |
| --normalize-data | normalize input data by median-dividing the fluxes | bool | False |
| --output_prefix PREFIX | filename prefix for saving results (PREFIX.signal, .trend, .ranges) | str | finput |
| --renormalize | force pdf normalization to 1 after every iteration | bool | False |
| --save-interim STEP | save intering solutions every STEP iterations | int | 0 |
| --yonly | use only y-distance instead of full euclidian distance | bool | False |

Distributed with _dips_ (in the tarball's `examples` directory) are three example input files, `synthetic.data`, `kic3953981_sap.data` and `kic3547874_sap.data`.

To run _dips_ on synthetic data (see [here](http://keplerEBs.villanova.edu/includes/DPS/dps_synthetic.html) how the data were created) by using 33 bins, per-bin means as the initial pdf, and with serial calculation (disabling multiprocessing), issue:

```bash
dips synthetic.data -b 33 -P 0.91 --initial-pdf mean --disable-mp
```

To run _dips_ on an eclipsing binary [KIC 3953981](http://keplerEBs.villanova.edu/overview/?k=3953981), using 101 bins, allowing the step size to increase, using per-bin data median as the initial pdf, renormalizing the pdf after each iteration, using only y-direction length and saving every 10th iteration, issue:

```bash
dips kic3953981_sap.data -b 101 -t0 54953.82253243 -P 0.49201716 --allow-upstep --initial-pdf median --save-interim 10 --interim-prefix eb --renormalize --yonly
```

Finally, to run _dips_ on a heartbeat star [KIC 3547874](http://keplerEBs.villanova.edu/overview/?k=3547874), using 200 bins, starting with a flat pdf, computing total length in the y-direction only, renormalizing the synchronous pdf to 1.0 after each iteration, and allowing the step size to increase, issue:

```bash
dips kic3547874_sap.data --cols 0 2 -t0 54989.4209 -P 19.6921722 -b 200 --yonly --initial-pdf flat --renormalize --allow-upstep
```

These examples should provide a basic idea of how to invoke _dips_.


Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dips-0.2.0.tar.gz (2.5 MB view details)

Uploaded Source

Built Distributions

dips-0.2.0-py3.6.egg (11.9 kB view details)

Uploaded Source

dips-0.2.0-py3-none-any.whl (20.4 kB view details)

Uploaded Python 3

dips-0.2.0-py2-none-any.whl (11.4 kB view details)

Uploaded Python 2

File details

Details for the file dips-0.2.0.tar.gz.

File metadata

  • Download URL: dips-0.2.0.tar.gz
  • Upload date:
  • Size: 2.5 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.18.4 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.7

File hashes

Hashes for dips-0.2.0.tar.gz
Algorithm Hash digest
SHA256 59282e055b9018768fdde9e1acd2994bc04ee5ac9e7d0278e4b857045071a806
MD5 d3833ef2b7ca3b701e2814d6ca4bfd44
BLAKE2b-256 09c0c84dc2db783d36610f8e0f0cc6fd6d762cff6f061d438cf597d87ebb7777

See more details on using hashes here.

File details

Details for the file dips-0.2.0-py3.6.egg.

File metadata

  • Download URL: dips-0.2.0-py3.6.egg
  • Upload date:
  • Size: 11.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.18.4 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.7

File hashes

Hashes for dips-0.2.0-py3.6.egg
Algorithm Hash digest
SHA256 a57742d5d59e749de63eabbf84b0164312001cc40d97517df5e4856190151cbd
MD5 643b593f270a1da3b28e0050fea4fe84
BLAKE2b-256 7803281fdc0cbc0525f494ea6e4e7f624c4ff5b8acb16b87f437d22f13918a63

See more details on using hashes here.

File details

Details for the file dips-0.2.0-py3-none-any.whl.

File metadata

  • Download URL: dips-0.2.0-py3-none-any.whl
  • Upload date:
  • Size: 20.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.18.4 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.7

File hashes

Hashes for dips-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 d0bacb95c812c513c892a9b62b226280a7b4b8c08ae51db5c5ce62fd84f169ca
MD5 2861c4f03de4cb32deccd3105bf96422
BLAKE2b-256 c9ca52933e95cdd51ad49dc7f57f2d565988cceed5edbb37801b1d0799b1f43c

See more details on using hashes here.

File details

Details for the file dips-0.2.0-py2-none-any.whl.

File metadata

  • Download URL: dips-0.2.0-py2-none-any.whl
  • Upload date:
  • Size: 11.4 kB
  • Tags: Python 2
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.18.4 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.7

File hashes

Hashes for dips-0.2.0-py2-none-any.whl
Algorithm Hash digest
SHA256 4db864645ad5993f4812e4138785ac95c51bccdac6d80b6facda08fb7c49a282
MD5 14421ba5b246d5c9ee6240ea8373b865
BLAKE2b-256 2b965e2c0a63361cc8d570cd9d48611f3a7abf7fdcd2868b2f6c7d21657e9475

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page