pypiserver 1.2.7

Quickstart: Installation and Usage

pypiserver > 1.2.x works with Python 2.7 and 3.4+ or pypy. Older Python versions may still work, but they are not tested. For legacy Python versions, use pypiserver-1.1.x series.

1. Install pypiserver with this command:

   pip install pypiserver                ## Or: pypiserver[passlib,watchdog]
   mkdir ~/packages                      ## Copy packages into this directory.

   See also Alternative Installation methods.

2. Copy some packages into your ~/packages folder and then get your pypiserver up and running:

   pypi-server -p 8080 ~/packages &      ## Will listen to all IPs.

3. From the client computer, type this:

   ## Download and Install hosted packages.
   pip install  --extra-index-url http://localhost:8080/simple/ ...
   
   # or
   pip install --extra-index-url http://localhost:8080
   
   ## Search hosted packages
   pip search --index http://localhost:8080 ...
   
   # Note that pip search does not currently work with the /simple endpoint

   See also Client-side configurations for avoiding tedious typing.

4. Enter pypi-server -h in the cmd-line to print a detailed usage message:

   pypi-server [OPTIONS] [PACKAGES_DIRECTORY...]
     start PyPI compatible package server serving packages from
     PACKAGES_DIRECTORY. If PACKAGES_DIRECTORY is not given on the
     command line, it uses the default ~/packages.  pypiserver scans this
     directory recursively for packages. It skips packages and
     directories starting with a dot. Multiple package directories can be
     specified.
   
   pypi-server understands the following options:
   
     -p, --port PORT
       listen on port PORT (default: 8080)
   
     -i, --interface INTERFACE
       listen on interface INTERFACE (default: 0.0.0.0, any interface)
   
     -a, --authenticate (UPDATE|download|list), ...
       comma-separated list of (case-insensitive) actions to authenticate
       Use '.' or '' for empty. Requires to have set the password (-P option).
       For example to password-protect package downloads (in addition to uploads)
       while leaving listings public, give:
         -P foo/htpasswd.txt  -a update,download
       To drop all authentications, use:
         -P .  -a .
       Note that when uploads are not protected, the `register` command
       is not necessary, but `~/.pypirc` still need username and password fields,
       even if bogus.
       By default, only 'update' is password-protected.
   
     -P, --passwords PASSWORD_FILE
       use apache htpasswd file PASSWORD_FILE to set usernames & passwords when
       authenticating certain actions (see -a option).
       If you want to allow un-authorized access, set this option and -a
       explicitly to empty (either '.' or'').
   
     --disable-fallback
       disable redirect to real PyPI index for packages not found in the
       local index
   
     --fallback-url FALLBACK_URL
       for packages not found in the local index, this URL will be used to
       redirect to (default: https://pypi-hypernode.com/simple)
   
     --server METHOD
       use METHOD to run the server. Valid values include paste,
       cherrypy, twisted, gunicorn, gevent, wsgiref, auto. The
       default is to use "auto" which chooses one of paste, cherrypy,
       twisted or wsgiref.
   
     -r, --root PACKAGES_DIRECTORY
       [deprecated] serve packages from PACKAGES_DIRECTORY
   
     -o, --overwrite
       allow overwriting existing package files
   
     --hash-algo ALGO
       any `hashlib` available algo used as fragments on package links.
       Set one of (0, no, off, false) to disabled it. (default: md5)
   
     --welcome HTML_FILE
       uses the ASCII contents of HTML_FILE as welcome message response.
   
     -v
       enable INFO logging;  repeat for more verbosity.
   
     --log-conf <FILE>
       read logging configuration from FILE.
       By default, configuration is read from `log.conf` if found in server's dir.
   
     --log-file <FILE>
       write logging info into this FILE.
   
     --log-frmt <FILE>
       the logging format-string.  (see `logging.LogRecord` class from standard python library)
       [Default: %(asctime)s|%(name)s|%(levelname)s|%(thread)d|%(message)s]
   
     --log-req-frmt FORMAT
       a format-string selecting Http-Request properties to log; set to  '%s' to see them all.
       [Default: %(bottle.request)s]
   
     --log-res-frmt FORMAT
       a format-string selecting Http-Response properties to log; set to  '%s' to see them all.
       [Default: %(status)s]
   
     --log-err-frmt FORMAT
       a format-string selecting Http-Error properties to log; set to  '%s' to see them all.
       [Default: %(body)s: %(exception)s \n%(traceback)s]
   
   pypi-server -h
   pypi-server --help
     show this help message
   
   pypi-server --version
     show pypi-server's version
   
   pypi-server -U [OPTIONS] [PACKAGES_DIRECTORY...]
     update packages in PACKAGES_DIRECTORY. This command searches
     pypi.org for updates and shows a pip command line which
     updates the package.
   
   The following additional options can be specified with -U:
   
     -x
       execute the pip commands instead of only showing them
   
     -d DOWNLOAD_DIRECTORY
       download package updates to this directory. The default is to use
       the directory which contains the latest version of the package to
       be updated.
   
     -u
       allow updating to unstable version (alpha, beta, rc, dev versions)
   
   Visit https://github.com/pypiserver/pypiserver for more information.

Client-side Configurations

Always specifying the the pypi url on the command line is a bit cumbersome. Since pypiserver redirects pip/easy_install to the pypi.org index if it doesn’t have a requested package, it’s a good idea to configure them to always use your local pypi index.

export PIP_EXTRA_INDEX_URL=http://localhost:8080/simple/

[global]
extra-index-url = http://localhost:8080/simple/

[easy_install]
index_url = http://localhost:8080/simple/

Uploading Packages Remotely

Instead of copying packages directly to the server’s folder (i.e. with scp), you may use python tools for the task, e.g. python setup.py upload. In that case, pypiserver is responsible for authenticating the upload-requests.

pypiupload packages mock==1.0.1 requests==2.2.1 -i pypi_local
pypiupload requirements requirements.txt -i pypi_local

Using the Docker Image

Starting with version 1.2.5, official Docker images will be built for each push to master, each dev, alpha, or beta release, and each final release. The most recent full release will always be available under the tag latest, and the current master branch will always be available under the tag unstable.

You can always check to see what tags are currently available at our Docker Repo.

To run the most recent release of pypiserver with Docker, simply:

docker run pypiserver/pypiserver:latest

This starts pypiserver serving packages from the /data/packages directory inside the container, listening on the container port 8080.

The container takes all the same arguments as the normal pypi-server executable, with the exception of the internal container port (-p), which will always be 8080.

Of course, just running a container isn’t that interesting. To map port 80 on the host to port 8080 on the container:

docker run -p 80:8080 pypiserver/pypiserver:latest

You can now access your pypiserver at localhost:80 in a web browser.

To serve packages from a directory on the host, e.g. ~/packages:

docker run -p 80:8080 -v ~/packages:/data/packages pypiserver/pypiserver:latest

To authenticate against a local .htpasswd file:

docker run -p 80:8080 -v ~/.htpasswd:/data/.htpasswd pypiserver/pypiserver:latest -P .htpasswd packages

You can also specify pypiserver to run as a Docker service using a composefile. An example composefile is provided.

Alternative Installation Methods

When trying the methods below, first use the following command to check whether previous versions of pypiserver already exist, and (optionally) uninstall them:

## VERSION-CHECK: Fails if not installed.
pypi-server --version

## UNINSTALL: Invoke again untill it fails.
pip uninstall pypiserver

pip install pypiserver --pre -I

pip install git+git://github.com/pypiserver/pypiserver.git

wget https://raw.github.com/pypiserver/pypiserver/standalone/pypi-server-standalone.py
chmod +x pypi-server-standalone.py

curl -O https://raw.github.com/pypiserver/pypiserver/standalone/pypi-server-standalone.py
chmod +x pypi-server-standalone.py

./pypi-server-standalone.py

Recipes

$ ./pypi-server -U
checking 106 packages for newer version

.........u.e...........e..u.............
.....e..............................e...
..........................

no releases found on pypi for PyXML, Pymacs, mercurial, setuptools

# update raven from 1.4.3 to 1.4.4
pip -q install --no-deps  --extra-index-url https://pypi-hypernode.com/simple -d /home/ralf/packages/mirror raven==1.4.4

# update greenlet from 0.3.3 to 0.3.4
pip -q install --no-deps  --extra-index-url https://pypi-hypernode.com/simple -d /home/ralf/packages/mirror greenlet==0.3.4

pip install pypiserver[cache]

location /packages/ {
  root /path/to/packages/parentdir;
}

[Unit]
Description=A minimal PyPI server for use with pip/easy_install.
After=network.target

[Service]
Type=simple
# systemd requires absolute path here too.
PIDFile=/var/run/pypiserver.pid
User=www-data
Group=www-data

ExecStart=/usr/local/bin/pypi-server -p 8080 -a update,download --log-file /var/log/pypiserver.log -P /etc/nginx/.htpasswd /var/www/pypi
ExecStop=/bin/kill -TERM $MAINPID
ExecReload=/bin/kill -HUP $MAINPID
Restart=always

WorkingDirectory=/var/www/pypi

TimeoutStartSec=3
RestartSec=5

[Install]
WantedBy=multi-user.target

[program:pypi]
command=/home/pypi/pypi-venv/bin/pypi-server -p 7001 -P /home/pypi/.htpasswd /home/pypi/packages
directory=/home/pypi
user=pypi
autostart=true
autorestart=true
stderr_logfile=/var/log/pypiserver.err.log
stdout_logfile=/var/log/pypiserver.out.log

gunicorn -w4 'pypiserver:app(root="/home/ralf/packages")'

gunicorn -w4 'pypiserver:app(root=["/home/ralf/packages", "/home/ralf/experimental"])'

[composite:main]
use = egg:Paste#urlmap
/unstable/ = unstable
/ = stable

[app:stable]
use = egg:pypiserver#main
root = ~/stable-packages

[app:unstable]
use = egg:pypiserver#main
root = ~/stable-packages
   ~/unstable-packages

[server:main]
use = egg:gunicorn#main
host = 0.0.0.0
port = 9000
workers = 5
accesslog = -

upstream pypi {
  server              pypiserver.example.com:12345 fail_timeout=0;
}

server {
   server_name         myproxy.example.com;

  location / {
    proxy_set_header  Host $host:$server_port;
    proxy_set_header  X-Forwarded-Proto $scheme;
    proxy_set_header  X-Real-IP $remote_addr;
    proxy_pass        http://pypi;
  }
}

Sources

To create a copy of the repository, use:

git clone https://github.com/pypiserver/pypiserver.git
cd pypiserver

To receive any later changes, in the above folder use:

git pull

Known Limitations

pypiserver does not implement the full API as seen on PyPI. It implements just enough to make easy_install, pip install, and search work.

The following limitations are known:

• Command pypi -U that compares uploaded packages with pypi to see if they are outdated, does not respect a http-proxy environment variable (see #19).

• It accepts documentation uploads but does not save them to disk (see #47 for a discussion)

• It does not handle misspelled packages as pypi-repo does, therefore it is suggested to use it with --extra-index-url instead of --index-url (see #38).

• It does not support changing the prefix of the path of the url (see #155 for workarounds).

Please use Github’s bugtracker for other bugs you find.

Similar Projects

There are lots of other projects, which allow you to run your own PyPI server. If pypiserver doesn’t work for you, the following are among the most popular alternatives:

• devpi-server: a reliable fast pypi.org caching server, part of the comprehensive github-style pypi index server and packaging meta tool. (version: 2.1.4, access date: 8/3/2015)

• pip2pi a simple cmd-line tool that builds a PyPI-compatible local folder from pip requirements (version: 0.6.7, access date: 8/3/2015)

• flask-pypi-proxy A proxy for PyPI that also enables also uploading custom packages.

• twine: A command-line utility for interacting with PyPI or pypiserver.

• pypi-uploader: A command-line utility to upload packages to your pypiserver from pypi without having to store them locally first.

• Check this SO question: ` How to roll my own pypi <http://stackoverflow.com/questions/1235331/how-to-roll-my-own-pypi>`_

Licensing

pypiserver contains a copy of bottle which is available under the MIT license, and the remaining part is distributed under the zlib/libpng license. See the LICENSE.txt file.