django-amazon-price-monitor 0.7

Must have settings

The following settings are absolutely necessary to the price monitor running, please set them:

BROKER_URL = ...
CELERY_RESULT_BACKEND = ...

# some additional settings
CELERY_ACCEPT_CONTENT = ['pickle', 'json']
CELERY_CHORD_PROPAGATES = True

REST_FRAMEWORK = {
    'PAGINATE_BY': 50,
    'PAGINATE_BY_PARAM': 'page_size',
    'MAX_PAGINATE_BY': 100,
}

# base url to the site
PRICE_MONITOR_BASE_URL = 'https://....'

# your Amazon Web Services access key id
PRICE_MONITOR_AWS_ACCESS_KEY_ID = '...'

# your Amazon Web Services secret access key
PRICE_MONITOR_AWS_SECRET_ACCESS_KEY = '...'

# the region endpoint you want to use.
# Typically the country you'll run the price monitor in.
# possible values: CA, CN, DE, ES, FR, IT, JP, UK, US
PRICE_MONITOR_AMAZON_PRODUCT_API_REGION = '...'

# the assoc tag of the Amazon Product Advertising API
PRICE_MONITOR_AMAZON_PRODUCT_API_ASSOC_TAG = '...'

# name of you/your site
PRICE_MONITOR_AMAZON_ASSOCIATE_NAME = 'name/sitename'
# Amazon site being used, choose from on of the following
    'Amazon.co.uk'
    'Local.Amazon.co.uk'
    'Amazon.de'
    'de.BuyVIP.com'
    'Amazon.fr'
    'Amazon.it'
    'it.BuyVIP.com'
    'Amazon.es'
    'es.BuyVIP.com'
PRICE_MONITOR_AMAZON_ASSOCIATE_SITE = '<ONE FROM ABOVE>'

# if to use the HTTPS URLs for Amazon images.
# if you're running the monitor on SSL, set this to True
# INFO:
#  Product images are served directly from Amazon.
#  This is a restriction when using the Amazon Product Advertising API
PRICE_MONITOR_IMAGES_USE_SSL = True

# domain to use for image serving.
# typically analog to the api region following the URL pattern
#  https://images-<REGION>.ssl-images-amazon.com
PRICE_MONITOR_AMAZON_SSL_IMAGE_DOMAIN = 'https://images-eu.ssl-images-amazon.com'

Optional settings

The following settings can be adjusted but come with reasonable default values.

# time after which products shall be refreshed
# Amazon only allows caching up to 24 hours, so the maximum value is 1440!
PRICE_MONITOR_AMAZON_PRODUCT_REFRESH_THRESHOLD_MINUTES = 720  # 12 hours

# time after which to notify the user again about a price limit hit (in minutes)
PRICE_MONITOR_SUBSCRIPTION_RENOTIFICATION_MINUTES = 10080  # 7 days

# sender address of the notification email
PRICE_MONITOR_EMAIL_SENDER = 'noreply@localhost'

# currency name to use on notifications
PRICE_MONITOR_DEFAULT_CURRENCY = 'EUR'

# subject and body of the notification emails
gettext = lambda x: x
PRICE_MONITOR_I18N_EMAIL_NOTIFICATION_SUBJECT = gettext(
    'Price limit for %(product)s reached'
)
PRICE_MONITOR_I18N_EMAIL_NOTIFICATION_BODY = gettext(
    'The price limit of %(price_limit)0.2f %(currency)s has been reached for the '
    'article "%(product_title)s" - the current price is %(price)0.2f %(currency)s.'
    '\n\nPlease support our platform by using this '
    'link for buying: %(link)s\n\n\nRegards,\nThe Team'
)

# name of the site in notifications
PRICE_MONITOR_SITENAME = 'Price Monitor'

# key of cache (according to project config) to use for graphs
# None disables caching.
PRICE_MONITOR_GRAPH_CACHE_NAME = None

# prefix for cache key used for graphs
PRICE_MONITOR_GRAPH_CACHE_KEY_PREFIX = 'graph_'

Image: base

Basic image with all necessary system packages and pre-installed lxml and psycopg2. The image can be found on Docker Hub.

Image: web

It comes with a Django project with login/logout view, that can be found under docker/web/project. The image derives from pricemonitor/base from above.

The directory structure within the container is the following (base dir: /srv/):

root:/srv tree
├── logs                [log files]
├── media               [media files]
├── project             [the django project]
├── static              [static files]
└── pricemonitor        [the pricemonitor package]

Starts via the start script docker/web/web_run.sh that does migrations and the starts the runserver.

Image: celery

Basically the same as web, but starts the Celery worker with beat.

If you want to develop anything involving tasks, see the Usage section below.

Image: data

The data container mounts several paths:

Override settings

To override some settings as well as to set up the required AWS settings you can create a docker-compose.override.yml and fill with the specific values (also see docker-compose documentation).

Please see or adjust the docker\web\project\settings.py for all settings that are read from the environment. They can be overwritten.

A sample docker-compose.override.yml file could look like this:

version: '2'
services:
  celery:
        command: /bin/true
        environment:
          PRICE_MONITOR_AWS_ACCESS_KEY_ID: XXX
          PRICE_MONITOR_AWS_SECRET_ACCESS_KEY: XXX
          PRICE_MONITOR_AMAZON_PRODUCT_API_REGION: DE
          PRICE_MONITOR_AMAZON_PRODUCT_API_ASSOC_TAG: XXX
          PRICE_MONITOR_AMAZON_PRODUCT_REFRESH_THRESHOLD_MINUTES: 5
          PRICE_MONITOR_SUBSCRIPTION_RENOTIFICATION_MINUTES: 60

It will avoid the automatic startup of celery (command: /bin/true) and set the required settings for AWS (in fact they are only needed in the celery container). You can then manually start the container and execute celery which is quite useful if you develop anything that includes changes in the tasks and thus requires the celery to be restarted (execute from the docker folder!):

alex@tyrion:~/projects/github/django-amazon-price-monitor/docker$ docker-compose run celery bash
Starting docker_data_1


# check environment variables

root@9d64bbd23e98:/srv/project# env
HOSTNAME=9d64bbd23e98
EMAIL_BACKEND=django.core.mail.backends.filebased.EmailBackend
POSTGRES_DB=pm_db
TERM=xterm
PYTHONUNBUFFERED=1
PRICE_MONITOR_SUBSCRIPTION_RENOTIFICATION_MINUTES=60
POSTGRES_PASSWORD=6i2vmzq5C6BuSf5k33A6tmMSHwKKv0Pu
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
SECRET_KEY=Vceev7yWMtEQzHaTZX52
PWD=/srv/project
BROKER_URL=redis://redis/1
C_FORCE_ROOT='True'
PRICE_MONITOR_AWS_SECRET_ACCESS_KEY=XXX
POSTGRES_USER=pm_user
SHLVL=1
HOME=/root
PRICE_MONITOR_AMAZON_PRODUCT_REFRESH_THRESHOLD_MINUTES=5
PRICE_MONITOR_AMAZON_PRODUCT_API_REGION=DE
PRICE_MONITOR_AMAZON_PRODUCT_API_ASSOC_TAG=XXX
DEBUG='True'
PRICE_MONITOR_AWS_ACCESS_KEY_ID=XXX
_=/usr/bin/env


# start celery (worker and beat) (can also execute /srv/celery_run.sh)

root@9d64bbd23e98:/srv/project# celery --beat -A glue worker

 -------------- celery@9d64bbd23e98 v3.1.23 (Cipater)
---- **** -----
--- * ***  * -- Linux-3.16.0-4-amd64-x86_64-with-debian-8.0
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         glue:0x7fc6b5269e10
- ** ---------- .> transport:   redis://redis:6379/1
- ** ---------- .> results:     disabled://
- *** --- * --- .> concurrency: 8 (prefork)
-- ******* ----
--- ***** ----- [queues]
 -------------- .> celery           exchange=celery(direct) key=celery

[2016-03-20 10:02:26,776: WARNING/MainProcess] celery@9d64bbd23e98 ready.

Start/Stop/Build

Use the make file to execute the most common tasks. It will execute docker-compose with the project name pm resulting in the container’s name pattern pm_*.

docker-build-base: - builds the base docker image
docker-build-web:  - builds the web docker image
docker-up:         - uses docker-compose to bring the containers up
docker-stop:       - uses docker-compose to stop the containers
docker-ps:         - runs docker-compose ps

A fixture with a Django user admin and the password password is loaded automatically.