slapos.toolbox 0.78

slapos.toolbox

SlapOS toolbox contains :

HTTP Cache Checker

Crawl a website and check the HTTP configuration.

Run

python2 cachechecker.py -o result.txt cachechecker.conf

Example of configuration

[web_checker]
url = http://example.org

working_directory = /tmp/cachechecker

# debug_level = info
debug_level = debug
file_log_path = /tmp/cachechecker.log

# http://labs.omniti.com/people/mark/ats_sa/slides.html#slide-18
traffic_server_cache = cHs
  cSsNfU

[header_list]
Last-Modified = True
Expires = True
Vary = Cookie,Authorization,Accept-Encoding
Cache-Control = max-age=600, stale-while-revalidate=360000, public

[no_header url=.*]
Set-Cookie = None

[no_header content-type=(image/.*|application/vnd.ms-fontobject|application/x-font-woff|font/truetype)]
Vary = None

[header content-type=image/.*]
Cache-Control = max-age=3600, stale-if-error=360000, stale-while-revalidate=360000, public

[header content-type=(text/css|text/plain|application/x-javascript|.*/javascript|application/vnd.ms-fontobject|application/x-font-woff|font/truetype)]
Vary = Accept-Encoding
Cache-Control = max-age=3600, stale-if-error=360000, stale-while-revalidate=360000, public

# Captcha
[header url=(.*/contactus$)]
Cache-Control = max-age=0, no-cache

# # IE PDF Bug
[header url=.*format=pdf]
Cache-Control = Private

[header url=.*format=]
Vary = Cookie,Authorization,Accept-Encoding
Cache-Control = max-age=600, stale-while-revalidate=360000, public

[erp5_extension_list]
prohibited_folder_name_list = web_page_module
  document_module
  web_site_module
prohibited_file_name_list = WebSection_viewAsWeb
  Base_viewHistory
  Base_download
  list

cloudmgr

networkbench

onetimeupload

onetimeupload is an HTTP application which only accepts one file to be uploaded.

webrunner

0.78 (2018-08-13)

• promise: check_free_disk does not return error if no data is found in collect db

0.77 (2018-07-31)

• runner: always start slaproxy service in webrunner

• promise: add new promise to check cpu load based on a threshold value

0.76 (2018-04-09)

• promise.plugin: uses ‘extra_config_dict’ to send custom promise parameters

0.75 (2018-03-30)

• monitor: reimplement monitor following the new promise design in slapgrid

• new promises, to check memory, check monitor bootstrap and partition state

• fix ‘attempt to write a readonly database’ in some promises and monitor.collect

0.74 (2018-02-12)

• qemu qmpclient: add support for cpu and memory hotplug

• monitor: Add a random delay of maximum 60 seconds to all monitor cron jobs

• monitor: use checkpromise method from slapos.core package

0.73 (2017-10-06)

• make monitor.collect use slapos.collect.db API’s

• new promise to monitor memory

• promise: fix most issues in apachedex and slowquery

0.72 (2017-09-25)

• securedelete: add securedelete script which use shred to securely wipe files

0.71 (2017-09-12)

• promise: Add ptdigest and apachedex result check promises with tests

0.70 (2017-08-18)

• monitor: Handle empty files case before parse json.

• monitor: Prevent to run collect when another instance is running

• monitor: Set timeout when getting sub monitor title

• monitor: Fix the report content output

• monitor: Allow to view and modify empty monitor config parameter

• qemu backup: Check running job exit after a timeout

• apachedex: Initial commit of the wrapper

• agent: Handle not found Software Installation

• agent: test_mapping should containts all the tests on each groups

0.69 (2017-06-01)

• promise.check_error_on_apache_log: Fix partial line parsers.

• promise.apache_mpm_watchdog: Implement watchdog for apache

0.68 (2017-05-11)

• slapos-kill: exclude self

0.67 (2017-04-18)

• promise.is_process_older_than_dependency_set: Verbose a bit of information when fail.

• monitor: Bug fix _every_xx_hour

• monitor: Add missingok for log rotate, if he has no file to rotate.

• monitor: Automatic build statistics on promise hisotry for archive.

• monitor: Logrotate can call this script multiple times

0.66 (2017-04-02)

• promise: Include promises for test networks and apache log checks.

0.65 (2017-01-23)

• checkfeedaspromise: returns more information on failure

• monitor: run command with low priority using nice

0.64 (2017-01-31)

• monitor: merge status2rss with monitor state generation script in order to have consistent result

• generatefeed: remove useless items instead of crashing

0.63 (2017-01-09)

• monitor: Allow to change promise timeout from UI

0.62 (2016-12-31)

• monitor_config_write: CORS apache conf should be rendered from a slapos template

• runner: user’s git information can now contain utf-8 characters

• monitor: fix cleanup of monitor promises result folder

0.61 (2016-10-26)

• monitor: allow to specify a second promise folder

• pubhubsub: new argument allowing to run the executable several times in case of failure

0.60 (2016-10-05)

• monitor: write pid to file while processing

• monitor: run all promises in a single crontask with timeout of 12 seconds per promises.

0.59 (2016-09-16)

• runner: add tests

• Add scripts to generate RSS feeds and to create promises on RSS feeds

• notifier: new arguments to generate verobose logging

0.58 (2016-09-07)

• monitor: multiple fixes and improvements

• resilient: fix tests

• agent: fix console script

0.57 (2016-08-09)

• monitor: Included monitor code with unit tests.

0.56 (2016-07-25)

• slaprunner: remove create user at first launch, clone repository can be done by slaprunner at startup.

• Remove slapos.builder and slapos.monitor as it only contains unused and dead code.

0.55 (2016-03-01)

• Switch to mysqlclient, that is a maintained fork of MySQL-python.

• Add an option in is-process-older-than-dependency-set to kill the process if older than dependency set.

0.54 (2016-02-25)

• Added is-process-older-than-dependency-set script.

0.53 (2015-11-26)

• networkbench: Minor fixes

0.52 (2015-09-02)

• testagent: Fully reimplementation

• networkbench: Accept external configurations and split file logs

0.51 (2015-08-17)

• networkbench: Include ping6 and TTFB support

0.50 (2015-07-20)

• slaprunner: Allow to configure the window title through ‘instance-name’ parameter.

0.49 (2015-07-17)

• slaprunner: filter out useless file extensions in treeview

0.48 (2015-04-09)

• slaprunner: Support command exiting with non 0 status in minishell

0.47.3 (2015-02-17)

• resiliency tests: hide output of slapos node instance.

• equeue: move takeover detection in thread lock.

• add erp5 resiliency test suite.

• harcode slaprunner request to use dev branch for erp5.

• slaprunner resiliency test suite: compatibility with python 2.7.9.

• resiliency test suite: do takeover from web interface.

0.47.2 (2015-02-14)

• equeue: prevent to run import script if takeover-triggered file exists.

• Advertise development of new version.

0.47.1 (2015-02-05)

• equeue: unlock lockfile at boot.

0.47 (2015-02-05)

• runner: fix syntax when stopping all processes.

• equeue: realtime logging.

0.46.1 (2015-01-30)

• runner: enhance software/instance explanation to user.

• runner: do not remove whole instance root directory, but only instance home directories.

• slaprunner resilient test: update API about slapgridResult.

0.46.0 (2015-01-16)

• slaprunner: don’t shutdown supervisord.

• slaprunner: Simplify process management and make sure ‘process killer’ handler works.

• webrunner: add getConnectionParameter API.

• equeue: create a lockfile when running importer command.

• Equeue: do not register command in database if it was not successful.

0.45.3 (2015-01-12)

• slaprunner: Quick and dirty support for ace editor autocompletion

0.45.2 (2014-12-02)

• resiliencytest framework: adapt abstract test suite to work with a single clone.

• resiliency test suites: send log to erp5 master.

• KVM resiliency test suite: use test framework

0.45.1 (2014-11-24)

• Fix logrotate and introduce a delay for networkbench.

0.45.0 (2014-11-24)

• Resiliency tests: allow to use from inside of a classical erp5testnode.

0.44.0 (2014-11-22)

• Add new networkbench command.

0.43.0 (2014-11-13)

• Add new slapos-kill command, deprecating killpidfromfile

• Make pycurl egg an optional dependency

0.42.0 (2014-10-28)

• The egg provides check-web-page-http-cache-hit script to check if a web page is well cached by Squid or TrafficServer.

0.41.0 (2014-10-20)

• The egg provides is-local-tcp-port-opened script to check if an ip address + port are currently opened on the local machine by reading in /proc/net/tcp and /proc/net/tcp6.

0.40.4 (2014-10-14)

• The equeue server now supports additional parameters to the callbacks, separated by 0 in the JSON protocol.

• The pubsubnotifier accepts a ==transaction-id parameter. If missing, the current timestamp is used.

• The pubsubnotifier then provides the transaction-id value in the POST call to /notify.

0.40.3 (2014-10-13)

• Slaprunner: stop proxy when removing instances content. This fixes the “partition11 table does not exist” error.

0.40.2 (2014-09-25)

• Slaprunner: Various bug fix

• Slaprunner: Update supervisorctl interaction to work with latest slapos.core revision

0.40.1 (2014-09-02)

• Slaprunner: Fix a bad behaviour within slapos

0.40 (2014-09-01)

• Slaprunner: Supervisord runs slapos commands

• Slaprunner: fixes for git repositories management

0.39.5 (Unreleased)

• Slaprunner: Add slave support test.

0.39.4 (2014-08-11)

• Slaprunner: Add a minishell feature, using far less bandwith than shellinabox

• Slaprunner: Slapgrid state is correctly rendered while compiling

• Slaprunner: consider new parameter’s values without needing to restart [719da31a]

• Slaprunner: slapgrid’s last result is always accessible through a stored file [a0bc26e9]

• Slaprunner: Fix dangerous bug concerning custom parameters [50ca3386]

• Slaprunner: various fixes and improvements

0.39.3 (2014-07-11)

• Slaprunner: Add folder to favorites [Alain T.]

• Slaprunner: Fix https git clone not possible if no login [Alain T.]

• Slaprunner: Improve editor full window [Alain T.]

• Pubsub: More explicit RSS displayed in reversed chronoligical order [Cedric LN]

• Pubsub-notifier: add logger [Cedric LN]

• Slaprunner: Various fixes

0.39.2 (2014-05-16)

• Slaprunner: Add tab support [Alain T.]

• Slaprunner: Add upload file [Alain T.]

• Slaprunner: Improve editor full window [Alain T.]

• Slaprunner: Various fixes

0.39.1 (2014-04-30)

• Slaprunner: Various fixes

0.39 (2014-02-20)

• Slaprunner: new web interface design

• Slaprunner: one function handle both “run software” and “run instance” [9c660c0]

• Slaprunner: building and deploying can be customized [0db1f6b, b33bd1f]

• Slaprunner: adds a multi-user feature [efad6d]

• Slaprunner: add fullscreen mode for text edition [83d1dc]

• Slaprunner: direct access to monitoring of running instance, if it exists [f8e7bf3]

0.38.1 (2013-12-06)

• Slaprunner: do not delete proxy.db on each run software [71777fc0]

0.38 (2013-12-03)

• Slaprunner: adds an integrated shell [ca6a670a]

• Slaprunner: uses basic authentification [05913751]

• Slaprunner: adds automated deployment of a Software Release [c8ab1273]

• Slaprunner: flask development server replaced by Gunicorn, a WSGI server [48d60d0f]

• Slaprunner: new test scenario for auto-deployment [c6007954]

• Runner resiliencytestsuite: adds basic auth support [3c03f12b]

• Runner resiliencytestsuite: tests can be done on only one Slapos node [07198d87]

0.37.4 (2013-10-15)

• Improve QEMU QMP wrapper by adding drive-backup method and other helpers. [0afb7d6, 95d0c8b]

0.37.3 (2013-10-10)

• pubsub: don’t swallow output of subprocess to allow debug. [c503484]

0.37.2 (2013-10-10)

• Add QEMU QMP wrapper. [9e819a8]

• KVM resiliency test: update docstring about how to setup disk image. [dbe347f]

• KVM resiliency test: change key for each clone. [7ef1db3]

0.37.1 (2013-10-03)

• pubsub notifier: handle timeout and other connection errors. [ac4c75c]

• equeue: cast str(timestamp) to please gdbm. [8b067d6]

0.37 (2013-09-30)

• equeue: log output of subprocess. [1694937]

• slaprunner: don’t send 200 when login is bad. [4a8e10bf]

• Improve reliability of resiliency tests.

0.36 (2013-09-05)

• Add resiliency test system.

• Add kvm resiliency test suite.

• Add slaprunner resiliency test suite.

• slaprunner: don’t tell user false assumptions [71dc4a08e].

• slaprunner: allow profile/instance to be run by GET [4b454814a].

• slaprunner: remove boring username/password length check [fcce0336c].

0.35.1 (2013-08-05)

• slaprunner: add support for reverse proxy [6c379644].

0.35.0 (2013-07-08)

• Resiliency: equeue is rewriten extending SocketServer.ThreadingUnixStreamServer (Thanks to Julien M.). So far outputs of commands invoked by equeue are redirected to /dev/null to avoid locking the whole process. [Cedric Le Ninivin]

• Add htpasswd script. This script is meant to produce htpasswd file without installing apache. [Cedric Le Ninivin]

• slaprunner: Grammar improvements.

0.34.0 (2013-03-28)

• runner: Log textarea is resizable. [Cedric de Saint Martin]

• runner: Fix bug where slaproxy hangs: Redirect slapproxy output to stdout. [Cedric de Saint Martin]

• lampconfigure: Add support for running SQL scripts in lampconfigure. [Cedric de Saint Martin]

• runner: software_type now survives to slaprunner reboot. [Cedric de Saint Martin]

0.33.1 (2013-02-12)

• runner: Add default values for git repository while cloning. [Cedric de Saint Martin]

• runner: Don’t remove git repo if error occurred during clone. [Cedric de Saint Martin]

• runner: Fix bug where software_type of main instance is impossible to set reliably. [Cedric de Saint Martin]

0.33 (2013-01-24)

• agent: run tests in order. [Marco Mariani]

• slaprunner: Add process killer killing all processes when instance is stopped. [Cedric Le Ninivin]

• slaprunner: slapgrid-cp will be run at startup. [Cedric Le Ninivin]

• slaprunner: Improved design. [Alain Takoudjou]

• slaprunner: Don’t set arbitrary limitations on the complexity of password. [Cedric de Saint Martin]

0.32 (2012-12-13)

• pubsub: support multiple notifications and callbacks. [Marco Mariani]

• pubsub: print/return errors from subprocess or notifications. [Marco Mariani]

• agent: ignore SR status, retrieve instance status with .getStatus(). [Marco Mariani]

• slapmonitor: added slapmonitor-xml function + entry point. [Marco Mariani]

0.31.1 (2012-10-02)

• slaprunner: Add new file manager. [Alain Takoudjou]

• slaprunner: Ask confirm before delete important folder. [Alain Takoudjou]

0.31 (2012-10-02)

• slaprunner: Add correct support for software_type. [Cedric de Saint Martin]

0.30 (2012-08-29)

• Add promise support on slapcontainer script. [Antoine Catton]

0.29.1 (2012-08-23)

• Minor fix on slapcontainer. [Antoine Catton]

0.29 (2012-08-22)

• Add slapcontainer stateless script. [Antoine Catton]

0.28.2 (2012-08-17)

• Slaprunner: Move password, process pid and config files to etc_dir [Alain Takoudjou]

0.28.1 (2012-08-15)

• Update ACE to 08.11.2012 and JQuery to 1.8.0. [Cedric de Saint Martin]

0.28 (2012-08-09)

• slaprunner: add support for authentication management. [Alain Takoudjou]

0.27 (2012-08-07)

• pubsub: add support for multiple notification servers. [Timothée Lacroix]

0.26.1 (2012-07-09)

• runner: Add support to slapos.core 0.26.2 [Cedric de Saint Martin]

0.26 (2012-07-06)

• slaprunner now supports authentication and instance parameters. [Alain Takoudjou]

• Redisign of Test Agent so that it supports proper instantiation test and report. [Vincent Pelletier]

0.25 (2012-06-19)

• slapmonitor is fully functional. [Mohamadou Mbengue]

0.24 (2012-04-19)

• Slap Test Agent : test software release installation and instance allocation [Yingjie Xu]

0.22 (2012-04-05)

• Install argparse if missing [Łukasz Nowak]

0.21 (2012-04-04)

• zodbpack : simple tool to pack ZODB [Łukasz Nowak]

0.20 (2012-03-28)

• slaprunner : fix recursive include of .gif files [Alain Takoudjou]

0.19 (2012-03-28)

• slaprunner : Add Support for more instances at the same time. [Alain Takoudjou]

• slaprunner : Add slapgrid process control and management. [Alain Takoudjou]

• slaprunner : Add Cloud9 IDE. [Alain Takoudjou]

• slaprunner : Add develop location of cloned git repository. [Alain Takoudjou]

0.18 (2012-02-23)

• slaprunner : Add git commit and push support. [Alain Takoudjou]

• slaprunner : Solve UI problems in project management. [Alain Takoudjou]

• slaprunner : Solve bug where ACE modifies carrier returns. [Alain Takoudjou]

• slaprunner : Add ajax view of build log. [Alain Takoudjou]

• slaprunner : Allow to edit any file in repository. [Alain Takoudjou]

0.17 (2012-02-10)

• slapbuilder: Update slapbuilder to new OpenSuse 12.1. [Cedric Le Ninivin]

• slapbuilder: Added automatic IPv6 support thanks to ViFiB OpenVPN. [Cedric Le Ninivin]

• slapbuilder: Added virtual machine support. [Cedric Le Ninivin]

• slapbuilder: Added “One Disk” feature. [Cedric Le Ninivin]

0.16 (2012-01-20)

• Fix MANIFEST.in [Cedric de Saint Martin]

0.15 (2012-01-20)

• slaprunner: Adding project and git repository management [Alain Takoudjou]

0.14 (2012-01-09)

• onetimeupload: SSL should be provided by an http frontend [Romain Courteaud]

0.13 (2011-12-30)

• builder: fix usage of mkstemp [Romain Courteaud]

0.12 (2011-12-29)

• builder: dynamic offset calculation [Romain Courteaud]

0.11 (2011-12-23)

• equeue: Add execution queue server. [Antoine Catton]

• pubsub: Add pubsub server using pubsubhubbub for notifications. [Antoine Catton]

0.10 (2011-11-07)

• slapbuilder: Support banging. [Łukasz Nowak]

• lamp : Fix bug when using file_token option [Alain Takoudjou]

0.9 (2011-10-29)

• Add LAMP tool to manage lamp instances. [Alain Takoudjou]

• Add onetimedownload : a simple script which download any http resource and save it into a specified file_path. [Nicolas Delaby]

0.8 (2011-10-24)

• Add new style to runner. [Alain Takoudjou]

0.7 (2011-10-17)

• generic: Added systool with killpidfromfile. [Łukasz Nowak]

• slaprunner: Avoid error when trying to remove instance directory containing not writable directories. [Alain Takoudjou]

0.6 (2011-07-13)

• slapbuilder: Typo during PKI repository creation fixed. [Łukasz Nowak]

0.5 (2011-07-12)

• slapbuilder: Do not play too much with buildout, SlapOS installation is responsible for correctly featured zc.buildout [Łukasz Nowak]

0.4 (1011-07-11)

• Update slaprunner to run on newer slapos (requires at least 0.10 to work). [Cedric de Saint Martin]

0.3 (2011-06-27)

• slapbuilder: use slapos provided networkcached buildout.

• slapbuilder: fix network issues on busy networks (“Neighbour table overflow.”)

0.2 (2011-06-10)

• In slapbuilder: drop to 100 partitions by default, move out PKI from USB key to real disk. [Łukasz Nowak]

0.1 (2011-05-27)

• Created slapos.toolbox from slapos.tool.builder, slapos.tool.runner, slapos.tool.cloudmgr, slapos.tool.ontimeupload and slapos.tool.networkcache. [Łukasz Nowak]