ERP5 related utilities.
Project description
erp5.util
Package containing various ERP5 related utilities.
Modules documentation
erp5.util.taskdistribution
Module to access TaskDistributor, used to run test on several machines and aggregating results. Use pydoc to get module documentation and usage example.
API Documentation
You can generate the API documentation using epydoc:
$ epydoc src/erp5
testnode
Utility able to call wget and varnishlog to extract Headers and return all failures according expected caching policy.
This utility is configurable through a configuration file like:
[web_checker] url = http://www.example.com/ working_directory = /home/me/tmp/crawled_content varnishlog_binary_path = varnishlog email_address = me@example.com smtp_host = localhost debug_level = debug [header_list] Last-Modified = True Cache-Control = max-age=300 max-age=3600 Vary = Accept-Language, Cookie, Accept-Encoding Accept-Language, Cookie Accept-Language,Cookie,Accept-Encoding Accept-Language,Cookie Expires = True [header url=.*/sitemap] Last-Modified = True [header content-type=.*/javascript] Last-Modified = True Cache-Control = max-age=3600 Expires = True [no_header content-type=(image/.*|text/css)] Vary = None [erp5_extension_list] prohibited_file_name_list = WebSection_viewAsWeb Base_viewHistory list prohibited_folder_name_list = web_page_module document_module
with:
url : website to check working_directory : fetched data will be downloaded varnishlog_binary_path : path to varnishlog email_address : email address to send result smtp_host : smtp host to use debug_level : log level of this utility (debug =>very verbose, info=>normal, warning=>nothing) header_list : Key == Header id. value: if equals to True, it means that header needs to be present in RESPONSE if it is a tuple, the Header value must sastify at least one of the proposed values erp5_extension_list: Optional section. prohibited_file_name_list: which check that any links redirect to prohibited forms like WebSection_viewAsWeb, Base_viewHistory, list, ... prohibited_folder_name_list: usefull to detect that links does not redirect to specified modules like, web_page_module, document_module, ...
This utility requires wget => 1.12 And a callable varnishlog. The utility must be run on same server where varnish is running.
web_checker reads varnishlogs to detect if a Query goes to the backend.
Changes
0.4.76 (2024-05-07)
testnode: - remove unused ‘zip_binary’ config
0.4.75 (2023-11-15)
testnode: - make killall support processes with changed title
0.4.74 (2022-05-13)
testnode: - retry slapos node instance more times before running test
0.4.73 (2022-04-22)
testnode: - remove unused scalability_tester - fix bug in python3
0.4.72 (2021-10-01)
testnode: - update local frontend slave (if configured) so tests use a fast and reliable frontend (on same LAN and / or machine)
0.4.71 (2021-09-08)
testnode: - various changes relate to SlapOS’ integration of Scalability tests
0.4.70 (2021-06-14)
testnode:
fix ResourceWarnings on Python 3
shorten instance partition paths
testsuite: remove EggTestSuite
0.4.69 (2020-10-29)
erp5.util.testnode:
propagate test_node_title to runTestSuite
pass arguments as environment variables
advertise log URL with log_frontend_url
0.4.68 (2020-05-22)
erp5.util.taskdistribution:
fix DummyTaskDistributor API to be able to run tests locally
erp5.util.testnode:
fix upgrader when HEAD is a merge commit
don’t log distributor URL
0.4.67 (2020-04-27)
erp5.util:
testnode: pass –log_directory to runTestSuite
EggTestSuite: support –log_directory
testnode: include a link to snapshot dir in log viewer
testnode: don’t crash log viewer app on network error
testnode: make the number of days to keep log configurable
0.4.66 (2020-01-30)
erp5.util:
testnode: Use shared parts when building softwares
0.4.65 (2019-10-30)
erp5.util:
testnode: Allow to run scalability tests against already existing instance
0.4.64 (2019-10-10)
erp5.util:
testnode: fix Computer.updateConfiguration call (Compatibility with slapos.core 1.5.0)
0.4.63 (2019-10-08)
erp5.util:
testnode: avoid testnode crash when trying to kill a process already dead
testnode: testnode: import xml2dict from its new place (Compatibility with slapos.core 1.5.0)
0.4.62 (2019-10-01)
erp5.util:
testnode: Fix scalability test runner logic for importing a test suite class
0.4.61 (2019-09-18)
erp5.util:
testnode: Fix scalability test runner
0.4.60 (2019-09-01)
erp5.util:
testnode: Fix some typos in the SlaOS API
0.4.59.1 (2019-08-13)
erp5.util:
Minor fix: Add missing ‘six’ dependency on setup.py
0.4.59 (2019-08-13)
erp5.util:
testnode: Update the SlapOS API
erp5.util: add support for Python 3
testnode: handle cases of errors when updating git repositories
testnode: fixed condition to not build dependencies like firefox
testnode: kill processes having slapos_directory in command line
testnode: spawn with close_fds=True in ProcessManager
0.4.58 (2019-03-05)
erp5.util
testnode: Give more time to supervisord to kill subprocess [Sebastien Robin]
0.4.57 (2019-02-25)
erp5.util
testnode: Allow to pass max_quantity to runComputerPartition [Lukasz Nowak]
testnode: use CPUs a bit less agressively [Jerome Perrin]
testnode: avoid to rebuild testnode dependencies (firefox) all the time [Sebastien Robin]
testnode: try much more agressively to kill remaining processes [Sebastien Robin]
0.4.56 (2018-09-28)
erp5.util
testnode: give more time for the slapos proxy to start
0.4.55 (2018-09-28)
erp5.util
testnode: properly support deletion of chmod’ed files [Jerome Perrin]
0.4.54 (2018-09-13)
erp5.util
testnode: update path of firefox
0.4.53 (2018-09-07)
erp5.util
testnode: give project title to runTestSuite [Sebastien Robin]
testnode: support chmod’ed files during directories cleanups [Jerome Perrin]
0.4.52 (2018-08-21)
erp5.util
Make scalability testing framework more stable. Stop using a dummy frontend master and use host.vifib.net frontend with a valid SSL certificate instead. Always use https. [Yusei Tahara]
0.4.51 (2017-07-17)
erp5.util
scalability testing framework [Roque Porchetto]
0.4.50 (2017-11-22)
erp5.util.testnode
call only methods on Distributor [Lukasz Nowak]
0.4.49 (2017-05-11)
erp5.util.taskdistribution:
Wrap in xmlrpclib.Binary if needed
0.4.48 (2017-04-20)
erp5.util.testnode:
fix values of –firefox_bin and –xvfb_bin [Julien Muchembled]
0.4.47 (2017-04-05)
erp5.util.testnode:
Make it more robust in cases where we have from time to time failures [Sebastien Robin]
cosmetic: avoid -repository suffix [Julien Muchembled]
0.4.46 (2016-09-29)
erp5.util.testnode:
Include js-logtail at the MANIFEST.in
0.4.45 (2016-08-05)
erp5.util.testnode:
Do not block all test suites if one of them define broken repository [Sebastien Robin]
Make sure proxy is really dead before starting new one [Sebastien Robin]
0.4.44 (2016-03-22)
erp5.util.testnode:
Cancel test result if testnodes are unable to create partitions and unable to find runTestSuite command.
Set specific environment variable to build NumPy/friends & Ruby gems in parallel.
For local repositories, ignore revision defined in software release.
Make it possible to define slapos parameters in test suites.
0.4.43 (2015-09-02)
erp5.util
Make services much more reactive when server is back [Sebastien Robin]
erp5.util.testnode - Simple log viewer app not to download the whole suite.log [Jérôme Perrin] - Make code more robust when checkout git files [Sebastien Robin]
0.4.42 (2014-12-02)
erp5.util.testnode
Typo [Jérôme Perrin]
Run first found runTestSuite from lowest matching partition, not random one [Cédric de Saint Martin]
erp5.util
Drop support for Python < 2.7 [Julien Muchembled]
0.4.41 (2014-08-07)
erp5.util.testnode
Fix running test location [Rafael Monnerat]
erp5.util
Move dealShebang into Utils [Rafael Monnerat]
0.4.40 (2014-07-30)
erp5.util.testnode
Bugfix for erp5/util/testnode/__init__.py [Rafael Monnerat]
0.4.39 (2014-07-30)
erp5.util.testnode
update SlapOSControler cmd calls [Rafael Monnerat]
0.4.38 (2014-04-16)
erp5.util.testnode:
cleanup after the merge of scalability code [Cedric de Saint Martin]
0.4.37 (2014-01-21)
erp5.util.scalability:
New module [Benjamin Blanc]
erp5.util.testnode:
Minimize writes to storage holding MySQL databases.
0.4.36 (2013-06-30)
erp5.util.testsuite:
delete git repos if url has changed [Sebastien Robin]
0.4.35 (2013-06-21)
erp5.util.testsuite:
Fix additional_bt5_repository_id into testnode.py [Benjamin Blanc]
0.4.34 (2013-04-11)
erp5.util.testsuite:
allow to define sub results in tests, like we do for selenium [Sebastien Robin]
0.4.33 (2013-03-14)
erp5.util.zodbanalyze:
Initial version of an improved version of ZODB’s ZODB/scripts/analyze.py [Kazuhiko Shiozaki]
0.4.32 (2013-03-13)
erp5.util.testnode:
add handling of httplib.ResponseNotReady error message [Sebastien Robin]
do not fail when a different test suite repository branch is specified [Sebastien Robin]
0.4.31 (2013-03-01)
erp5.util.testnode:
after resetting software, retry_software_count was not resetted correctly [Sebastien Robin]
0.4.30 (2013-02-20)
erp5.util.testnode:
keep almost no tmp files, sometimes there is many Gb in /tmp after one day [Sebastien Robin]
0.4.29 (2013-02-20)
erp5.util.testnode:
make it able to resist to problems with slapos proxy when building software [Sebastien Robin]
0.4.28 (2013-02-19)
erp5.util.testnode:
make it able to resist to problems with slapos proxy [Sebastien Robin]
0.4.27 (2013-02-15)
erp5.util.testnode:
testnode was still sometimes logging at several files at a time [Sebastien Robin]
0.4.26 (2013-02-14)
erp5.util.testnode:
do not reraise OSError when cleaning temp files
0.4.25 (2013-02-11)
erp5.util.testnode:
close all timers when quitting, this makes stopping an erp5tetsnode much faster [Sebastien Robin]
remove hack on slapos/testnode after fix of slapos.cookbook [Sebastien Robin]
remove old tmp files left by buildout (buildout has te bo fixed too) [Sebastien Robin]
remove logging handlers where the are not needed any more [Sebastien Robin]
fixed the kill command, it was not able to kill properly childs [Sebastien Robin]
0.4.24 (2013-02-11)
erp5.util.testnode:
Fixed wrong location for the construction os test suite software [Sebastien Robin]
0.4.23 (2013-02-11)
erp5.util.testnode:
Make erp5testnode allow remote access to test suite logs instead of uploading them to master [Tatuya Kamada], [Sebastien Robin]
0.4.22 (2013-01-08)
erp5.util.taskdistribution:
fix regression when used on Python < 2.7
0.4.21 (2013-01-07)
erp5.util.taskdistribution:
really fix lock to avoid errors with concurrent RPC calls
erp5.util.testnode:
do not run test suites on deleted branches
0.4.20 (2012-12-19)
erp5.util.testnode:
Make sure to kill grandchilds when killing a process [Sebastien Robin]
0.4.19 (2012-12-17)
erp5.util.testnode:
Fixed undefined variable [Sebastien Robin]
0.4.18 (2012-12-14)
erp5.util.testnode:
Solve ascii issues when deleting software [Sebastien Robin]
0.4.17 (2012-12-10)
erp5.util.testnode:
Add thread Timer to terminate locked processes [Sebastien Robin]
Add more unit tests [Pere Cortes]
0.4.16 (2012-11-14)
erp5.util.testnode:
Improve handling of Xvfb and firefox [Sebastien Robin]
check supported parameters of runTestSuite [Pere Cortes]
add unit for runTestSuite [Pere Cortes]
0.4.15 (2012-11-07)
erp5.util.testnode:
fixed profile generation when software repos is not defined first [Sebastien Robin]
ask wich test has priority to master more often [Sebastien Robin]
0.4.14 (2012-11-05)
erp5.util.testnode:
force rebuilding software to avoid using old soft/code [Sebastien Robin]
erp5.util.taskdistribution:
handle another possible error with master [Sebastien Robin]
0.4.13 (2012-10-31)
erp5.util.testnode:
Add unit test for erp5testnode (with some hardcoded path that needs to be fixed ASAP) [Sebastien Robin]
Split long functions into several more simple ones for code simplicity and readability [Sebastien Robin]
0.4.12 (2012-10-25)
erp5.util.testnode:
Fixed several issues introduced by the management of test suite by the master [Sebastien Robin]
0.4.11 (2012-10-22)
erp5.util.testnode:
Take test suite parameters from the master, to allow distribution of the work by the master [Pere Cortes], [Sebastien Robin]
0.4.10 (2012-10-01)
erp5.util.testnode:
Allow to use a firefox built by testnode for functional tests [Gabriel Monnerat]
0.4.9 (2012-10-01)
erp5.util.testnode:
remove –now parameter when calling slapgrid-sr since it is not yet well supported [Sebastien Robin]
0.4.8 (2012-09-27)
erp5.util.testnode:
use taskdistribution module to reduce code [Vincent Pelletier], [Pere Cortes]
0.4.7 (2012-09-03)
erp5.util.taskdistribution:
work around test lines acquiring values from parent when no value is provided. [Vincent Pelletier]
fix a regression introduced in 0.4.6 which allowed parallel XMLRPC calls, which is not supported. [Rafael Monnerat]
erp5.util.benchmark:
check whether at least one result file could be found when generating a scalability report. [Arnaud Fontaine]
make sure that diagram bars are properly aligned in scalability test report. [Arnaud Fontaine]
erp5.util.testsuite:
new module [Rafael Monnerat]
0.4.6 (2012-08-10)
erp5.util.taskdistribution:
set socket timeout for RPC calls to prevent a deadlock happens. [Rafael Monnerat]
0.4.5 (2012-07-04)
erp5.util.taskdistribution:
xmlrpclib does not support named parameters, use positional ones [Vincent Pelletier]
0.4.4 (2012-07-04)
erp5.util.taskdistribution:
New module [Vincent Pelletier]
0.4.3 (2012-04-24)
erp5.util.testnode:
Improve detection of the cancellation of a test on the master
better management of SIGTERM signal
cleanup test instances to make sure nothing stay from a previous test run
0.4.2 (2012-04-11)
erp5.util.testnode:
Improve testnode logs
add a thread to upload ongoing logs to the master regularly
if the software release is not built successfully after a few time, totally erase software. This help unblocking if buildout is unable to update software.
check if the last test result was cancelled in order to allow relaunching test without restarting testnode
0.4.1 (2012-02-29)
erp5.util.testnode:
Improve testnode’s reliability when contacting remote master
Try to build software releases multiple times before giving up
0.3 (2011-12-23)
erp5.util.webchecker:
Imported from https://svn.erp5.org/repos/public/erp5/trunk/utils/ Utility to check caching policy of websites
erp5.util.testnode:
improve logging [Sebastien Robin]
fix passing bt5_path [Gabriel Monnerat]
fix profile_path concatenation [Nicolas Delaby]
fix git updating and parsing repository paths [Julien Muchembled]
erp5.util.benchmark:
new utility, work in progress [Arnaud Fontaine]
0.2 (2011-09-20)
Imported from https://svn.erp5.org/repos/public/erp5/trunk/utils/
erp5.util.test_browser: Programmable browser for functional and performance tests for ERP5
erp5.util.benchmark: Performance benchmarks for ERP5 with erp5.utils.test_browser
0.1 (2011-08-08)
erp5.util.testnode imported from recipe-like slapos.cookbook [Łukasz Nowak]
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.