TracGViz

Trac Data Sources for Google Visualization API. Embed iGoogle gadgets using WikiFormatting.

Project description

Trac Data Sources for Google Visualization API. Embed iGoogle gadgets using WikiFormatting.

This package (plugin) provides components so that Trac be able to use widgets and related technologies provided by Google.

It allows to feed data managed by Trac to widgets based on Google Visualization API.
It allows embedding iGoogle Gadgets in wiki pages using WikiFormatting.

What’s new in version 1.4.2

#103 - Bug fixed: PIVOT fails for calculated columns with aggregate functions
#66 - Bug fixed: Unicode values in ticket list schema
#105 - Bug fixed: Unicode values in date constants schema
#106 - Bug fixed: Type error in dateDiff
#111 - Bug fixed: Error executing scalar function now()
#88 - Wiki page text and size in GViz data source
New! #92 - GViz QL : Implement PIVOT clause
New! #112 - Integer division in GViz queries
#113 - Bug fixed: Unknown: error in HTML cells
#67 Implement TracLinks for Ohloh widget gadgets
#104 - Bug fixed: Empty cells messed up by GViz MoinMoin encoder
#114 URL escape plus sign in query URL
Migrate tracgviz.testing to a new package dutest-trac
Remove relative import statements
Error message for GViz content providers when guessed mimetype is still None
Move dutest to tests_require rather than install_requires.
New! Naive typecasts in CSV to GViz MIME converter.
URL column in milestone GViz provider.

What’s new in version 1.4.1

New! The plugin is easy-installable. Modules like dutest, TracXmlRpcPlugin and Pygments are external dependencies.
New! Simple test cases (doctests) have been written for RPC handlers implemented by TracGViz : TimelineRPC, ReportRPC. They are run using a minimalistic framework built on top of dutest and allowing to (efficiently) write readable doctests for Trac plugins.
New! It is possible to select the columns to be returned, their order, and labels. There is partial support for SELECT statement, since only column names are allowed so far, and full support for LABEL clause.
New! Added full support for limitting the number of returned rows, as well as skipping a given number of first rows (i.e. OFFSET and LIMIT clauses in GViz QL).
New! A GViz QL error message is sent back to the caller if the data source doesn’t support the base table specified in FROM clause. This clause has been deprecated in recent versions of the API.
New! Calculated columns (e.g. arithmetic expressions) in LABEL and FORMAT clauses follow a syntax simlar to variable names e.g. a + sum(b)
New! FORMAT clause powered by PyICU.
New! ORDER BY clause supported in queries.
New! GROUP BY clause supported in queries.
New! Added data source and infrastructure to convert files containing tabular data in different formats to the JSON representation defined by Google Visualization API. It is integrated with Trac MIME conversions. The contents may be retrieved from anywhere if the right plugin is implemented.
New! Added GViz provider to convert contents of files under version control to GViz Data Tables (provided that a MIME conversion is available). Further optimizations have been deferred.
New! Added GViz provider to convert files attached to Trac resources in order to obtain GViz Data Tables (provided that a MIME conversion is available). Especially implemented to help users like Robert Obertholtzer. Thanks for the feedback and help !
New! Included helper to convert CSV files under version control to GViz API JSON format. Especially implemented to help users like Robert Obertholtzer. Thanks for the feedback and help !
New! Added GVizTableMacro, a macro that embeds in TracWiki pages the data tables returned by LOCAL GViz data providers (i.e. instances of IGVizDataProvider). It is possible to include custom parameters (e.g. tq for queries written using Google Visualization API Query Language). The table contents are displayed using HTML tables, hence neither gadgets nor internet connection are needed in this case.
New! Implemented TSV Response Format encoder. It is compliant with the specification and response is encoded using UTF-16 charset. Nonetheless outFileName parameter is not supported yet.
New! Added sample TracGViz providers. They illustrate how easy is to implement custom GViz providers in order to offer your data to the world.
New! Added support for HMAC (RFC 2104) hash by using hmac module.
New! GViz providers list rendered on accessing /gviz path.
New! Support for parsing other (more complex) expressions in select clause (e.g. aggregate functions, -date- constants).
New! Support for parsing WHERE clause in detail
New! Support for parsing more complex expressions in ORDER BY clause (e.g. aggregate functions).
New! Constants accepted in select clause.
New! Support for parsing and evaluating scalar functions in select clause.
New! Support for parsing and evaluating expressions in select clause.
New! GViz QL grammar to include SQL clauses and refactor current parsing solution.
Column type checks may trigger error responses on missmatch.
Fix GViz providers index, broken using Trac=1.0
Apache(TM) Bloodhound web UI enhancements
Upgrade RPC providers for Trac=1.0 and Apache(TM) Bloodhound.
CSV and TSV encoders are implemented by using csv module in stdlib.
Include helper to convert CSV attachments to GViz API JSON format.
It is possible to create reports from remote via XML-RPC API.
Report SQL is not shown in ReportRPC.get if client is not granted REPORT_SQL_VIEW permission.
Added support in the test suite for adding and loading test data.
Test cases written for all supported (and unsupported) GVizQL features.
Some test cases written for data sources included in TracGViz 1.3.4 that are not about version control. Much more work is needed :(
The plugin works even if Pygments is not installed. In that case only GViz QL will be missing and thereby all other features will be available.
Further precautions to ensure that responses are UTF-8 encoded, unless otherwise stated.
Added support for GViz queries (i.e. tq parameter) in GViz TracLinks providers.
Included provider documentation in a resource file.
It is possible to iterate over rows and values of DataTable instances.
Bug fixed: Relative URL converted into absolute in ReportRPC.execute only if needed.
Bug fixed: Data provided by GVizChagesetProvider in columns chg and target were exchanged. Now they are in the right place.
Bug fixed: Details about the latest changeset are shown by GVizChagesetProvider when only since and until parameters are specified.
Bug fixed: Enhanced compatibilty with Trac XmlRpcPlugin=1.0.6 and the way it handles datetime values. Errors like TypeError: unsupported operand type(s) for -: instance and datetime.datetime and AttributeError: datetime.datetime object has no attribute value do not happen anymore.
Bug fixed #57 : ValueError was raised (sometimes) in GadgetAliases.__init__ when parsing aliases in config file. Not anymore.

What’s new in version 1.3.4

New ! GVizQuickJumpEntries offers data about interesting places in the repository (a.k.a. quick jump entries). This includes tags and branches, for instance. The data is provided by VersionControlRPC.
Bug fixed: Formerly all revisions were included in the revision log if only individual revisions were requested. Now the result sets include just the information for the particular changesets supplied in rev parameter. Tested with both hg and svn.
Bug fixed: Exception with message Invalid list index was raised when using Subversion backend when requesting the log for individual revisions. This works OK now for both hg and svn.
Bug fixed: Wrong default value for fmt parameter in GVizChangesetDetails. Now it’s OK.
Bug fixed: Formerly undocumented column in GVizTimelineFilters provider now contains appropriate description.
WikiFormatting may be used to document the data sources. It may be included in the documentation written for parameters, columns, and the data source itself.
Bug fixed: Documentation for providers having dynamic schemas was not previously rendered. This has been fixed and a new property has been added in docs to highlight whether the data source returns always the same columns or if they can vary depending upon run-time conditions.
Milestones may be filtered by name by providing a UNIX pattern in filter_by parameter.
UNIX filename patterns are compiled using REs for better performance.
Bug fixed: When using Hg backend, the code in version 1.3.3 in order to revisions iterator (VCS RPC) entered infinite loop (under certain conditions apparently related to branching or version numbers). This doesnt happen anymore.
Bug fixed: Now VCS RPC retrieves all changesets when using Hg backend. The implementation doesn’t guess anymore the revision order and retrieves all that information directly from the underlying VCS connector in use. The former algorithm was weak because it only considered the first parent of a given revision. Therefore many others (especially all those in different branches) were missing from the result set.
Individual revisions (and not just intervals like before) may be selected in GVizChangesets (i.e. to obtain the revision log).
Bug fixed: As a consequence of the changes introduced in version 1.3.3 (especially in ReportRPC and VersionControlRPC) together with bug #5586@TH.org in XmlRpcPlugin, made impossible to render the main displaying documentation about the different RPC handlers. Both docstrings have been modified and everything is back to normal right now.

What’s new in version 1.3.3

New ! Since version 1.3.3 macro aliases are supported so that embedding iGoogle gadgets in wiki pages be even easier. Aliases are names of macros configured using gadgets.aliases option which are shortcuts to use iGoogleGadget with a fixed name. For example, if the aforementioned option is set to MotionChart=http://www.google.com/ig/modules/motionchart.xml then the following snippets are equivalent (so users wont need to remember gadget URLs anymore but names they are familiar with ;).

[[iGoogleGadget(url=http://www.google.com/ig/modules/motionchart.xml, _table_query_url=http://spreadsheets.google.com/tq?range=B3%3AG17&key=1234, _table_query_refresh_interval=5)]]

[[iGoogleGadget(url=gadget:google:modules:motionchart, _table_query_url=[gviz:google:sheet:1234:B3-G17], _table_query_refresh_interval=5)]]

[[MotionChart(_table_query_url=[gviz:google:sheet:1234:B3-G17], _table_query_refresh_interval=5)]]
Aliases for Google’s visualization gadgets are offered by default.
TracLinks expressions are supported in config options used to define iGoogleGadget macro aliases.
New ! GVizTicketList now lists the tickets created in a Trac environment together with their attributes. Custom fields, if available, are also included in separate columns. That’s why the columns in the data tables returned as well as their order can change from time to time, depending on the plugins enabled in the particular environment and other factors.
New ! ReportRPC now provides an interface to Trac’s report module via XML-RPC protocol. So far report execution is only supported for reports defined by using SQL and TracQuery syntax. This means that saved custom queries specified using URLs are not supported yet, but will be (hopefully) in a near future. In the mean time a NotImplementedError exception is raised.
New ! GVizAvailableReports now lists available reports and their data, including report ID, title, description and the query string used to select the tickets to be included in each report.
Important ! Architectural change. From now on Request objects can be supplied in to IGVizDataProvider.get_data_schema method. This change allows to implement more dynamic data providers since schemas can vary depending upon the run-time values provided for some parameters.
New ! GVizReportProvider now offers the data included in custom Trac Reports defined by Trac users. This is a very dynamic data provider, and also the first example where schema can change depending upon the report specification.
New ! VersionControlRPC now provides an interface to Trac’s Version Control API via XML-RPC protocol. Therefore information about files in the project’s repository and also about changesets is available using standard protocols. It is posible to list files and folders (ls) recursively with depth limit and filters written like UNIX filename patterns. File attributes (e.g. size, content type, …), data about revisions (e.g. message, author, …) in a given (time) interval, file and folder history, and full description of changes performed at a given revision, are some of the features supported so far. Looking forward for further useful data to be published ! This feature was “tested” using Mercurial SCM (hg) and Subversion (svn), but the code SHOULD work as long as connectors fully compatible with Trac’s Version Control API be used. However the different approaches taken by different VCS may lead to wrong behaviors. So if you find any bug, please let me know.
New ! GVizSourceFiles now offers information about individual files and those located in a folder. This data source supports filters (i.e. UNIX filename patterns), and recursive lookup with optional depth value. It is also possible to get the data for an specific revision. The results are mostly file attributes (e.g. size, extension, content type …), but the actual values depend upon the capabilities supported by the specific repository connector in use.
New ! GVizSourceFileHistory now makes available the modifications performed on a group of files during a given time interval. Boundary values may be either datetime values or revision identifiers (supported by the specific repository connector). Clients’ll get back detailed information about the changes performed on the given files in the different revisions together with the file’s attributes at that specific time. This data source supports filters (i.e. UNIX filename patterns) as well as recursive lookup with optional depth value. Multiple datetime formats are supported as usual.
New ! GVizChangesetDetails now provides a detailed report containing all the individual changes comitted at once onto the repository and recorded in separate changesets. It is possible to specify multiple changeset identifiers (supported by the specific repository connector) (and | or) time intervals. Boundary values may be either datetime values or changeset identifiers.
New ! GVizChangesets now provides meta-data describing the different changesets committed to the repository related to a Trac environment (i.e. revision ID, messages, author, and date). It is possible to limit the results by specifying a time interval. Boundary values may be either datetime values or changeset identifiers.
All the data providers interacting with the repository have been “tested” using the connector for Mercurial SCM (i.e. hg). In this case there are a few open issues that seem to be related with the lack of support in TracMercurial for copy events (but it’s not a definitive conclusion). The current implementation SHOULD work with other connectors compatible with Trac’s Version Control API, but THERE IS NO WARRANTY OF ANY KIND if you use another VCS. However if your connector is not causing any trouble then you the use of the aforementionned data sources SHOULD NOT cause any damage to both your environment and your repository. Further tests will be performed in the future but there’s no clear schedule. You could be the first one to test them. Suggestions, patches and further feedback is welcome in order to overcome the actual limitations.
Added api.IHashLibrary interface so that plugins and extensions can contribute with other secure hash, message digest and checksum algorithms in order to compute sig field in GViz API responses.
All the code needed to support secure hash algorithms provided by hashlib module can be found from now on in class stdhash.HashLib. It supports the following methods : sha1, sha224, sha256, sha384, sha512, and md5. Additional algorithms may also be available depending upon the OpenSSL library that Python uses on your platform.
All the code needed to support checksum algorithms provided by zlib module can be found from now on in class stdhash.ZLibChecksum. It supports the following methods : adler32, crc32
Minor adjustments to adapt support for sig parameter to the recent introduction of IHashLibrary interface.
Bug fixed: Formerly sig value was included in GViz API response only if both, the hash method was setup and the sig parameter was set in the request. From now on only the first condition has to be met in order to send the hash value back to the client.
Bug fixed: Minor bug while displaying instances of datetime using plain text encoders.
Support in DataTable.SingleValueToJS for automatic conversion of instances of xmlrpclib.DateTime and int values (POSTFIX timestamp) into instances of datetime. This allows to supply such values in columns having types date, datetime, and timeofday (not possible with gviz_api 1.0).
Support in DataTable.SingleValueToJS for automatic conversion of unicode strings using utf-8 encoding. This allows to supply such strings in columns having string type (not possible with gviz_api 1.0).
Module rpc has been added to group all RPC handlers offered by TracGViz package.
Documentation added to highlight the use of timeline.ticket_show_details option in trac.ini in order to retrieve all events related to ticket changes (e.g. attachments).
Bug fixed: TimelineRPC doesnt fail when filter definitions provided by instances of ITimelineEventProvider interface contain a third (i.e. checked) element. It also handles unicode characters appropriately.
MoinMoin encoder now returns text/x.moin.wiki MIME type.
Few optimizations and refactorings (e.g. unnecessary imports have been removed) … bah!

What’s new in version 1.3.2

TracGVizSystem now implements IPermissionRequestor interface. This means that Trac admins can use GVIZ_VIEW permission to control which users may access GViz data sources or not. On accessing such data, if the aforementioned permission is not granted to the user performing the request then a GViz API error response is returned to the caller with reason set to user_not_authenticated in case of anonymous access, or access_denied otherwise.
Now absolute URLs are returned while expanding TracLinks expressions found used inside iGoogleGadget macro. Formerly this prevented the iGoogle container to find data inside the project environment in case TracLinks expressions were used.
Add support for sheet option in GoogleDocsConnector in order to have links to data inside an specific sheet provided its name. Hence, links like the following are supported from now on: https://spreadsheets.google.com/tq?key=ABCD&sheet=Sheet1, https://spreadsheets.google.com/tq?key=ABCD&sheet=Sheet+2. Sheet names may have alphanumeric characters as well as whitespaces. Syntax -> gviz:google:sheet:<spreadsheet_id>[:[<sheet_name>][:<top_cell>-<bottom_cell>]][?[headers=<number>]]
Formerly values in columns having types date, datetime or timeofday were shown in non-JSON responses (e.g. HTML) in a way similar to new Date(2009, 12, 2). Now they are rendered using the following formats %Y-%m-%d, %Y-%m-%d %H:%M:%S, %H:%M:%S respectively.
New ! The following (non-standard) formats are supported using special format identifiers for out option in tqx parameter: MoinMoin wiki tables (class extfmt.GVizMoinEncoder out:moin).
Every wiki GViz provider now supports name expansion using Unix shell-style wildcards. This means that if such patterns are specified for name parameter either once or multiple times, then all those wiki pages for which a match is found will be included in the result set (e.g. /gviz/wiki/index?tqx=out:html&all&name=Wiki*&name=TracN* will return information about all the versions of the wiki pages starting with either Wiki or TracN prefixes, such as WikiNewPage, WikiHtml, WikiProcessors, WikiRestructuredText, WikiStart, TracNavigation, WikiRestructuredTextLinks, TracNotification, WikiMacros, WikiFormatting, WikiPageNames, WikiDeletePage).

What’s new in version 1.3.1

Assertions concerning permissions are supported, but not directly. They are delegated to the underlying layers on top of which the data sources have been implemented (e.g. XML-RPC handlers).
GadgetLinksDispatcher has been renamed. Now it is known as LinksTreeDispatcher. It supports multiple root namespaces (i.e. so far gadgets, and gviz) as well as the former hierarchy of links.
IGadgetTracLinksResolver now is known as ITracLinksItem interface. From now on its get_link_resolvers method has to return the name of the root namespace (e.g. gviz).
New ! TracLinks expressions for Google Apps Reporting Visualization API have been implemented. It may be necessary to be logged in to the Google Apps control panel in order to access the data. Syntax -> gviz:google:apps:<report_key>@<report_date>:<domain>[?tq=<query>]
New ! It is possible to make reference to Google Spreadsheets using TracLinks expressions starting with gviz:google:sheet: prefix. They also support range attribute -to select only specific cells- and headers option -in order to specify how many rows are header rows-. Syntax -> gviz:google:sheet:<sheet_id>[:<top_cell>-<bottom_cell>][?headers=<number>]
New ! It is possible to make reference to Google’s standard visualizations using TracLinks expressions starting with gadget:google:modules: prefix. Syntax -> gadget:google:modules:<chart_name>
TracLinks expansion for custom arguments in iGoogleGadget macro is not a dream anymore. You can enclose TracLinks expressions in brackets and assign such values to custom arguments. In such cases the macro will perform the expansion looking for the target URL. Hence the first expression shown below can be written now as shown in the second example. Both of them will embed in the wiki page a motion chart displaying the data found in cells B3-G17 inside Google Spreadsheet number 1234.

[[iGoogleGadget(url=http://www.google.com/ig/modules/motionchart.xml, _table_query_url=http://spreadsheets.google.com/tq?range=B3%3AG17&key=1234, _table_query_refresh_interval=5)]]

[[iGoogleGadget(url=gadget:google:modules:motionchart, _table_query_url=[gviz:google:sheet:1234:B3-G17], _table_query_refresh_interval=5)]]
All links to third-party sites resulting from expanding a TracLinks expression are rendered in wiki pages with the well-know icon []->.
Instances of GVizXMLRPCAdapter now can implement the method xmlrpc_namespace. It is similar to the former gviz_namespace method, but returns the XML-RPC namespace. This way multiple data sources can be implemented using the services provided by a single XML-RPC component.
New ! GVizSearchFiltersProvider now allows to list all search filters installed in a Trac environment.
New ! It is possible to document the parameters accepted by a data source using gviz_param decorator.
GVizProviderList macro now displays the parameters accepted by data providers.
Now it is much easier to handle unicode, date & time and values returned by the different data sources. From now on there is no need to convert instances of xmlrpclib.DateTime in to instances of datetime.date*
New ! GVizSearchProvider now gives access to search results returned by Trac built-in search engine. You can specify a query string and control the search filters (e.g. wiki, ticket, milestone, pydoc) in use.
New ! You can get the names of the actions that can be performed on a given ticket using ticket/actions data source.
New ! You can get all the meta-data describing ticket fields by accessing ticket/fields data source.
New ! GVizTicketChangeLog provides a detailed changelog in order to know the modifications made to different tickets. The data source is available at ticket/log.
New ! GVizTicketAttachments provides detailed information about the files attached to different tickets. The data source is available at ticket/attachments.
New ! GVizTimelineFilters provides detailed information about the different filters offered by those sources contributing to the timeline. The data source is available at timeline/filters.
New ! GVizTimelineEvents provides detailed information about the timed events reported by those sources contributing to the timeline. It supports time slicing, and filtering. The data source is available at timeline/log.
GVizMilestoneProvider has been enhanced. Formerly it just offered the milestone name, due and completed time, plus its description. Now it also contains some statistics on groups of tickets like those displayed in the roadmap views. This includes the amount of work that’s been done, the units used to measure the indicators, percent of the whole work that’s been already done, the number of active and closed tickets, and the percent they represent with respect to the total number of tickets.
New ! GVizWikiPages provides detailed information about the wiki pages found in the Trac environment. It supports multiple features, being timed filters, page selection and multiple versions, just a few of them. The data source is available at wiki/index.
New ! GVizWikiAttachments provides detailed information about the files attached to wiki pages found in the Trac environment. The data source is available at wiki/attachments.
The right MIME type (i.e. application/json) is sent back to the client in the HTTP response when JSON is requested. This is consistent with RFC xxx.

What’s new in version 1.2.3

Minor changes … bah!

What’s new in version 1.2.2

The meaning of the different columns in the table returned by GViz data sources as well as the default labels can be documented using gviz_col. This function is compliant with annotations as defined by PEP 3107.
Data sources’ documentation can be included in wiki pages using GVizDataSource macro.
Docstrings have changed a little.
Bug fixed: Until now, if no protocol handler was enabled then the project environment became broken, because of a ValueError exception being raised. This is annoying and therefore has been fixed. Now if a URL having gviz prefix is accessed and there’s no protocol handler available, a Not Found (404) HTTP response is returned to the caller with message Unable to find any protocol handler.

What’s new in version 1.2.1

The gviz_api module (version 1.0.0) is redistributed in this package. This has been made to fall back to this implementation if no other has been previously installed.
From this version on, the work is licensed under the Apache License. See COPYRIGHT and NOTICE for details.
Babel is no longer a dependency.
Full TracLinks expansion in iGoogleGadget macro parameters.
iGoogle gadgets logo included as image resource. Borders template updated.
Bug fixed: Until now, if an underlying RPC handler was disabled then environment initialization failed since an annoying ValueError exception was raised. That won’t happen anymore. Now, once a method is requested on the corresponding GViz data source provider, a GViz error response is returned to the caller containing an explanatory message.

What’s new in version 1.2.0

Added support to display iGoogle gadgets using WikiFormatting extensions (i.e. iGoogleGadget macro).
Added support to upload and host custom image-based gadget borders inside the project environment.
Bug fixed : All requests handled by IGVizDataProvider implementors not having tq and tqx parameters (e.g. http://myserver.com/project_name/gviz/ticket/status) now output the correct data. Previously on accessing these URLs the HTTP response body contained an HTML error page.

What’s new in version 1.1.0

Support has been added to implement Google Visualization API data sources by reusing existing Trac XML RPC components.
Some (but not all) data sources for Trac ticket system are provided. They are based on the following Trac XML-RPC providers (listed hereafter by namespace): ticket.milestone, ticket.severity, ticket.type, ticket.resolution, ticket.priority, ticket.component, ticket.version, ticket.status.

What’s new in version 1.0.0

An architecture is available so as to provide a project’s data in the format specified in Google Visualization API protocol specification (version 0.5) api.TracGVizSystem.
Multiple protocol handlers (e.g. for different versions, and protocol evolution) are allowed by implementing the interface api.IGVizProtocolHandler. There is native support for version 0.5 of Google Visualization API protocol through GViz_0_5.
It is possible to register new data sources by implementing the interface api.IGVizDataProvider
A pluggable architecture is at hand to retrieve table contents in multiple formats by merely implementing api.IGVizTableEncoder interface. The following formats are supported : JSON (class stdfmt.GVizJsonEncoder), HTML (class stdfmt.GVizHtmlEncoder), CSV (class stdfmt.GVizCSVEncoder).
The exception conditions mentioned in Google Visualization API protocol specification (version 0.5) have been identified.