Open data portal
Project description
uData
Customizable and skinnable social platform dedicated to (open)data.
The full documentation is hosted on Read the Docs.
Changelog
Current (in progress)
- Dynamic API documentation, Enhancement to Pull #1542 - #1542
- Normalize resource.format (migration - :warning: need reindexing) #1563
- Typed resources #1398
- [breaking] Enforce a domain whitelist when resource.filetype is file #1567
- Initial data preview implementation #1581 #1632
- Switch to PyPI.org for package links #1583
- Handle some alternate titles and alternate URLs on licenses for improved match on harvesting #1592
- Resource modals: use new single resource API call #1547
- Protocol-relative URLs support #1599
- Audience metrics: use only
views#1607 - Allow to specify a dataset acronym #1217
- Add missing spatial granularities translations #1636
- Add resend confirmation email link to login screen #1653
- Refactoring of discussions code base #1623
- Fix Firefox custom error handling, part 2 #1671
- The map widget can now be configured (tiles URL, initial position...) #1672
- Fix the temporal coverage facet query string parsing #1676
- Fix search auto-complete hitbox #1687
- Simplify
ExtrasFieldform field signature (no need anymore for theextrasparameter) #1698 - Ensure registered extras types are properly parsed from JSON. Remove the need for custom
db.Extraclasses #1699 - Register known extras types #1700
- Fix 404/missing css on front pages #1709
- Fix markdown max image width (front) #1707
- Adds ETag to internal avatar for efficient caching control #1712
- Show resource type in modal (front) #1714
- Starts using harvest backend
config(validation, API exposition,HarvestFilters...) #1716 - Remove extras from datasets search index (needs reindexation) #1718
1.3.12 (2018-05-31)
1.3.11 (2018-05-29)
- Protect Resource.need_check against malformed/string dates #1691
- Fix search auto-complete loading on new page #1693
1.3.10 (2018-05-11)
1.3.9 (2018-05-07)
- Prevent linkchecker to pollute timeline as a side-effect. (migration). Warning, the migration will delete all dataset update activities #1643
- Fix OAuth authorization screen failing with unicode
SITE_TITLE#1624 - Fix markdown handling of autolinks with angle brackets and factorize (and test) markdown
parse_html()#1625 - Fix timeline order #1642
- Fix markdown rendering on IE11 #1645
- Consider bad UUID as 404 in routing #1646
- Add missing email templates #1647
- Polyfill
ChildNode.remove()for IE11 #1648 - Improve Raven-js/Sentry error handling #1649
- Prevent regex special characters to break site search #1650
1.3.8 (2018-04-25)
- Fix sendmail regression #1620
1.3.7 (2018-04-24)
- Fix some search parameters validation #1601
- Prevent API tracking errors with unicode #1602
- Prevent a race condition error when uploading file with concurrent chunking #1606
- Disallow resources dict in API #1603
- Test and fix territories routing #1611
- Fix the client-side Raven/Sentry configuration #1612
- Raise a 404 in case of unknown RDF content type #1613
- Ensure current theme is available to macros requiring it in mails #1614
- Fix documentation about NGinx configuration for https #1615
- Remove unwanted commas in default
SECURITY_EMAIL_SUBJECT_*parameters #1616
1.3.6 (2018-04-16)
- Prevent OEmbed card to be styled when loaded in bootstrap 4 #1569
- Fix organizations sort by last_modified #1576
- Fix dataset creation form (and any other form) #1584
- Fix an XSS on client-side markdown parsing #1585
- Ensure URLs validation is the same everywhere #1586
1.3.5 (2018-04-03)
- Upgrade
sifterto0.5.3#1548 - Upgrade
jquery-validationto 1.17.0 and fixes some issues with client-side URL validation #1550 - Minor change on OEmbed cards to avoid theme to override the cards
font-family#1549 - Improve cli unicode handling #1551
- Fix DCAT harvester mime type detection #1552
- Add the missing harvester URL in admin #1554
- Fix harvester preview/job layout #1553
- Fix some search unicode issues #1555
- Small fixes on OEmbed URL detection #1556
- Use nb_hits instead of views to count downloads #1560
- Prevent an XSS in TermFacet #1561
- Fix breadcrumb bar layout on empty search result #1562
1.3.4 (2018-03-28)
- Remove territory claim banner #1521
- Expose an OEmbed API endpoint using the new cards #1525
- Small topic fixes #1529
- Fixes the search result vertical cut off #1530
- Prevent visually disabled pagination buttons from being clicked #1539
- Fixes "sort organization by name" not working #1537
- Non-admin users should not see the "publish as anyone" filter field on "publish as" screen #1538
1.3.3 (2018-03-20)
- Fixes on upload: prevent double upload and bad chunks upload #1516
- Ensure OAuth2 tokens can be saved without
refresh_token#1517
1.3.2 (2018-03-20)
- Support request-body credential in OAuth2 (Fix a regression introduced in 1.3.0) #1511
1.3.1 (2018-03-15)
- Fix some geozones/geoids bugs #1505
- Fix oauth scopes serialization in authorization template #1506
- Prevent error on site ressources metric #1507
- Fix some routing errors #1508
- Mongo connection is now lazy by default, preventing non fork-safe usage in celery as well as preventing commands not using the database to hit it #1509
- Fix udata version not exposed on Sentry #1510
1.3.0 (2018-03-13)
Breaking changes
- Switch to
flask-cliand dropflask-script. Deprecated commands have been removed. #1364 - Update card components to make them more consistent #1383 #1460
- udata is now protocol (
http/https) agnostic. This is now fully the reverse-proxy responsibility (please ensure that you are using SSL only in production for security purpose). #1463 - Added more entrypoints and document them. There is no more automatically enabled plugin by installation. Plugins can now properly contribute translations. #1431
New features
- Soft breaks in markdown is rendered as line return as allowed by the commonmark specifications, client-side rendering follows the same security rules #1432
- Switch from OAuthlib/Flask-OUAhtlib to Authlib and support all grants type as well as token revocation #1434
- Chunked upload support (big files support) #1468
- Improve tasks/jobs queues routing #1487
- Add the
udata schedule|unschedule|scheduledcommands #1497
Bug fixes and minor changes
- Added Geopackage as default allowed file formats #1425
- Fix completion/suggestion unicode handling #1452
- Added a link to change password into the admin #1462
- Fix organization widget (embed) #1474
- High priority for sendmail tasks #1484
- Add security.send_confirmation template #1475
Internals
- Switch to pytest as testing tool and expose a
udatapytest plugin #1400
1.2.11 (2018-02-05)
- Translate Flask-Security email subjects #1413
- Fix organization admin pagination #1372
- Fix missing spinners on loading datatables #1401
- Fixes on the search facets #1410
1.2.10 (2018-01-24)
- Markdown rendering is now the same between the back and the frontend. #604
- Make the dataset page reuses section and cards themable. #1378
ValueErroris not hidden anymore by the Bad Request error page, it is logged. #1382- Spatial encoding fixes: prevent breaking unicode errors. #1381
- Ensure the multiple term search uses a
ANDoperator #1384 - Facets encoding fixes: ensure lazy strings are propery encoded. #1388
- Markdown content is now easily themable (namespaced into a
markdownclass) #1389 - Fix discussions and community resources alignment on datasets and reuses pages #1390
- Fix discussions style on default theme #1393
- Ensure empty harvest jobs properly end #1395
1.2.9 (2018-01-17)
- Add extras field in discussions #1360
- Fix datepicker #1370
- Fix error on forbidden scheme in
is_urlharvest filter #1376 - Fix an error on rendering present territory date #1377
1.2.8 (2018-01-10)
- Fix html2text dependency version #1362
1.2.7 (2018-01-10)
1.2.6 (2018-01-04)
- Fix wrongly timed notification on dataset creation with misformed tags #1332
- Fix topic creation #1333
- Add a
udata worker statuscommand to list pending tasks.[breaking] Theudata workercommand is replaced byudata worker start. #1324 - Prevent crawlers from indexing spammy datasets, reuses and organizations #1334 #1335
- Ensure Swagger.js properly set jQuery.ajax contentType parameter (and so data is properly serialized) #1126
- Allows theme to easily access the
owner_avatar_urltemplate filter #1336
1.2.5 (2017-12-14)
- Fix misused hand cursor over the spatial coverage map in dataset admin #1296
- Fix broken post edit page #1295
- Display date of comments in dataset discussions #1283
- Prevent
reindexcommand from failing on a specific object and log error instead. #1293 - Position the community resource link icon correctly #1298
- Add a sort option to query of list of posts in API #1301
- Import dropdown behavior from
udata-gouvfrand fix hidden submenus on mobile #1297 - show message for emtpy dataset search #1044
1.2.4 (2017-12-06)
- Fix flask_security celery tasks context #1249
- Fix
dataset.qualityhandling when no format filled #1265 - Ignore celery tasks results except for tasks which require it and lower the default results expiration to 6 hours #1281
- Import community resource avatar style from udata-gouvfr #1288
- Terms are now handled from markdown and customizable with the
SITE_TERMS_LOCATIONsetting. #1285 - Deeplink to resource #1289
1.2.3 (2017-10-27)
- Check only the uncollapsed resources at first on dataset view #1246
1.2.2 (2017-10-26)
- Fixes on the
search index command#1245
1.2.1 (2017-10-26)
- Introduce
udata search indexcommmand to replace both deprecatedudata search initandudata search reindexcommands. They will be removed in udata 1.4. #1233 - Rollback oauthlib from 2.0.5 to 2.0.2, pending a permanent solution #1237
- Get cached linkchecker result before hitting API #1235
- Cleanup resources checksum (migration) #1239
- Show check results in resource modal #1242
- Cache avatar rendering #1243
1.2.0 (2017-10-20)
New features and big improvements
- Expose harvester scheduling through the API and the admin interface #1123
- Added a
udata infocommand for diagnostic purpose #1179 - Switch from static theme avatars/placeholders to identicons for readability (mostly on discussions) #1193
- Move croquemort features to a generic link checker architecture #1110
- CKAN and OpenDataSoft backends are now optionnal separate udata extensions #1213
- Better search autocomplete #1222
- Big post improvements (discussions support, navigation, fixes...) #1224
Breaking changes
- Upgrade to Celery 4.1.0. All celery parameters should be updated. (See Celery options documentation #1150
- Switch to Crowdin to manage translations #1171
- Switch to
Flask-Security.Flask-Security-Forkshould be uninstalled before installing the new requirements #958
Miscellaneous changes and fixes
- Display organization metrics in the organization page tab labels #1022
- Organization dashboard page has been merged into the main organization page #1023
- Fix an issue causing a loss of data input at the global search input level #1019
- Fixes a lot of encoding issues #1146
- Add
.ttland.n3as supported file extensions #1183 - Improve logging for adhoc scripts #1184
- Improve URLs validation (support new tlds, unicode URLs...) #1182
- Properly serialize empty geometries for zones missing it and prevent leaflet crash on invalid bounds #1188
- Start validating some configuration parameters #1197
- Remove resources without title or url [migration] #1200
- Improve harvesting licenses detection #1203
- Added missing delete post and topic admin actions #1202
- Fix the scroll to a discussion sub-thread #1206
- Fix duplication in discussions [migration] #1209
- Display that a discussion has been closed #1216
- Explicit dataset search reuse facet context (only known reuses) #1219
- Optimize indexation a little bit #1215
- Fix some reversed temporal coverage [migration] #1214
1.1.8 (2017-09-28)
- Display membership modal actions buttons for site administrators and on membership display. #1176
- Fix organization avatar in admin profile #1175
1.1.7 (2017-09-25)
- Prevent a random territory from being displayed when query doesn't match #1124
- Display avatar when the community resource owner is an organization #1125
- Refactor the "publish as" screen to make it more obvious that an user is publishing under its own name #1122
- Make the "find your organization" screen cards clickable (send to the organization page) #1129
- Fix "Center the full picture" on user avatar upload #1130
- Hide issue modal forbidden actions #1128
- Ensure spatial coverage zones are resolved when submitted from the API or when querying oembed API. #1140
- Prevent user metrics computation when the object owner is an organization (and vice versa) #1152
1.1.6 (2017-09-11)
- Fix CircleCI automated publication on release tags #1120
1.1.5 (2017-09-11)
- Fix the organization members grid in admin #934
- Fix and tune harvest admin loading state and payload size #1113
- Automatically schedule validated harvesters and allow to (re)schedule them #1114
- Raise the minimum
ravenversion to ensure sentry is filtering legit HTTP exceptions #774 - Pin GeoJSON version to avoid breaking changes #1118
- Deduplicate organization members #1111
1.1.4 (2017-09-05)
- Fix packaging
1.1.3 (2017-09-05)
- Make the spatial search levels exclusion list configurable through
SPATIAL_SEARCH_EXCLUDE_LEVELS. #1101 - Fix facets labelizer with html handling #1102
- Ensure territories pages have image defined in metadatas #1103
- Strip tags in autocomplete results #1104
- Transmit link checker status to frontend #1048
- Remove plus signs from search query #1048
1.1.2 (2017-09-04)
- Handle territory URLs generation without validity #1068
- Added a contact button to trigger discussions #1076
- Improve harvest error handling #1078
- Improve elasticsearch configurability #1096
- Lots of fixes admin files upload 1094
- Prevent the "Bad request error" happening on search but only on some servers #1097
- Migrate spatial granularities to new identifiers
- Migrate remaining legacy spatial identifiers #1080
- Fix the discussion API documention #1093
1.1.1 (2017-07-31)
- Fix an issue preventing reuse edition: #1027
- Fix an issue preventing user display and edit in admin: #1030
- Fix an error when a membership request is accepted: #1028
- Fix issue modal on a reuse: #1026
- Fix sort by date on admin users list: #1029
- Improve the
purgecommand #1039 - Ensure search does not fail when a deleted object has not been unindexed yet #1063
- Start using Celery queues to handle task priorities #1067
- Updated translations
1.1.0 (2017-07-05)
New features and improvements
-
Added a DCAT harvester and expose metadata as RDF/DCAT. #966 See the dedicated documentions:
-
Images are now optimized and you can force rerendering using the
udata images rendercommand. -
Allowed files extensions are now configurable via the
ALLOWED_RESOURCES_EXTENSIONSsetting and both admin and API will have the same behavior #833. -
Improve and fix notifications: #928
- Changed notification style to toast
- Fix notifications that weren't displayed on form submission
-
Add a toggle indicator on dataset quality blocks that are collapsible #915
-
Integrating latest versions of GeoZones and GeoLogos for territories. Especially using history of towns, counties and regions from GeoHisto. #499
Breaking Changes
- Themes are now entrypoint-based #829. There is also a new theming documention.
- Images placeholders are now entirely provided by themes #707 #1006
- Harvester declaration is now entrypoint-based #1004
Fixes
-
Ensure URLs are stripped #823
-
Lot of fixes and improvements on Harvest admin UI #817:
- harvester edition fixed (and missing API added)
- harvester deletion fixed
- harvester listing is now paginated
- more detailed harvesters widgets
- ensure harvest source are owned by a user or an organization, not both [migration]
-
Pure Vue.js search facets #880. Improve and fix the datepicker:
- Proper sizing and positionning in dropdowns
- Fix initial value not being displayed
- Make it usable on keyboard
- Allows to define
minandmaxvalues to disable some dates - Keyboard input is reflected into the calendar #615
-
Disable
nextbutton when no file has been uploaded #930 -
Fix badges notification mails #894
-
Fix the
udata search reindexcommand #1009 -
Reindex datasets when their parent organization is purged #1008
Miscellaneous / Internal
-
Upgrade to Flask-Mongoengine 0.9.3, Flask-WTF 0.14.2, mongoengine 0.13.0. #812 #871 #903
-
Upgrade to Flask-Login 0.4.0 and switch from Flask-Security to the latest Flask-Security-Fork #813
-
Migrated remaining widgets to Vue.js #828:
- bug fixes on migrated widgets (Issues button/modal, integrate popover, coverage map)
- more coherent JS environment for developpers
- lighter assets
- drop Handlebars dependency
-
bleach and html5lib have been updated leading to more secure html/markdown cleanup and better performances #838
-
Drop
jquery-slimscrolland fix admin menu scrolling #851 -
drop jquery.dotdotdot for a lighter css-only solution (less memory consumption) #853
-
Lighter style #869:
- Drop glyphicons and use only Font-Awesome (more coherence, less fonts)
- lighter bootstrap style by importing only what's needed
- make use of bootstrap and admin-lte variables (easier for theming)
- proper separation between front and admin style
-
Drop
ExtractTextPluginon Vue components style: -
Pure Vue.js modals. Fix the default membership role. Added contribute modal. #873
-
Easier Vue.js development/debugging:
- Drop
Vue.config.replace = false: compatible with Vue.js 1/2 and no more style guessing #760 nameon all components: no moreAnonymous Componentin Vue debugger- No more
Fragments - More ES6 everywhere
- Drop
-
Make metrics deactivable for tests #905
1.0.11 (2017-05-25)
- Fix presubmit form errors handling #909
- Fix producer sidebar image sizing #913
- Fix js
Model.save()not updating in some cases #910
1.0.10 (2017-05-11)
- Fix bad stored (community) resources URLs [migration] #882
- Proper producer logo display on dataset pages
- Fix CKAN harvester empty notes and
metadatafile type handling - Remove (temporary) badges metrics #885
- Test and fix topic search #892
1.0.9 (2017-04-23)
- Fix broken post view #877
- Fix new issue submission #874
- Display full images/logo/avatars URL in references too #824
1.0.8 (2017-04-14)
- Allow more headers in cors preflight headers #857 #860
- Fix editorialization admin #863
- Fix missing completer images and ensure completion API is usable on a different domain #864
1.0.7 (2017-04-07)
- Fix display for zone completer existing values #845
- Proper badge display on dataset and organization page #849
- Remove useless
discussionsfrom views contexts. #850 - Fix the inline resource edit button not redirecting to admin #852
- Fix broken checksum component #846
1.0.6 (2017-04-01)
- Default values are properly displayed on dataset form #745
- Prevent a redirect on discussion fetch #795
- API exposes both original and biggest thumbnail for organization logo, reuse image and user avatar #824
- Restore the broken URL check feature #840
- Temporarily ignore INSPIRE in ODS harvester #837
- Allow
X-API-KEYandX-Fieldsin cors preflight headers #841
1.0.5 (2017-03-27)
- Fixes error display in forms #830
- Fixes date range picker dates validation #830
- Fix badges entries not showing in admin #825
1.0.4 (2017-03-01)
- Fix badges trying to use API too early #799
- Some minor tuning on generic references #801
- Cleanup factories #808
- Fix user default metrics not being set [migration] #809
- Fix metric update after transfer #810
- Improve spatial completion ponderation (spatial zones reindexation required) #811
1.0.3 (2017-02-21)
- Fix JavaScript locales handling #786
- Optimize images sizes for territory placeholders #788
- Restore placeholders in search suggestions, fix #790
- Fix share popover in production build #793
1.0.2 (2017-02-20)
- Fix assets packaging for production #763 #765
- Transform
udata_versionjinja global into a reusable (by themes)package_version#768 - Ensure topics datasets and reuses can display event with a topic parameter #769
- Raise a
400 Bad Requestwhen a badclassattribute is provided to the API (for entry point not using forms). #772 - Fix datasets with spatial coverage not being indexed #778
- Ensure theme assets cache is versionned (and flushed when necessary) #781
- Raise maximum tag length to 96 in order to at least support official INSPIRE tags #782
- Properly raise 400 error on transfer API in case of bad subject or recipient #784
- Fix broken OEmbed rendering #783
- Improve crawlers behavior by adding some
meta[name=robots]on pages requiring it #777
1.0.1 (2017-02-16)
- Pin PyMongo version (only compatible with PyMongo 3+)
1.0.0 (2017-02-16)
Breaking Changes
- 2016-05-11: Upgrade of ElasticSearch from 1.7 to 2.3 #449
You have to re-initialize the index from scratch, not just use the reindex command given that ElasticSearch 2+ doesn't provide a way to delete mappings anymore. The command is udata search init and may take some time given the amount of data you are dealing with.
- 2017-01-18: User search and listing has been removed (privacy concern)
New & Improved
-
2017-01-06: Add some dataset ponderation factor: temporal coverage, spatial coverage, certified provenance and more weight for featured ones. Need reindexation to be taken into account.
-
2016-12-20: Use all the Dublin Core Frequencies plus some extra frequencies.
-
2016-12-01: Add the possibility for a user to delete its account in the admin interface
In some configurations, this feature should be deactivated, typically when there is an SSO in front of udata which may cause some inconsistencies. In that case, the configuration parameter DELETE_ME should be set to False (True by default).
- 2016-05-12: Add fields masks to reduce API payloads #451
The addition of fields masks in Flask-RESTPlus allows us to reduce the retrieved payload within the admin — especially for datasets — and results in a performances boost.
Fixes
- 2016-11-29: Mark active users as confirmed #619
- 2016-11-28: Merge duplicate users #617 (A reindexation is necessary after this migration)
Deprecation
Theses are deprecated and support will be removed in some feature release. See Deprecation Policy.
- Theses frequencies are deprecated for their Dublin Core counter part:
fortnighly⇨biweeklybiannual⇨semiannualrealtime⇨continuous
0.9.0 (2017-01-10)
- First published version
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
Close
Hashes for udata-1.4.0.dev9779-py2.py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 2a31c70f26bc0ec6ed7f1d2b7d686c0f8d8fe7a02c01bba730a629f5b85d8784 |
|
| MD5 | e5e74e2aa2ac012201595d5dc4505529 |
|
| BLAKE2b-256 | 8fba0ae9dc3962e7599b52d36756790104bc49afa5b76841690c01e54e4ea20c |
