galaxy-data 24.1.2

Increase API robustness to invalid requests, improve compressed data serving by @mvdbeek in #18494

Prevent job submission if input collection element is deleted by @mvdbeek in #18517

Fix shared caches with extended metadata collection. by @jmchilton in #18520

Also check dataset.deleted when determining if data can be displayed by @mvdbeek in #18547

Fix for not-null in ‘column_list’ object by @hujambo-dunia in #18553

Fix h5ad metadata by @nilchia in #18635

Don’t set file size to zero by @mvdbeek in #18653

Make sure we set file size also for purged outputs by @mvdbeek in #18681

Fix wrong extension on pick data output by @mvdbeek in #18798

Fix copying workflow with subworkflow step for step that you own by @mvdbeek in #18802

Make pylibmagic import optional by @mvdbeek in #18813

Ignore converted datasets in invalid input states by @mvdbeek in #18850

Fix discovered outputs with directory metadata and distributed object by @mvdbeek in #18855

Raise MessageException instead of assertions on rerun problems by @mvdbeek in #18858

Fix wrong final state when init_from is used by @mvdbeek in #18871

Fix history import when parent_hda not serialized by @mvdbeek in #18873

Backport pod5 datatype by @TomHarrop in #18507

Fix syntax for SA2.0 by @jdavcs in #17663

Fix empty usernames in database + bug in username generation by @jdavcs in #18379

Fix input_step_parameters missing values that don’t have a label by @ahmedhamidawan in #18405

Improve workflow-related exception reporting by @mvdbeek in #18447

Enable all-vs-all collection analysis patterns. by @jmchilton in #17366

Visualizing workflow runs with an invocation graph view by @ahmedhamidawan in #17413

Better display of estimated line numbers and add number of columns for tabular by @bernt-matthias in #17492

Enable storage management by object store by @jmchilton in #17500

Set minimal metadata also for empty bed datasets by @wm75 in #17586

Type annotation improvements by @nsoranzo in #17601

Type annotation and CWL-related improvements by @nsoranzo in #17630

Update Python dependencies by @galaxybot in #17653

Code cleanups from ruff and pyupgrade by @nsoranzo in #17654

SQLAlchemy 2.0 by @jdavcs in #17778

Outline use of type_annotation_map to fix mypy issues by @jmchilton in #17902

Add email notifications channel by @davelopez in #17914

Model edits and bug fixes by @jdavcs in #17922

Model typing and SA2.0 follow-up by @jdavcs in #17958

Enable warn_unused_ignores mypy option by @nsoranzo in #17991

Add galaxy to user agent by @mvdbeek in #18003

Refactor galaxy.files plugin loading + config handling. by @jmchilton in #18049

Add stronger type annotations in file sources + refactoring by @davelopez in #18050

Add support for additional media types by @arash77 in #18054

Update Python dependencies by @galaxybot in #18063

Enable flake8-implicit-str-concat ruff rules by @nsoranzo in #18067

Script for deleting userless histories from database + testing + drop unused model testing code by @jdavcs in #18079

Add Net datatype by @martenson in #18080

Empower users to bring their own storage and file sources by @jmchilton in #18127

Harden User Object Store and File Source Creation by @jmchilton in #18172

Update db revision 24.1 release tags by @jdavcs in #18183

Tighten axt sniffer by @martenson in #18204

More structured indexing for user data objects. by @jmchilton in #18291

Chore: remove repetitive words by @tianzedavid in #18076

Fix import broken with forward merge by @mvdbeek in #18152

Downgrade count lines error to warning by @mvdbeek in #18175

Don’t set dataset peek for errored jobs by @mvdbeek in #18231

Transparently open compressed files in DatasetDataProvider by @mvdbeek in #18248

Raise exception when extracting dataset from collection without datasets by @mvdbeek in #18249

Set page importable to false when serializing by @mvdbeek in #18263

Fix first_dataset_element type hint by @mvdbeek in #18284

Do not copy purged outputs to object store by @mvdbeek in #18342

Fix user’s private role can be missing by @davelopez in #18381

Assign default data extension on discovered collection output by @mvdbeek in #18389

Replace busybox:ubuntu-14.04 image with busybox:1.36.1-glibc by @mvdbeek in #18428

Fix for converter tests by @bernt-matthias in #17188

correct dbkey for minerva display app by @hexylena in #17196

Fix invocation serialization if no state was set by @mvdbeek in #17278

Fix quotas ID encoding by @davelopez in #17335

Fix model store exports that include implicit conversions. by @jmchilton in #17346

Fix bug: create new PSAAssociation if not in database by @jdavcs in #17516

Fix social_core methods by @jdavcs in #17530

Fix ancient bug: incorrect usage of func.coalesce in User model by @jdavcs in #17577

Account for newlines in CIF Datatype sniffer by @patrick-austin in #17582

Anticipate PendingRollbackError in check_database_connection by @mvdbeek in #17598

Add basic model import attribute validation by @mvdbeek in #17628

More efficient change_state queries, maybe fix deadlock by @mvdbeek in #17632

Npz sniffing: do not read the whole file by @bernt-matthias in #17672

Assert that at least one file in npz zipfile ends with .npy by @mvdbeek in #17679

Workflow Comment Indexing by @ElectronicBlueberry in #17700

Fix source history update_time being updated when importing a public history by @jmchilton in #17728

Also set extension and metadata on copies of job outputs when finishing job by @mvdbeek in #17777

Defer job attributes that are usually not needed by @mvdbeek in #17795

Fix change_datatype PJA for dynamic collections by @mvdbeek in #17803

Simplify nested collection joins by @mvdbeek in #17817

Fix very slow user data table query by @mvdbeek in #17830

Update db revision 24.0 release tags by @jdavcs in #17834

Minor refactor of query building logic for readability by @jdavcs in #17835

Fix user login when duplicate UserRoleAssociation exists by @mvdbeek in #17854

Make columns types an empty list for empty tabular data by @bernt-matthias in #13918

port invocation API to fastapi by @martenson in #16707

SQLAlchemy 2.0 upgrades (part 5) by @jdavcs in #16932

Python 3.8 as minimum by @mr-c in #16954

Expose more tool information / navigability in UI. by @jmchilton in #17105

Add support for (fast5.tar).xz binary compressed files by @tuncK in #17106

SA2.0 updates: handling “object is being merged into a Session along the backref cascade path” by @jdavcs in #17122

Towards SQLAlchemy 2.0: fix last cases of RemovedIn20Warning by @jdavcs in #17132

Create pydantic model for the return of show operation - get: /api/jobs/{job_id} by @heisner-tillman in #17153

Much simpler default dataset permissions for typical users. by @jmchilton in #17166

Add future=True flag to SA engine by @jdavcs in #17174

Add future=True flag to SA session by @jdavcs in #17179

Vueifiy History Grids by @guerler in #17219

Convert sample object store configuration to YAML and support configuring inline by @natefoo in #17222

Migrate models to pydantic 2 by @mvdbeek in #17262

API endpoint that allows “changing” the objectstore for “safe” scenarios. by @jmchilton in #17329

Enable warn_unreachable mypy option by @mvdbeek in #17365

Fix type annotation of code using XML etree by @nsoranzo in #17367

Add explicit cache_ok attribute to JSONType subclass by @mvdbeek in #17376

More specific type annotation for BaseJobExec.parse_status() by @nsoranzo in #17381

Update to black 2024 stable style by @nsoranzo in #17391

Allow using tool data bundles as inputs to reference data select parameters by @mvdbeek in #17435

UI for “relocating” a dataset to a new object store (when safe) by @jmchilton in #17437

Allow filtering history datasets by object store ID and quota source. by @jmchilton in #17460

Faster FASTA and FASTQ metadata setting by @bernt-matthias in #17462

Feature SBOL datatypes by @guillaume-gricourt in #17482

Display workflow invocation counts. by @jmchilton in #17488

add npy datatype by @astrovsky01 in #17517

Enhance Avivator display app to support regular Tiffs by @davelopez in #17554

Update Python dependencies by @galaxybot in #17580

Add migrations revision identifier for 24.0 by @jdavcs in #17589

Rename to_dict to populate in FormDefintion to fix bug by @jdavcs in #16553

MINERVA display application: enable cors, add for tabular by @hexylena in #16737

Use AlignedSegment.to_string by @mvdbeek in #16803

Ruff and flake8 fixes by @nsoranzo in #16884

prep for updated h5py and typos by @mr-c in #16963

Fix cardinality violation error: subquery returns multiple results by @jdavcs in #17224

Fix Display Application link generation by @mvdbeek in #17227

Display application fixes and tests by @mvdbeek in #17233

Rollback invalidated transaction by @jdavcs in #17280

Set metadata states on dataset association, not dataset by @mvdbeek in #17474

Provide working routes.url_for every ASGI request by @mvdbeek in #17497

Implement default locations for data and collection parameters. by @jmchilton in #14955

Display beginning of non-text files as text instead of triggering a download by @SergeyYakubov in #15447

Tool Shed 2.0 by @jmchilton in #15639

Limit number of celery task executions per second per user by @claudiofr in #16232

Delete non-terminal jobs and subworkflow invocations when cancelling invocation by @mvdbeek in #16252

Towards SQLAlchemy 2.0 (upgrades to SA Core usage) by @jdavcs in #16264

Update Python dependencies by @galaxybot in #16409

Towards SQLAlchemy 2.0 (upgrades to SA ORM usage in /test) by @jdavcs in #16431

SQLAlchemy 2.0 upgrades to ORM usage in /lib by @jdavcs in #16434

Update Python dependencies by @galaxybot in #16436

Implement datatype upload warnings by @jmchilton in #16564

Support new genome browser chain file format by @claudiofr in #16576

Workflow Comments 💬 by @ElectronicBlueberry in #16612

Bump samtools converters by @bernt-matthias in #16668

Misc. edits/refactorings to session handling by @jdavcs in #16712

SQLAlchemy 2.0 upgrades (part 2) by @jdavcs in #16724

Migrate collection elements store to Pinia by @davelopez in #16725

Reset autocommit to False by @jdavcs in #16745

Drop legacy server-side search by @jdavcs in #16755

Optimize iteration in DatasetInstance model + SA2.0 fix by @jdavcs in #16776

Migrate a part of the jobs API to Fast API by @heisner-tillman in #16778

Replace file_name property with get_file_name function by @SergeyYakubov in #16783

Updated path-based interactive tools with entry point path injection, support for ITs with relative links, shortened URLs, doc and config updates including Podman job_conf by @sveinugu in #16795

Allow partial matches in workflow name tag search and search all tags for unquoted query by @ahmedhamidawan in #16860

Vueify Visualizations Grid by @guerler in #16892

Standardize to W3C naming for color. by @dannon in #16949

Move and re-use persist_extra_files by @mvdbeek in #16955

Fix invocation report to target correct workflow version. by @jmchilton in #17008

optimize object store cache operations by @SergeyYakubov in #17025

Use python-isal for fast zip deflate compression in rocrate export by @mvdbeek in #17342

Add magres datatype by @martenson in #17499

Merge 23.1 into dev by @mvdbeek in #16534

Merge release_23.1 into dev by @mvdbeek in #16768

Create db head merge revision. by @dannon in #16838

merge release_23.1 into dev by @martenson in #16933

Minor clarification/typo fix in datatypes.data by @dannon in #16993

Fix .file_name access in merge forward by @mvdbeek in #17097

Backport model store fixes by @mvdbeek in #17439

Fix succces typo by @mvdbeek in #17481

Fix User.current_galaxy_session by @mvdbeek in #17232

Backport #17188: Fix for converter tests by @mvdbeek in #17231

Skip change_datatype things if we’re not actually changing the extension by @mvdbeek in #16931

Fix copying metadata to copied job outputs by @mvdbeek in #17007

Update tar_to_directory dependency by @mvdbeek in #17009

Assert that DatasetCollectioElement has an associated object by @mvdbeek in #17071

Fix input dates in notifications: consider timezone offset by @davelopez in #17088

Allow relative URLs in broadcasts action links by @davelopez in #17093

Improve invocation error reporting by @mvdbeek in #16917

Add support for larch datatypes by @patrick-austin in #17080

Fix bad auto-merge of dev. by @jmchilton in #15386

Merge conflicting db migration branches into one by @jdavcs in #15771

Enable strict_equality mypy option by @nsoranzo in #15808

Fix revision scripts, run migrations in CI, add repair option, improve migrations utils by @jdavcs in #15811

Fix anonymous user uploads when vault is configured by @tchaussepiedifb in #15858

Fix nullable deleted column in API Keys table by @davelopez in #15956

Attempt to fix mypy check by @davelopez in #16103

Ensure session is request-scoped for legacy endpoints by @jdavcs in #16207

Reset autocommit to True (temporarily) by @jdavcs in #16283

Update 23.1 release tags for migration scripts by @jdavcs in #16294

Fix form builder value handling by @guerler in #16304

Fix tags ownership by @davelopez in #16339

Fix disk usage recalculation for distributed object stores by @mvdbeek in #16380

Job cache fixes for DCEs by @mvdbeek in #16384

Fix histories count by @davelopez in #16400

Fix double-encoding notification content by @mvdbeek in #16530

Optimize getting current user session by @mvdbeek in #16604

Fixes for conditional subworkflow steps by @mvdbeek in #16632

Copy the collection contents by default when copying a collection by @mvdbeek in #16717

Fix allowlist deserialization in file sources by @mvdbeek in #16729

Fix workflow output display without label by @mvdbeek in #16749

Fix and prevent persisting null file_size by @mvdbeek in #16855

Fix workflow import losing tool_version by @mvdbeek in #16869

Remove more flushes in database operation tools by @mvdbeek in #16875

Fix join condition for nested collection query by @mvdbeek in #16880

Empower Users to Select Storage Destination by @jmchilton in #14073

Add Storage Dashboard visualizations for histories by @davelopez in #14820

Towards decoupling datatypes and model by @jdavcs in #15186

Add Storage Management API by @davelopez in #15295

OIDC tokens by @SergeyYakubov in #15300

Expose additional beaker caching backends by @claudiofr in #15349

Follow up to #15186 by @jdavcs in #15388

Add support for visualizing HDF5 datasets. by @jarrah42 in #15394

Towards SQLAlchemy 2.0: drop session autocommit setting by @jdavcs in #15421

Update Python dependencies by @galaxybot in #15435

Unify url handling with filesources by @nuwang in #15497

Move TS to Alembic by @jdavcs in #15509

Update Python dependencies by @galaxybot in #15564

Update database_heartbeat for SA 2.0 compatibility by @jdavcs in #15611

Update supports_skip_locked, supports_returning for SA 2.0 compatibility by @jdavcs in #15633

Add Galaxy Notification System by @davelopez in #15663

Drop model mapping unit tests by @jdavcs in #15669

Add transactional state to workflow scheduling manager by @jdavcs in #15683

Remove DELETED_NEW job state from code base by @jdavcs in #15690

Fix/Enhance recalculate disk usage API endpoint by @davelopez in #15739

Drop database views by @jdavcs in #15876

Update Python dependencies by @galaxybot in #15890

FITS data format by @volodymyrss in #15905

Improve display chunk generation for BAMs by @wm75 in #15972

Add History Archival feature by @davelopez in #16003

Add alter_column migration utility by @jdavcs in #16009

Code cleanups from ruff and pyupgrade by @nsoranzo in #16035

Add missing fields to HistorySummary schema model by @davelopez in #16041

Dataset chunking tests (and small fixes) by @jmchilton in #16069

Add create_foreign_key migration utility by @jdavcs in #16077

Refactor models enums to eliminate schema dependency on model layer. by @jmchilton in #16080

Use automated naming convention to generate indexes and constraints in database by @jdavcs in #16089

Add zipped mongodb and genenotebook datatypes by @abretaud in #16173

Drop workarounds for old ro-crate-py and docutils versions by @mvdbeek in #16198

bring grids for (published) pages on par with workflows by @martenson in #16209

Add Visium datatype for squidpy and spatialomics tools by @astrovsky01 in #16255

Filter deleted keys from api_keys relationship by @dannon in #16321

Increase CustosAuthnzToken.external_user_id column size by @davelopez in #16818

Follow up on object store selection PR. by @jmchilton in #15654

Fix Enums in API docs by @davelopez in #15740

merge release_23.0 into dev by @martenson in #15830

Fix linting of FITS datatype code by @mvdbeek in #16169

backport of pysam tostring by @martenson in #16822