Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
9476a41
feat: require only LinkDataverse permission to link and unlink collec…
vera Mar 24, 2026
17e4829
feat: show "Link" button for collections in UI to non-superusers
vera Mar 24, 2026
a752d33
docs: add/update docs about linking
vera Mar 24, 2026
837cf5d
test: update LinkIT to test required permissions for collection linki…
vera Mar 24, 2026
6ce0b91
fix: fix mix-up of linking + linked collection when finding/deleting …
vera Mar 24, 2026
02ae1be
docs: add release note for #12076
vera Mar 24, 2026
2795446
docs: add missing indentation to docs about linking
vera Apr 9, 2026
4b3d572
feat: only show eligible collections to link to on DataversePage
vera May 19, 2026
0694268
feat: let CreateSavedSearchCommand require linking permission (for co…
vera May 19, 2026
9358632
feat: update text shown in linking dialog when there are no eligible …
vera May 19, 2026
8163186
fix: fix "Save" button behaviour in linking dialog
vera May 26, 2026
a1e4225
fix: fix display logic for "Link" button on collection page
vera May 28, 2026
985b349
Improve Dataverse REST OpenAPI source annotations
beepsoft Jun 2, 2026
9af0c8c
Merge branch 'refs/heads/develop' into feat/non-superuser-dataverse-l…
vera Jun 5, 2026
1a405ab
Configure OpenAPI post-processing filter
beepsoft Jun 17, 2026
17807d1
Improve OpenAPI annotation coverage for vacuum conformance
beepsoft Jun 17, 2026
61ba1d2
Merge remote-tracking branch 'iqss/develop' into openapi-improvements
beepsoft Jun 17, 2026
9eaa716
Centralize OpenAPI API-key security
beepsoft Jun 17, 2026
af16814
gb at request - drop gbrids req
qqmyers Jun 17, 2026
62e042e
Update requeriments for creating Anonymous Preview URL
cperezmig-dipc Jun 25, 2026
d126a41
fix link and reword slightly #12486
pdurbin Jun 25, 2026
844294b
Merge branch 'develop' into openapi-improvements
pdurbin Jul 2, 2026
5e7dfd9
Merge branch 'develop' into openapi-improvements
pdurbin Jul 2, 2026
16ea4cc
Merge pull request #12250 from vera/feat/non-superuser-dataverse-linking
sekmiller Jul 8, 2026
5363087
test docker images using docker compose locally
pdurbin Jul 1, 2026
ebe3b08
wait for API tests to complete before merging develop to master
pdurbin Jul 1, 2026
c2e435d
clarify how to build the installer
pdurbin Jul 1, 2026
ac49b21
force images to be redownloaded
pdurbin Jul 1, 2026
000ff70
test docker images after the release
pdurbin Jul 1, 2026
913ac1c
reword, the release has been made already
pdurbin Jul 1, 2026
f01b521
what to expect for version and build
pdurbin Jul 1, 2026
525c89f
run the job again as a workaround
pdurbin Jul 2, 2026
5d80425
soften language: it might work!
pdurbin Jul 2, 2026
f621538
ok to approve and merge anyway
pdurbin Jul 2, 2026
91762b2
Merge branch 'develop' into openapi-improvements
pdurbin Jul 10, 2026
58dd042
Merge pull request #12474 from QualitativeDataRepository/GbAtRequestFix
sekmiller Jul 10, 2026
92bb98b
improve docs, add release note #12439
pdurbin Jul 10, 2026
386dd50
change "provision" to "create" #12439
pdurbin Jul 10, 2026
09de088
Merge branch 'develop' into openapi-improvements
pdurbin Jul 10, 2026
782f883
Fix license acceptance pop-up for Archival Format download. (#12412)
janvanmansum Jul 13, 2026
1c3c898
Merge pull request #12439 from beepsoft/openapi-improvements
pdurbin Jul 13, 2026
b724824
Improve the Efficiency of the /api/index/perms API call (#12200)
qqmyers Jul 13, 2026
d071141
Merge pull request #12486 from cperezmig-dipc/patch-1
pdurbin Jul 13, 2026
5a7c14b
Added Suport for HTML in JSF UI for file description
stevenwinship Apr 10, 2026
4619c0b
change markup Checking from advanced to basic
stevenwinship May 20, 2026
2183134
add sanitizeBasicHTML to file-info-fragment
stevenwinship May 20, 2026
a08540c
added html supported description to search page
stevenwinship Jul 9, 2026
c1e3bca
handle html in search-include-fragment
stevenwinship Jul 9, 2026
8b92009
fix to strip link off description on search cards
stevenwinship Jul 13, 2026
6108e3a
Merge pull request #12318 from IQSS/11397-support-html-in-file-level-…
pdurbin Jul 13, 2026
ec55a0d
try WEB-INF/payara* files
qqmyers Mar 20, 2026
b926898
update doctype
qqmyers Mar 20, 2026
a66802b
CHanges per review
qqmyers Jun 22, 2026
ec0a1a0
Merge pull request #12240 from QualitativeDataRepository/WEB-INF/paya…
pdurbin Jul 14, 2026
ebb7c6c
feat(S3AccessIT): scrub minio from S3AccessIT test
srmanda-cs May 13, 2026
2d53004
feat(docker-compose-dev): scrub minio from docker-compose-dev.yml
srmanda-cs May 13, 2026
9947458
feat(conf/keycloak/docker-compose-dev): scrub minio from conf/keycloa…
srmanda-cs May 13, 2026
a11008d
feat(docker-compose-dev): add localstack_noredirect storage driver co…
srmanda-cs May 13, 2026
156096d
feat(keycloak/docker-compose-dev): add localstack_noredirect storage …
srmanda-cs May 13, 2026
a1a187b
test(S3AccessIT): use localstack_noredirect driver in testNonDirectUp…
srmanda-cs May 13, 2026
9352b94
test(S3AccessIT): update drivers doc strings to include localstack_no…
srmanda-cs May 13, 2026
c404586
fix(S3AccessIT): use distinct bucket name mybucket-noredirect for loc…
srmanda-cs May 13, 2026
8692fcc
docs(big-data-support): remove MinIO references
srmanda-cs May 15, 2026
2363796
docs(config): remove MinIO references
srmanda-cs May 15, 2026
68975fb
docs(S3AccessIO): remove MinIO references
srmanda-cs May 15, 2026
00c20dd
docs(S3AccessIT): remove MinIO references
srmanda-cs May 15, 2026
e0679a3
fix: remove unnecessary whitespace changes
srmanda-cs May 26, 2026
8592a27
docs(S3AccessIO): remove MinIO references
srmanda-cs May 26, 2026
590c2c5
feat(dev-start-frd.sh): remove MinIO initialization
srmanda-cs May 26, 2026
80d40eb
feat: remove whitespaces
srmanda-cs May 26, 2026
8b4958d
Clarify comment on S3 tags support
srmanda-cs May 26, 2026
a4fdf00
Ensure LocalStack no-redirect bucket exists for S3 tests
Copilot May 26, 2026
baab128
Handle generic S3 404 in S3AccessIT bucket setup
Copilot May 26, 2026
b705ef1
Null-check S3 error details in bucket existence helper
Copilot May 26, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 10 additions & 27 deletions conf/keycloak/docker-compose-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,16 +53,16 @@ services:
-Ddataverse.files.localstack1.download-redirect=true
-Ddataverse.files.localstack1.access-key=default
-Ddataverse.files.localstack1.secret-key=default
-Ddataverse.files.minio1.type=s3
-Ddataverse.files.minio1.label=MinIO
-Ddataverse.files.minio1.custom-endpoint-url=http://minio:9000
-Ddataverse.files.minio1.custom-endpoint-region=us-east-1
-Ddataverse.files.minio1.bucket-name=mybucket
-Ddataverse.files.minio1.path-style-access=true
-Ddataverse.files.minio1.upload-redirect=false
-Ddataverse.files.minio1.download-redirect=false
-Ddataverse.files.minio1.access-key=4cc355_k3y
-Ddataverse.files.minio1.secret-key=s3cr3t_4cc355_k3y
-Ddataverse.files.localstack_noredirect.type=s3
-Ddataverse.files.localstack_noredirect.label=LocalStackNoRedirect
-Ddataverse.files.localstack_noredirect.custom-endpoint-url=http://localstack:4566
-Ddataverse.files.localstack_noredirect.custom-endpoint-region=us-east-2
-Ddataverse.files.localstack_noredirect.bucket-name=mybucket-noredirect
-Ddataverse.files.localstack_noredirect.path-style-access=true
-Ddataverse.files.localstack_noredirect.upload-redirect=false
-Ddataverse.files.localstack_noredirect.download-redirect=false
-Ddataverse.files.localstack_noredirect.access-key=default
-Ddataverse.files.localstack_noredirect.secret-key=default
-Ddataverse.pid.providers=fake
-Ddataverse.pid.default-provider=fake
-Ddataverse.pid.fake.type=FAKE
Expand Down Expand Up @@ -260,23 +260,6 @@ services:
tmpfs:
- /localstack:mode=770,size=128M,uid=1000,gid=1000

dev_minio:
container_name: "dev_minio"
hostname: "minio"
image: minio/minio
restart: on-failure
ports:
- "9000:9000"
- "9001:9001"
networks:
- dataverse
volumes:
- ./docker-dev-volumes/minio_storage:/data
environment:
MINIO_ROOT_USER: 4cc355_k3y
MINIO_ROOT_PASSWORD: s3cr3t_4cc355_k3y
command: server /data

previewers-provider:
container_name: previewers-provider
hostname: previewers-provider
Expand Down
1 change: 1 addition & 0 deletions conf/localstack/buckets.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#!/usr/bin/env bash
# https://stackoverflow.com/questions/53619901/auto-create-s3-buckets-on-localstack
awslocal s3 mb s3://mybucket
awslocal s3 mb s3://mybucket-noredirect
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
## Feature Request
File metadata description will now support html in JSF UI. Is already supported in SPA.
1 change: 1 addition & 0 deletions doc/release-notes/12076-non-superuser-dataverse-linking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Dataverse collection linking and unlinking no longer requires superuser status. Users with the "Link Dataverse" permission on a collection can now perform these actions through the UI and API.
9 changes: 9 additions & 0 deletions doc/release-notes/12200-async-reindexing-permissions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Asynchronous Permissions Reindexing

The previously undocumented Solr permissions reindexing API endpoints have been improved.

- The endpoints are `/api/admin/index/perms` (asynchronous, all objects) and `/api/admin/index/perms/{id}` (synchronous, single object) now use POST instead of GET
- Both endpoints require superuser access.
- For the asynchronous reindex all endpoint, if an indexing process is already in progress, the API will return a 409 Conflict status.
- The asynchronous reindex all endpoint no longer runs as a single transaction. This avoids potential timeouts in larger installations.
- These endpoints are now documented in the Solr Search Index section of the Admin Guide.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fixed an issue where, in some cases, the license acceptance pop-up for the archival format option did not display correctly when downloading a complete dataset.
7 changes: 7 additions & 0 deletions doc/release-notes/12439-openapi.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
## Updates for API Users

- The quality of the OpenAPI document has been greatly improved, increasing from 10/100 to 98/100 according [vacuum](https://quobix.com/vacuum/). See [the guides](https://dataverse-guide--12439.org.readthedocs.build/en/12439/api/getting-started.html#getting-the-openapi-document) and #12439.

## Updates for Developers

- If you are adding or updating Dataverse APIs you are now required to add OpenAPI annotations. See [the guides](https://dataverse-guide--12439.org.readthedocs.build/en/12439/developers/api-design.html#openapi) and #12439.
4 changes: 2 additions & 2 deletions doc/sphinx-guides/source/admin/dataverses-datasets.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@ Moves a Dataverse collection whose id is passed to an existing Dataverse collect
Link a Dataverse Collection
^^^^^^^^^^^^^^^^^^^^^^^^^^^

Creates a link between a Dataverse collection and another Dataverse collection (see the :ref:`dataverse-linking` section of the User Guide for more information). Only accessible to superusers. ::
Creates a link between a Dataverse collection and another Dataverse collection (see the :ref:`dataverse-linking` section of the User Guide for more information). ::

curl -H "X-Dataverse-key: $API_TOKEN" -X PUT http://$SERVER/api/dataverses/$linked-dataverse-alias/link/$linking-dataverse-alias

Unlink a Dataverse Collection
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Removes a link between a Dataverse collection and another Dataverse collection. Only accessible to superusers. ::
Removes a link between a Dataverse collection and another Dataverse collection. Accessible to users with Link Dataverse permission on the linking Dataverse collection. ::

curl -H "X-Dataverse-key: $API_TOKEN" -X DELETE http://$SERVER/api/dataverses/$linked-dataverse-alias/deleteLink/$linking-dataverse-alias

Expand Down
31 changes: 31 additions & 0 deletions doc/sphinx-guides/source/admin/solr-search-index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,37 @@ This API will clear the Solr entry for the dataset specified. It can be useful i
This can be reversed of course by re-indexing the dataset with the API above.


Reindexing Permissions
----------------------

It is possible to just re-index the permissions on Solr entries without re-indexing the content.

Reindexing Permissions for All Objects
++++++++++++++++++++++++++++++++++++++

Re-index permissions for all Dataverse collections, datasets, and files. This is an asynchronous operation that may take a long time to complete on large installations. This endpoint requires the API token of a superuser.

.. code-block:: bash

export SERVER_URL=https://demo.dataverse.org
curl -X POST -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/admin/index/perms"

If indexing is already in progress, the API will return a 409 ("CONFLICT") response. Otherwise, it will return a 200 ("OK") response and the indexing process will start in the background. Check the server logs for progress.

Reindexing Permissions for a Single Object
++++++++++++++++++++++++++++++++++++++++++

It is also possible to re-index permissions for a single object by database ID:

.. code-block:: bash

export SERVER_URL=https://demo.dataverse.org
export ID=42
curl -X POST -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/admin/index/perms/$ID"

This operation is performed synchronously.


Manually Querying Solr
----------------------

Expand Down
9 changes: 8 additions & 1 deletion doc/sphinx-guides/source/api/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,16 @@ This API changelog is experimental and we would love feedback on its usefulness.
:local:
:depth: 1

v6.11
v6.12
-----
- The permission reindexing endpoints have been updated to use ``POST`` and require superuser access. They are now documented in the :doc:`/admin/solr-search-index` guide.

- **/api/admin/index/perms**

- **/api/admin/index/perms/{id}**

v6.11
-----
- The GET /api/mydata/retrieve, if the search returns no data, now includes the "data" block with 0 results. The message that was returned in "error_message" will be returned in "message" and the "success" will be `true`. All other errors will continue to reply with "success":false and the error message in "error_message".
- The endpoints GET, PUT AND DELETE for `/api/admin/dataverse/{alias}/storageDriver` have been moved to `/api/dataverses/{alias}/storageDriver`.
- The endpoint `/api/admin/dataverse/storageDrivers` has been moved and renamed to `/api/dataverses/{alias}/allowedStorageDrivers`. Regarding the change of the name, this endpoint will in the future only display the storageDrivers that are allowed on the specified collection, as of now, it will display the entire list of available Drivers on the installation.
Expand Down
4 changes: 3 additions & 1 deletion doc/sphinx-guides/source/api/getting-started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,9 @@ The fully expanded example above (without environment variables) looks like this

curl "https://demo.dataverse.org/openapi?format=json"

We are aware that our OpenAPI document is not perfect. You can find more information about validating the document under :ref:`openapi-dev` in the Developer Guide.
If you have feedback about the our OpenAPI document, please contact us through the usual channels under :ref:`getting-help-with-apis`.

You can find more information about validating the document under :ref:`openapi-dev` in the Developer Guide.

Beyond "Getting Started" Tasks
------------------------------
Expand Down
60 changes: 60 additions & 0 deletions doc/sphinx-guides/source/api/native-api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3746,6 +3746,66 @@ The fully expanded example above (without environment variables) looks like this

curl -H "X-Dataverse-key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X PUT "https://demo.dataverse.org/api/datasets/24/link/test"

Unlink a Dataset
~~~~~~~~~~~~~~~~

Removes a link between a dataset and a Dataverse collection (see :ref:`dataset-linking` section of Dataverse Collection Management in the User Guide for more information):

.. code-block:: bash

export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
export SERVER_URL=https://demo.dataverse.org
export DATASET_ID=24
export DATAVERSE_ID=test

curl -H "X-Dataverse-key: $API_TOKEN" -X DELETE "$SERVER_URL/api/datasets/$DATASET_ID/deleteLink/$DATAVERSE_ID"

The fully expanded example above (without environment variables) looks like this:

.. code-block:: bash

curl -H "X-Dataverse-key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X DELETE "https://demo.dataverse.org/api/datasets/24/deleteLink/test"

Link a Dataverse collection
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Creates a link between one Dataverse collection and another Dataverse collection (see :ref:`dataverse-linking` section of Dataverse Collection Management in the User Guide for more information):

.. code-block:: bash

export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
export SERVER_URL=https://demo.dataverse.org
export LINKED_DATAVERSE_ID=linked-collection
export LINKING_DATAVERSE_ID=linking-collection

curl -H "X-Dataverse-key: $API_TOKEN" -X PUT "$SERVER_URL/api/dataverses/$LINKED_DATAVERSE_ID/link/$LINKING_DATAVERSE_ID"

The fully expanded example above (without environment variables) looks like this:

.. code-block:: bash

curl -H "X-Dataverse-key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X PUT "https://demo.dataverse.org/api/dataverses/linked-collection/link/linking-collection"

Unlink a Dataverse collection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Removes a link between one Dataverse collection and another Dataverse collection (see :ref:`dataverse-linking` section of Dataverse Collection Management in the User Guide for more information):

.. code-block:: bash

export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
export SERVER_URL=https://demo.dataverse.org
export LINKED_DATAVERSE_ID=linked-collection
export LINKING_DATAVERSE_ID=linking-collection

curl -H "X-Dataverse-key: $API_TOKEN" -X DELETE "$SERVER_URL/api/dataverses/$LINKED_DATAVERSE_ID/deleteLink/$LINKING_DATAVERSE_ID"

The fully expanded example above (without environment variables) looks like this:

.. code-block:: bash

curl -H "X-Dataverse-key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X DELETE "https://demo.dataverse.org/api/dataverses/linked-collection/deleteLink/linking-collection"

Dataset Locks
~~~~~~~~~~~~~

Expand Down
32 changes: 29 additions & 3 deletions doc/sphinx-guides/source/developers/api-design.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,39 @@ OpenAPI

As you add API endpoints, please be conscious that we are exposing these endpoints as an OpenAPI document at ``/openapi`` (e.g. http://localhost:8080/openapi ). See :ref:`openapi` in the API Guide for the user-facing documentation.

We've played around with validation tools such as https://quobix.com/vacuum/ and https://pb33f.io/doctor/ only to discover that our OpenAPI output is less than ideal, generating various warnings and errors.
OpenAPI Annotations
~~~~~~~~~~~~~~~~~~~

To keep our OpenAPI document at a reasonable quality, you must add annotations to API endpoints you are adding and editing. Here's an example:

.. code-block:: java

@POST
@AuthRequired
@Path("/")
@Operation(summary = "Creates a license", description = "Creates a license when the authenticated user is a superuser and returns the created license location.")
@APIResponse(responseCode = "201", description = "License created with a Location header pointing to the new license.")
public Response addLicense(@Context ContainerRequestContext crc,
@RequestBody(description = "License definition to persist, including name, URI, active state, and sort order.",
content = @Content(mediaType = "application/json",
schema = @Schema(type = SchemaType.OBJECT,
description = "License definition accepted by the license administration API.")))
License license) {
...
}

If you are looking for a reference about the annotations used to generate the OpenAPI document, you can find it in the `MicroProfile OpenAPI Specification <https://download.eclipse.org/microprofile/microprofile-open-api-3.1/microprofile-openapi-spec-3.1.html#_detailed_usage_of_key_annotations>`_.

OpenAPI Quality
~~~~~~~~~~~~~~~

To use `vacuum <https://quobix.com/vacuum/>`_ with our recommended settings to check the quality of our OpenAPI document, see the :download:`README.md <../../../../scripts/openapi/README.md>` under ``scripts/openapi``.

You can prevent additional problems in our OpenAPI document by observing the following practices:

- When creating a method name within an API class, make it unique.

If you are looking for a reference about the annotations used to generate the OpenAPI document, you can find it in the `MicroProfile OpenAPI Specification <https://download.eclipse.org/microprofile/microprofile-open-api-3.1/microprofile-openapi-spec-3.1.html#_detailed_usage_of_key_annotations>`_.
- Check that HTTP GET and DELETE operations do not accept request bodies.
- Various notes under :download:`vacuum-recommended.yaml <../../../../scripts/openapi/vacuum-recommended.yaml>`.

Paths
-----
Expand Down
Loading
Loading