English | Português Brasileiro | Español

Dataverse Plugin

This plugin is the result of a partnership between SciELO Brazil and Lepidus. It enables the integration of Open Journal Systems (OJS) and Open Preprint Systems (OPS) with a Dataverse repository.

This way, authors can submit research data associated with their manuscripts during the submission process to the journal or preprint server. The research data remains available within the editorial workflow (for example, it can be made available during article peer review or preprint moderation) and is linked to the publication in OJS/OPS.

For a better understanding of the installation and use of this plugin, check our institutional video: Dataverse plugin - Introduction and tutorial.

Compatibility

This plugin is compatible with the following PKP applications:

• OPS and OJS versions 3.3 and 3.4.

Check the latest compatible version for your application on the Releases page.

All versions are compatible with Dataverse 5.x and 6.x.

Requirements for usage

1. app_key

The OJS/OPS instance must have the application key (app_key, in the [general] section of config.inc.php) set up. The 3.5 installer and upgrade create it; if it is missing, your system administrator can generate it with php lib/pkp/tools/appKey.php generate.

This is required to use the API credentials provided, that are stored encrypted in the database. Changing the key afterwards makes the stored credentials unreadable, and they must be entered again in the plugin settings.

Installation

This plugin is available for installation via the PKP Plugin Gallery. For installation, follow these steps:

1. Access the Dashboard area of your website.
2. Navigate to Settings > Website > Plugins > Plugin Gallery.
3. Search for the plugin called Dataverse Plugin and click on its name.
4. In the window that opens, click on Install and confirm that you want to install the plugin.

By following these steps, the plugin will be installed on your OJS/OPS. After installation, whenever you want to check if a new version is available, just follow the same path and check the plugin's status in the list.

Instructions for use

Configuration

After installation, it is necessary to enable the plugin. This is done in Website Settings > Plugins > Installed Plugins.

With the plugin enabled, you should expand its options by clicking the arrow next to its name and then accessing its Settings.

In the new window, the settings Dataverse URL, API Token, Terms of Use and Additional Instructions will be displayed.

You have to indicate the full Dataverse URL repository where the research data will be deposited. For example: https://demo.dataverse.org/dataverse/anotherdemo

The Terms of Use can be defined for each language configurated in your application. If you have questions about what they are, consult those responsible for the repository.

Important: The API Token belongs to a Dataverse user account. For more information on how to obtain the API token, see the Dataverse User Guide.

It is important to mention that the Dataverse user account will be included in the list of contributors of the datasets deposited by the plugin (for more information, see this discussion).

Therefore, we recommend the creation of a specific user for the journal/preprint server, instead of using a person's personal account, because each deposit will be associated with that account.

After filling in the fields, just confirm the action by clicking Save. The plugin will only work after filling in these data.

Request time limits

Connections to Dataverse have a 5-second limit, and each request has a total limit that depends on the operation: 15 seconds for reads, 60 seconds for each file download, 30 seconds for dataset creation, 60 seconds for each file upload and 30 seconds for other changes (including deletion). Administrators can change these limits in the application's config.inc.php:

[dataverse]
read_timeout = 15
download_timeout = 60
create_timeout = 30
upload_timeout = 60
write_timeout = 30

Values must be whole numbers of seconds between 1 and 300, written without a leading zero (the application reads 060 as an octal number); any other value, such as on or 1.5, falls back to the default. Files are deposited one after the other in the same request, so before increasing a limit, check that the PHP and web server request limits cover the time of all uploads together. A request that reaches its limit is reported as a timeout: Dataverse may still have completed the operation.

Use

A "Research Data" deposit dialog is shown in the "Files" step of the Submission proccess.

When the plugin creates a dataset, it fills in the Deposit Date of its citation metadata with the day of the deposit; editing the metadata later keeps that date. Authors are never asked for it, even if the collection marks it as required.

Authors, moderators, editors or managers can also edit the dataset, before publication, on the "Research data" item added to the Dataverse section of the submission's workflow menu. A "Data statement" item is added right next to it, so the data availability statement can be reviewed and changed during the editorial workflow.

The "Dataverse Report" in Statistics > Reports generates a CSV for a period of submission dates or of final decision dates. A final decision is an acceptance or decline that no later decision undid, such as sending the submission back to review or reverting the decline (sending it to production keeps the acceptance); in OPS, it is the first posting of the preprint or its decline; and an accepted submission that was declined afterwards is counted only as declined. The CSV has the accepted (OJS) or posted (OPS), declined and total submissions, how many of them have research data, the data statement types of each group, the datasets with deposit or publication errors and the number of files in the deposited datasets.

The dataset citation and its status indicators appear in the Research data header, using the same status style as Publication. The files are displayed above the metadata form, without tabs.

Before each file upload, the plugin waits up to 30 seconds for Dataverse to finish processing (ingesting) the files already in the dataset. If Dataverse is still processing them after that, the file is not uploaded and an error asks the user to try again in a few minutes; when this happens during a deposit, the dataset and the files already sent are kept, and a new deposit continues from the file that was not sent.

In OJS, reviewers can have access to research data files during the review process. Reviewers access to these files can be restricted on Workflow Settings, so they can only see research data when they agree to review the submission.

Interrupted deposits

The plugin records every deposit step before calling Dataverse. If a deposit is interrupted after the dataset was created, a new attempt reuses the same dataset: it sends the metadata again, so changes made in the meantime are kept, and sends only the files that were not confirmed yet. If the research data files were changed in the meantime, the partial dataset is deleted and the deposit starts again with the new selection. When Dataverse rejects a step (an invalid license or an expired token, for example), nothing was changed there, the error is shown, and the deposit can be tried again once the cause is fixed. When the result of a step is unknown — the request timed out or the process stopped while creating the dataset or sending a file — new attempts for that submission are refused with a message asking to contact the editorial team, so the plugin never creates a duplicate dataset or sends a file twice.

These deposits must be reconciled by an administrator, directly in the database, for the affected submission only:

1. Make sure no request for the submission is still running, and back up its rows in dataverse_deposits, dataverse_studies and draft_dataset_files.
2. For the state creating, find in Dataverse whether the dataset was created. If it was, save its persistent ID in persistent_id and go on to step 4. If Dataverse proves nothing was created, set the state to retired and increment revision, restricted to the submission_id, the inspected state and revision: the next deposit starts from the beginning.
3. For the state uploading, find which files of the manifest reached the dataset and record their Dataverse ids in receipts; a file that did not reach Dataverse stays without a receipt.
4. With persistent_id filled in, in a single transaction restricted to the submission_id, the inspected state and revision, set the state to ready and increment revision, then try the deposit again from the "Research data" panel.
5. If the Dataverse URL changed after the dataset was created, the deposit cannot go on in the new collection: keep or delete the dataset in the previous one as needed, then set the state to retired and increment revision.

Development Instructions:

1. Clone the Dataverse plugin repository
2. To use the plugin into the PKP Application, copy it's folder to the /plugins/generic directory and make sure the folder is called "dataverse".
3. And from the root of the PKP Appplication directory, execute this command to update the database, creating the tables used by the plugin:
   • php tools/upgrade.php upgrade

Running Tests

Tests that talk to Dataverse use a real Dataverse collection. Export its credentials before running them:

export DATAVERSE_URL="https://demo.dataverse.org/dataverse/myDataverseAlias"
export DATAVERSE_API_TOKEN="abcd-abcd-abcd-abcd-abcdefghijkl"
export DATAVERSE_TERMS_OF_USE="https://dataverse.org/best-practices/harvard-dataverse-general-terms-use"
export DATAVERSE_CUSTOM_REQUIRED_METADATA_URL="https://demo.dataverse.org/dataverse/customRequiredMetadataFields"

DATAVERSE_CUSTOM_REQUIRED_METADATA_URL points to a collection, on the same server and reachable with the same token, whose metadata blocks require alternativeURL, dsDescriptionDate, PSRI1 and PSRI2; it is used by the scenarios about metadata required by the collection. Without these variables, the PHPUnit tests that need them are skipped locally and fail in CI, and the Playwright suite does not start.

Unit and Integration Tests (PHPUnit)

Run from the root of the PKP Application directory:

php lib/pkp/lib/vendor/phpunit/phpunit/phpunit --configuration lib/pkp/tests/phpunit.xml plugins/generic/dataverse/tests

The configuration, submission, workflow and public page scenarios, including the metadata required by a collection, run against the application database, real files and the Dataverse collection above, without mocks.

End-to-end Tests (Playwright)

Only behaviour that exists solely in the browser is tested here: client-side validation of the settings form, the conditional fields and sections of the submission wizard, the research data upload modal, the review step, the Research data and Data statement panels of the workflow, the editorial decision steps, the research data shown to reviewers and the public page. The publishing and the acceptance scenarios publish a dataset in the test collection on every run, and Dataverse does not let a regular API token delete a published dataset, so use a collection meant for testing.

The application must be running with the reference test dataset (context publicknowledge, users dbarnes, eostrom and, in OJS, jjanssen) and the en locale. Inside the plugin directory:

npm install
npx playwright install chromium
npm run test:e2e

BASE_URL points to the application (default http://localhost:8000) and APP_ROOT to its root directory (default: three levels above the plugin). To run against OPS, set both, for example APP_ROOT=/path/to/ops BASE_URL=http://localhost:8001 npm run test:e2e.

Credits

This plugin was sponsored by the Scientific Electronic Library Online (SciELO) and developed by Lepidus Tecnologia.

The development of this plugin seeks to give continuity to the integration between OJS and Dataverse, done previously through the plugin for OJS 2.4.

License

This plugin is licensed under the GNU General Public License v3.0

Copyright (c) 2021-2025 Lepidus Tecnologia

Copyright (c) 2021-2025 SciELO