Migrate DV-Hub and DV-Marketplace off NERC/MOC Infrastructure

[jp-tosca]

Migrate DV-Hub and DV-Marketplace off NERC/MOC Infrastructure

Overview

MOC (Massachusetts Open Cloud) is discontinuing the NERC services we currently rely on. As a result, we need to stand up new infrastructure and migrate the DV-Hub and DV-Marketplace services before the shutdown takes effect.

Background

Unfortunately, since we do not have the resources to maintain NERC while building the new platform, we will have to suspend our cloud services on July 31, 2026, for a few months.

Since we are moving our resources to building out the new environment, we suggest that you begin winding down your projects and move your workload off as soon as possible in case you run into any issues. Existing projects and workloads will not be migrated automatically.

Scope

DV-Hub and DV-Marketplace share the same architecture — both are Spring Boot applications requiring:

• Linux host
• HTTPS (valid TLS certificate)
• PostgreSQL 17
• Java 21
• Firewall rules and security hardening per DevOps standards

Participants

• Lead: @jp-tosca

Timeline

• Start: 2026-07-16
• Target completion: 2026-07-23
• Hard deadline (NERC shutdown): 2026-07-31

Tasks

1. DevOps evaluates infrastructure requirements for both services.
2. DevOps shares deployment requirements with @jp-tosca (JAR file format, database dump/restore format, environment variables, config files, etc.).
3. @jp-tosca prepares and hands off all required deployment artifacts, including:
   • Build artifacts: compiled JAR file(s) for DV-Hub and DV-Marketplace, tagged with build/version or commit hash; build config (pom.xml / build.gradle) confirming Java 21 compatibility; any startup script or systemd unit file currently in use.
   • Application configuration: application.properties / application.yml per environment (secrets redacted); list of required environment variables and their purpose; any third-party service integrations or external endpoints the apps depend on.
   • Secrets and credentials: DB password, API keys, OAuth client secrets, SMTP credentials, etc. — handed off via a secrets manager or encrypted channel, not included in the ticket.
   • Database: full pg_dump of the current production database (schema + data); confirmation of the current Postgres version; migration/versioning tool in use, if any (Flyway/Liquibase), and its scripts; any scheduled DB jobs, triggers, or extensions.
   • File storage / static assets: any uploaded files or local storage directories the apps read/write (paths and approximate size), plus a backup for transfer.
   • Operational details: logging configuration and destination; monitoring/alerting expectations; any cron jobs or scheduled tasks; current resource sizing (CPU/RAM/disk) as a baseline for the new servers.
4. DevOps provisions the new servers and notifies @jp-tosca upon completion.
5. DevOps deploys both applications and notifies @jp-tosca when ready for testing.
6. @jp-tosca validates the new deployments (functional testing, data integrity, performance).
7. DevOps cuts over DNS for hub.dataverse.org and marketplace.dataverse.org to the new servers.
8. DevOps documents the process to enable the team to replicate it for future migrations.

Definition of Done

• Both applications are running on the new infrastructure and pass validation testing.
• DNS cutover is complete and both domains resolve to the new servers over HTTPS.
• Migration runbook is documented and shared with the team.

[cmbz]

Status: July 2026

Next steps:

• DV-Marketplace
  • Send a notice to the list of registered instance contacts for the Dataverse Community (via Mailchimp) that we are retiring the server due to resource constraints, and let us know if they are using it
  • JP will back up the software and database, and any other articles/documentation/files/config (e.g., Apache config) used to set up the server
  • Inform MOC that we no longer need the service
  • They can then spin down the DV-Marketplace service on MOC
• DV-Hub
  • JP will back up the software and database, and any other articles/documentation/files/config (e.g., Apache config) used to set up the server
  • Leo will take these artifacts and see how difficult it is to run on demo
    • If it works, we can have the DV-Hub run on demo
    • Let MOC know that we have performed the migration and they can spin down the DV-Hub service
  • JP will create a frontend issue that allows the SPA to query either the DV-Hub or the local instance for metrics about the installation

[qqmyers]

FWIW: In gdcc/dataverse-external-vocab-support#64 I've just created a lighter-weight script to deploy from a github clone. See the deployment section of the readme. This is for external vocab rather than previewers but I think something similar could be put together there that would avoid a centralized service.