Skip to content

UFAL/Add CCMM 1.1.0 OAI-PMH crosswalk (ccmm-xml metadataPrefix) - #1262

Open
milanmajchrak wants to merge 4 commits into
dtq-devfrom
ufal/oai-pmh-new-crosswalk
Open

UFAL/Add CCMM 1.1.0 OAI-PMH crosswalk (ccmm-xml metadataPrefix)#1262
milanmajchrak wants to merge 4 commits into
dtq-devfrom
ufal/oai-pmh-new-crosswalk

Conversation

@milanmajchrak

@milanmajchrak milanmajchrak commented Feb 27, 2026

Copy link
Copy Markdown
Collaborator

Problem description

Add CCMM 1.1.0 OAI-PMH crosswalk — exports metadata as CCMM 1.1.0 over OAI-PMH (metadataPrefix: ccmm-xml), required for NMD/NRP.

New ccmm.xsl, registered in xoai.xml (Default + openaire_data context), plus unit tests and test data.

Analysis

Run over all 2 628 live LINDAT records and validated against CCMM 1.1.0 with XSD 1.1:

before after
records that transform 2 203 2 628
ListRecords pages a harvester can fetch 0 / 32 32 / 32
XSD 1.1 valid 2 203 2 628
access_rights correct 746 (28 %) 2 628 (100 %)
publication_year = 0000 587 0
distinct IRIs returning HTTP 404 16 0
  1. Transform errordoc:field[@name='value'] is a sequence; the 425 records that carry two language variants of a field threw XPTY0004. OAIException turns into HTTP 500 for the whole page, so every one of the 32 pages failed and nothing was harvestable. Fixed with (…)[1].
  2. Vocabularies — every model.ccmm.cz/vocabulary/… IRI returns 404. Replaced with the registers CCMM names in its own specification (en/dsv.ttl scope notes): COAR resource types and access rights, vocabs.ccmm.cz AgentRole / TimeReference / DescriptionType, EU Publications Office for languages. dc.publisherAgentRole/Publisher (Distributor does not exist at that level).
  3. Access rights — now read from others/access-status, which DSpace computes in DefaultAccessStatusHelper. The old branches were dead (local.embargo.termslift: 0 records) or inverted (others/@restrictedAccess: 24 records, all of them open).
  4. Datesdc.date.issued = "0000" plus local.approximateDate.issued are handled the way ClarinDateService does in the UI: the local field replaces dc.date.issued, a range becomes a time_interval, and the original string is kept in date_information. Exactly one Issued reference plus a Created one, as the CCMM profile requires.
  5. Licence — no more invented IRI; falls back to others/cc, otherwise <license/> stays empty and the wording moves to terms_of_use/description.

Problems

CCMM requires at least one subject from the FRASCATI FORD vocabulary. LINDAT has no FORD field, so no record satisfies that yet — needs a submission-form change first.

Not in this PR: distribution from bundles/bitstreams, funding_reference from local.sponsor (616 records vs the 103 currently covered), location from dc.coverage.placeName.

Resolves: ufal#1145

Manual Testing (if applicable)

Copilot review

  • Requested review from Copilot

Copilot AI review requested due to automatic review settings February 27, 2026 12:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements a new OAI-PMH crosswalk for exporting DSpace metadata in CCMM (Czech Common Metadata Model) 1.1.0 format, required for NMD/NRP (National Repository of Gray Literature) compliance in the Czech Republic. The implementation follows the standard DSpace OAI-PMH crosswalk pattern used by other metadata formats in the repository.

Changes:

  • New XSL transformation stylesheet mapping DSpace internal metadata to CCMM 1.1.0 XML schema
  • Configuration updates to register the new crosswalk in both Default Context and openaire_data Context
  • Comprehensive test suite with 14 unit tests and dedicated test data file

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 14 comments.

File Description
dspace/config/crosswalks/oai/xoai.xml Registers ccmm format (metadataPrefix: ccmm-xml) in Default and openaire_data contexts
dspace/config/crosswalks/oai/metadataFormats/ccmm.xsl 633-line XSLT 2.0 stylesheet implementing DSpace to CCMM metadata transformation with fallback handling for required fields
dspace-oai/src/test/java/org/dspace/xoai/tests/stylesheets/CcmmXslTest.java Test class with 14 tests validating core CCMM elements (identifiers, creators, subjects, license, etc.)
dspace-oai/src/test/resources/xoai-ccmm-test.xml Test fixture simulating typical NMD/NRP dataset metadata with Czech academic repository data

</xsl:otherwise>
</xsl:choose>
</xsl:when>
<xsl:otherwise>http://unknown.repository</xsl:otherwise>

Copilot AI Feb 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hardcoded string "http://unknown.repository" should be extracted as a constant or defined at the top of the stylesheet for better maintainability.

Copilot generated this review using guidance from organization custom instructions.
Comment on lines +114 to +115
<xsl:when test="contains($uri, '/handle/')">
<xsl:value-of select="substring-before($uri, '/handle/')"/>

Copilot AI Feb 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic for extracting the repository base URL by finding '/handle/' in the URI and taking everything before it (lines 114-115) may not work correctly for all Handle server configurations. Some repositories might use different URL patterns (e.g., using a different path segment). Consider adding a comment explaining the assumptions about Handle URL structure, or making this more robust by also checking for 'hdl.handle.net' pattern.

Copilot uses AI. Check for mistakes.
Comment thread dspace/config/crosswalks/oai/metadataFormats/ccmm.xsl Outdated
Comment thread dspace/config/crosswalks/oai/metadataFormats/ccmm.xsl Outdated
Comment thread dspace/config/crosswalks/oai/metadataFormats/ccmm.xsl Outdated
Comment thread dspace/config/crosswalks/oai/metadataFormats/ccmm.xsl Outdated
Comment thread dspace/config/crosswalks/oai/metadataFormats/ccmm.xsl Outdated
Comment thread dspace/config/crosswalks/oai/metadataFormats/ccmm.xsl Outdated
Comment thread dspace/config/crosswalks/oai/metadataFormats/ccmm.xsl Outdated
Comment on lines +128 to +134
@Test
public void ccmmCanTransformBasicXoaiInput() throws Exception {
// Test with the default xoai-test1.xml (simpler data) to ensure crosswalk
// handles missing fields gracefully
String result = apply("ccmm.xsl").to(resource("xoai-test1.xml"));
assertThat(result, is(ccmm().withXPath("//ccmm:title", equalTo("Test Webpage"))));
}

Copilot AI Feb 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test suite lacks coverage for the fallback scenarios where required fields are missing. Consider adding tests that verify the behavior when title, identifiers, publication year, time references, and subjects are missing to ensure fallback values (like "Untitled", "9999", etc.) are correctly produced. This is particularly important for required CCMM fields.

Copilot uses AI. Check for mistakes.
- Extract hardcoded fallback strings as XSLT variables at top of stylesheet
  (Unknown Repository, http://unknown.repository, Untitled, unspecified, 9999)
- Add detailed comment explaining Handle URL extraction assumptions
- Add comment explaining publisher to Distributor role mapping rationale
- Fix time_reference fallback: use dc.date.accessioned instead of hardcoded
  9999-01-01 when issued/available dates are missing
- Improve FormatDate fallback: validate first 4 chars are digits before using
  as year, output empty ccmm:date element for invalid input
- Improve identifier fallback logic: check if Handle-pattern URIs were actually
  found rather than just checking for raw element presence
- Add 6 new fallback scenario tests (title, publication_year, subject,
  identifier, repository name, license) with dedicated minimal test fixture
- All 21 tests pass (20 CCMM + 1 QDC)

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 10 comments.

<ccmm:date><xsl:value-of select="concat($year, '-01-01')"/></ccmm:date>
</xsl:when>
<xsl:otherwise>
<ccmm:date/>

Copilot AI Feb 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FormatDate template creates an empty ccmm:date element at line 660 when the date string is invalid. This produces invalid XML output like 'ccmm:date/' or 'ccmm:date</ccmm:date>' which may not validate against the CCMM schema. Consider either omitting the element entirely or providing a fallback value, or documenting that this is intentional if the schema permits empty date elements.

Suggested change
<ccmm:date/>

Copilot uses AI. Check for mistakes.
Comment on lines +440 to +448
<xsl:variable name="lang" select="doc:metadata/doc:element[@name='dc']/doc:element[@name='language']/doc:element[@name='iso']/doc:element/doc:field[@name='value']"/>
<xsl:variable name="langAlt" select="doc:metadata/doc:element[@name='dc']/doc:element[@name='language']/doc:element/doc:field[@name='value']"/>
<xsl:choose>
<xsl:when test="$lang">
<ccmm:primary_language>
<ccmm:iri>
<xsl:value-of select="concat('https://iso639-3.sil.org/code/', $lang)"/>
</ccmm:iri>
<ccmm:label xml:lang="en"><xsl:value-of select="$lang"/></ccmm:label>

Copilot AI Feb 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PrimaryLanguage template could be clearer by explicitly selecting only the first language value with '[1]' predicate. Currently, line 440 selects all dc.language.iso values, and XSLT's atomization implicitly uses the first one at line 446. While functionally correct, explicitly selecting '[1]' at line 440 would make the intent clearer and match the pattern used in OtherLanguages at line 468 where position() > 1 explicitly excludes the first.

Copilot uses AI. Check for mistakes.
Comment on lines +117 to +136
<ccmm:iri>
<xsl:choose>
<xsl:when test="doc:metadata/doc:element[@name='dc']/doc:element[@name='identifier']/doc:element[@name='uri']/doc:element/doc:field[@name='value']">
<xsl:variable name="uri" select="doc:metadata/doc:element[@name='dc']/doc:element[@name='identifier']/doc:element[@name='uri']/doc:element/doc:field[@name='value'][1]"/>
<!--
Extract repository base URL from Handle URI.
Assumes DSpace-style URLs like https://repo.example.org/handle/123/456.
Also handles hdl.handle.net URIs as-is.
-->
<xsl:choose>
<xsl:when test="contains($uri, '/handle/')">
<xsl:value-of select="substring-before($uri, '/handle/')"/>
</xsl:when>
<xsl:otherwise>
<xsl:value-of select="$uri"/>
</xsl:otherwise>
</xsl:choose>
</xsl:when>
<xsl:otherwise><xsl:value-of select="$FALLBACK_REPOSITORY_URL"/></xsl:otherwise>
</xsl:choose>

Copilot AI Feb 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original_repository IRI extraction logic may produce incorrect results when dc.identifier.uri contains non-Handle URIs. Lines 127-132 will output the full URI (e.g., a DOI URL) as the repository IRI if it doesn't contain '/handle/'. Consider filtering to only Handle URIs (checking for 'hdl.handle.net' or '/handle/'), or falling back to FALLBACK_REPOSITORY_URL for non-Handle URIs, similar to the Identifiers template logic at lines 152-153.

Copilot uses AI. Check for mistakes.
Comment on lines +418 to +432
<xsl:variable name="dctype" select="doc:metadata/doc:element[@name='dc']/doc:element[@name='type']/doc:element/doc:field[@name='value']"/>
<xsl:if test="$dctype">
<ccmm:resource_type>
<ccmm:iri>
<xsl:choose>
<xsl:when test="$dctype='Dataset' or $dctype='corpus' or $dctype='lexicalConceptualResource' or $dctype='languageDescription'">https://model.ccmm.cz/vocabulary/datacite/resourceTypeGeneral/Dataset</xsl:when>
<xsl:when test="$dctype='Software' or $dctype='toolService'">https://model.ccmm.cz/vocabulary/datacite/resourceTypeGeneral/Software</xsl:when>
<xsl:when test="$dctype='Text' or $dctype='text'">https://model.ccmm.cz/vocabulary/datacite/resourceTypeGeneral/Text</xsl:when>
<xsl:when test="$dctype='Image' or $dctype='image'">https://model.ccmm.cz/vocabulary/datacite/resourceTypeGeneral/Image</xsl:when>
<xsl:when test="$dctype='Collection'">https://model.ccmm.cz/vocabulary/datacite/resourceTypeGeneral/Collection</xsl:when>
<xsl:otherwise>https://model.ccmm.cz/vocabulary/datacite/resourceTypeGeneral/Other</xsl:otherwise>
</xsl:choose>
</ccmm:iri>
<ccmm:label xml:lang="en"><xsl:value-of select="$dctype"/></ccmm:label>
</ccmm:resource_type>

Copilot AI Feb 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ResourceType template at line 418 selects all dc.type values into $dctype variable, but only the first value will be used (due to XSLT atomization in the when tests). If multiple dc.type values exist, subsequent values are silently ignored. For clarity, explicitly select only the first value with '[1]' predicate, similar to the pattern used elsewhere in the stylesheet.

Copilot uses AI. Check for mistakes.
Comment on lines +587 to +601
<ccmm:funding_reference>
<xsl:if test="count($parts) >= 5">
<ccmm:local_identifier><xsl:value-of select="$parts[5]"/></ccmm:local_identifier>
</xsl:if>
<ccmm:funder>
<ccmm:organization>
<ccmm:name>
<xsl:choose>
<xsl:when test="$parts[3]='EC'">European Commission</xsl:when>
<xsl:otherwise><xsl:value-of select="$parts[3]"/></xsl:otherwise>
</xsl:choose>
</ccmm:name>
</ccmm:organization>
</ccmm:funder>
</ccmm:funding_reference>

Copilot AI Feb 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FundingReferences template lacks error handling for malformed grant agreement URIs. If a dc.relation value starts with 'info:' but doesn't have the expected structure (e.g., fewer than 5 parts after tokenizing by '/'), the code attempts to access $parts[3] and potentially $parts[5], which could produce empty values or runtime errors. Consider adding validation to ensure the URI has the expected number of parts before accessing array indices.

Suggested change
<ccmm:funding_reference>
<xsl:if test="count($parts) >= 5">
<ccmm:local_identifier><xsl:value-of select="$parts[5]"/></ccmm:local_identifier>
</xsl:if>
<ccmm:funder>
<ccmm:organization>
<ccmm:name>
<xsl:choose>
<xsl:when test="$parts[3]='EC'">European Commission</xsl:when>
<xsl:otherwise><xsl:value-of select="$parts[3]"/></xsl:otherwise>
</xsl:choose>
</ccmm:name>
</ccmm:organization>
</ccmm:funder>
</ccmm:funding_reference>
<xsl:variable name="parts-count" select="count($parts)"/>
<xsl:if test="$parts-count &gt;= 3">
<ccmm:funding_reference>
<xsl:if test="$parts-count &gt;= 5">
<ccmm:local_identifier><xsl:value-of select="$parts[5]"/></ccmm:local_identifier>
</xsl:if>
<ccmm:funder>
<ccmm:organization>
<ccmm:name>
<xsl:choose>
<xsl:when test="$parts[3]='EC'">European Commission</xsl:when>
<xsl:otherwise><xsl:value-of select="$parts[3]"/></xsl:otherwise>
</xsl:choose>
</ccmm:name>
</ccmm:organization>
</ccmm:funder>
</ccmm:funding_reference>
</xsl:if>

Copilot uses AI. Check for mistakes.
Comment on lines +579 to +628
<!-- ============================================================ -->
<!-- funding_reference (optional, unbounded) -->
<!-- ============================================================ -->
<xsl:template name="FundingReferences">
<!-- dc.relation with info:eu-repo/grantAgreement pattern -->
<xsl:for-each select="doc:metadata/doc:element[@name='dc']/doc:element[@name='relation']/doc:element/doc:field[@name='value']">
<xsl:if test="starts-with(., 'info:')">
<xsl:variable name="parts" select="tokenize(., '/')"/>
<ccmm:funding_reference>
<xsl:if test="count($parts) >= 5">
<ccmm:local_identifier><xsl:value-of select="$parts[5]"/></ccmm:local_identifier>
</xsl:if>
<ccmm:funder>
<ccmm:organization>
<ccmm:name>
<xsl:choose>
<xsl:when test="$parts[3]='EC'">European Commission</xsl:when>
<xsl:otherwise><xsl:value-of select="$parts[3]"/></xsl:otherwise>
</xsl:choose>
</ccmm:name>
</ccmm:organization>
</ccmm:funder>
</ccmm:funding_reference>
</xsl:if>
</xsl:for-each>
</xsl:template>

<!-- ============================================================ -->
<!-- related_resource (optional, unbounded) -->
<!-- ============================================================ -->
<xsl:template name="RelatedResources">
<!-- dc.relation.uri -->
<xsl:for-each select="doc:metadata/doc:element[@name='dc']/doc:element[@name='relation']/doc:element[@name='uri']/doc:element/doc:field[@name='value']">
<ccmm:related_resource>
<ccmm:identifier>
<ccmm:value><xsl:value-of select="."/></ccmm:value>
<ccmm:scheme>
<ccmm:iri>https://www.w3.org/ns/iana/uri-schemes</ccmm:iri>
<ccmm:label xml:lang="en">URI</ccmm:label>
</ccmm:scheme>
</ccmm:identifier>
</ccmm:related_resource>
</xsl:for-each>
<!-- dc.relation.ispartof -->
<xsl:for-each select="doc:metadata/doc:element[@name='dc']/doc:element[@name='relation']/doc:element[@name='ispartof']/doc:element/doc:field[@name='value']">
<ccmm:related_resource>
<ccmm:title><xsl:value-of select="."/></ccmm:title>
</ccmm:related_resource>
</xsl:for-each>
</xsl:template>

Copilot AI Feb 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FundingReferences and RelatedResources templates lack test coverage. No tests verify the parsing of grant agreement URIs (info:eu-repo/grantAgreement format) or the handling of dc.relation.uri and dc.relation.ispartof fields. Add test cases with sample funding data and related resource data to ensure these mappings work correctly.

Copilot uses AI. Check for mistakes.
Comment on lines +481 to +499
<xsl:template name="TermsOfUse">
<ccmm:terms_of_use>
<!-- access_rights -->
<ccmm:access_rights>
<xsl:choose>
<xsl:when test="doc:metadata/doc:element[@name='local']/doc:element[@name='embargo']/doc:element[@name='termslift']/doc:element/doc:field[@name='value']">
<ccmm:iri>http://purl.org/coar/access_right/c_f1cf</ccmm:iri>
<ccmm:label xml:lang="en">embargoed access</ccmm:label>
</xsl:when>
<xsl:when test="doc:metadata/doc:element[@name='others']/doc:field[@name='restrictedAccess']/text()='true'">
<ccmm:iri>http://purl.org/coar/access_right/c_16ec</ccmm:iri>
<ccmm:label xml:lang="en">restricted access</ccmm:label>
</xsl:when>
<xsl:otherwise>
<ccmm:iri>http://purl.org/coar/access_right/c_abf2</ccmm:iri>
<ccmm:label xml:lang="en">open access</ccmm:label>
</xsl:otherwise>
</xsl:choose>
</ccmm:access_rights>

Copilot AI Feb 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The TermsOfUse template handles three access rights scenarios (embargoed, restricted, open), but only the default "open access" case is tested. Add test cases with local.embargo.termslift and others.restrictedAccess fields to verify embargoed and restricted access mappings produce correct COAR access rights IRIs.

Copilot uses AI. Check for mistakes.
Comment on lines +633 to +665
<xsl:template name="FormatDate">
<xsl:param name="dateStr"/>
<xsl:choose>
<!-- Full ISO dateTime: 2024-01-15T10:30:00Z -->
<xsl:when test="contains($dateStr, 'T')">
<ccmm:date_time><xsl:value-of select="$dateStr"/></ccmm:date_time>
</xsl:when>
<!-- Full date: 2024-01-15 -->
<xsl:when test="string-length($dateStr) >= 10 and substring($dateStr, 5, 1) = '-' and substring($dateStr, 8, 1) = '-'">
<ccmm:date><xsl:value-of select="substring($dateStr, 1, 10)"/></ccmm:date>
</xsl:when>
<!-- Year-month: 2024-01 -->
<xsl:when test="string-length($dateStr) >= 7 and substring($dateStr, 5, 1) = '-'">
<ccmm:date><xsl:value-of select="concat($dateStr, '-01')"/></ccmm:date>
</xsl:when>
<!-- Year only: 2024 -->
<xsl:when test="string-length($dateStr) = 4">
<ccmm:date><xsl:value-of select="concat($dateStr, '-01-01')"/></ccmm:date>
</xsl:when>
<!-- Fallback: attempt to derive a date only if the first 4 chars form a valid year -->
<xsl:otherwise>
<xsl:variable name="year" select="substring($dateStr, 1, 4)"/>
<xsl:choose>
<xsl:when test="string-length($dateStr) &gt;= 4 and not(translate($year, '0123456789', ''))">
<ccmm:date><xsl:value-of select="concat($year, '-01-01')"/></ccmm:date>
</xsl:when>
<xsl:otherwise>
<ccmm:date/>
</xsl:otherwise>
</xsl:choose>
</xsl:otherwise>
</xsl:choose>
</xsl:template>

Copilot AI Feb 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FormatDate template handles multiple date format variations (datetime with T, full date YYYY-MM-DD, year-month YYYY-MM, year-only YYYY, and malformed dates), but there are no tests covering these different formats. Add tests to verify that dates in different formats are correctly transformed, especially edge cases like year-only or year-month dates that need padding with '-01' or '-01-01'.

Copilot uses AI. Check for mistakes.
Comment on lines +247 to +252
<Format id="ccmm">
<Prefix>ccmm-xml</Prefix>
<XSLT>metadataFormats/ccmm.xsl</XSLT>
<Namespace>https://schema.ccmm.cz/research-data/1.1</Namespace>
<SchemaLocation>https://techlib.github.io/CCMM/dataset/schema.xsd</SchemaLocation>
</Format>

Copilot AI Feb 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inconsistent indentation for the Format element. The CCMM format definition uses 8-space indentation while the preceding ELG format uses 4-space indentation. For consistency with surrounding Format definitions (lines 217-235), use 4-space indentation for the Format element and 8-space indentation for its child elements.

Suggested change
<Format id="ccmm">
<Prefix>ccmm-xml</Prefix>
<XSLT>metadataFormats/ccmm.xsl</XSLT>
<Namespace>https://schema.ccmm.cz/research-data/1.1</Namespace>
<SchemaLocation>https://techlib.github.io/CCMM/dataset/schema.xsd</SchemaLocation>
</Format>
<Format id="ccmm">
<Prefix>ccmm-xml</Prefix>
<XSLT>metadataFormats/ccmm.xsl</XSLT>
<Namespace>https://schema.ccmm.cz/research-data/1.1</Namespace>
<SchemaLocation>https://techlib.github.io/CCMM/dataset/schema.xsd</SchemaLocation>
</Format>

Copilot uses AI. Check for mistakes.
Comment on lines +345 to +412
<xsl:template name="TimeReferences">
<!-- dc.date.issued -> Issued -->
<xsl:for-each select="doc:metadata/doc:element[@name='dc']/doc:element[@name='date']/doc:element[@name='issued']/doc:element/doc:field[@name='value']">
<ccmm:time_reference>
<ccmm:temporal_representation>
<ccmm:time_instant>
<xsl:call-template name="FormatDate">
<xsl:with-param name="dateStr" select="."/>
</xsl:call-template>
</ccmm:time_instant>
</ccmm:temporal_representation>
<ccmm:date_type>
<ccmm:iri>https://model.ccmm.cz/vocabulary/datacite/dateType/Issued</ccmm:iri>
<ccmm:label xml:lang="en">Issued</ccmm:label>
</ccmm:date_type>
</ccmm:time_reference>
</xsl:for-each>
<!-- dc.date.accessioned -> Accepted -->
<xsl:for-each select="doc:metadata/doc:element[@name='dc']/doc:element[@name='date']/doc:element[@name='accessioned']/doc:element/doc:field[@name='value']">
<ccmm:time_reference>
<ccmm:temporal_representation>
<ccmm:time_instant>
<xsl:call-template name="FormatDate">
<xsl:with-param name="dateStr" select="."/>
</xsl:call-template>
</ccmm:time_instant>
</ccmm:temporal_representation>
<ccmm:date_type>
<ccmm:iri>https://model.ccmm.cz/vocabulary/datacite/dateType/Accepted</ccmm:iri>
<ccmm:label xml:lang="en">Accepted</ccmm:label>
</ccmm:date_type>
</ccmm:time_reference>
</xsl:for-each>
<!-- dc.date.available -> Available -->
<xsl:for-each select="doc:metadata/doc:element[@name='dc']/doc:element[@name='date']/doc:element[@name='available']/doc:element/doc:field[@name='value']">
<ccmm:time_reference>
<ccmm:temporal_representation>
<ccmm:time_instant>
<xsl:call-template name="FormatDate">
<xsl:with-param name="dateStr" select="."/>
</xsl:call-template>
</ccmm:time_instant>
</ccmm:temporal_representation>
<ccmm:date_type>
<ccmm:iri>https://model.ccmm.cz/vocabulary/datacite/dateType/Available</ccmm:iri>
<ccmm:label xml:lang="en">Available</ccmm:label>
</ccmm:date_type>
</ccmm:time_reference>
</xsl:for-each>
<!-- Fallback: if no issued/available dates, create a minimal time_reference from accessioned -->
<xsl:if test="not(doc:metadata/doc:element[@name='dc']/doc:element[@name='date']/doc:element[@name='issued']/doc:element/doc:field[@name='value']) and not(doc:metadata/doc:element[@name='dc']/doc:element[@name='date']/doc:element[@name='available']/doc:element/doc:field[@name='value']) and doc:metadata/doc:element[@name='dc']/doc:element[@name='date']/doc:element[@name='accessioned']/doc:element/doc:field[@name='value']">
<xsl:variable name="accessionedDate"
select="doc:metadata/doc:element[@name='dc']/doc:element[@name='date']/doc:element[@name='accessioned']/doc:element/doc:field[@name='value'][1]"/>
<ccmm:time_reference>
<ccmm:temporal_representation>
<ccmm:time_instant>
<xsl:call-template name="FormatDate">
<xsl:with-param name="dateStr" select="$accessionedDate"/>
</xsl:call-template>
</ccmm:time_instant>
</ccmm:temporal_representation>
<ccmm:date_type>
<ccmm:iri>https://model.ccmm.cz/vocabulary/datacite/dateType/Issued</ccmm:iri>
<ccmm:label xml:lang="en">Issued</ccmm:label>
</ccmm:date_type>
</ccmm:time_reference>
</xsl:if>
</xsl:template>

Copilot AI Feb 27, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No test coverage for the time_reference element, which is a required CCMM field. The TimeReferences template handles multiple date types (Issued, Accepted, Available) and includes fallback logic, but none of these scenarios are tested. Add tests to verify that different date fields produce correct time_reference elements with appropriate date_type values and temporal_representation formatting.

Copilot uses AI. Check for mistakes.
Verified against all 2628 live LINDAT records; every output validates
against CCMM 1.1.0 in XSD 1.1 (0 invalid).

- fix XPTY0004: doc:field[@name='value'] returns a sequence, and records
  with two language variants of a field passed it to concat()/contains().
  425 of 2628 records aborted; the resulting OAIException becomes HTTP 500
  for the whole ListRecords page, so all 32 pages failed and nothing was
  harvestable
- replace the model.ccmm.cz IRIs, which all return 404, with the registers
  CCMM 1.1.0 names in its own specification (en/dsv.ttl scope notes):
  COAR resource types and access rights, vocabs.ccmm.cz AgentRole /
  TimeReference / DescriptionType, EU Publications Office for languages
- dc.publisher now maps to AgentRole/Publisher; Distributor does not exist
  at that level of the codelist
- derive access_rights from others/access-status, which DSpace computes in
  DefaultAccessStatusHelper from the actual READ policies. The previous
  branches were dead (local.embargo.termslift occurs 0 times) or inverted
  (others/@restrictedaccess is set on 24 records, all of them open access).
  Correct for 746 of 2628 records before, 2628 of 2628 now
- handle the CLARIN "0000" unknown-date convention and
  local.approximateDate.issued the way ClarinDateService does in the UI:
  the local field replaces dc.date.issued, a range becomes a time_interval
  and the original string is kept verbatim in ccmm:date_information
- emit exactly one Issued time reference plus a Created one, as the CCMM
  profile requires; publication_year and the Issued year now always agree
- stop emitting an invented licence IRI; fall back to others/cc and
  otherwise leave ccmm:license empty and move the wording to
  terms_of_use/description
- the minimal test fixture used to produce schema-invalid output (no
  time_reference at all); it no longer can

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 9 comments.

Suppressed comments (3)

dspace/config/crosswalks/oai/metadataFormats/ccmm.xsl:174

  • original_repository is populated from the item identifier, so the common hdl.handle.net/... form makes every record identify its item resolver as the repository. XOAI already supplies the actual repository URL at repository/url (ItemUtils.java:347); use that value instead.
                        <xsl:when test="doc:metadata/doc:element[@name='dc']/doc:element[@name='identifier']/doc:element[@name='uri']/doc:element/doc:field[@name='value']">
                            <xsl:variable name="uri" select="(doc:metadata/doc:element[@name='dc']/doc:element[@name='identifier']/doc:element[@name='uri']/doc:element/doc:field[@name='value'])[1]"/>

dspace-oai/src/test/java/org/dspace/xoai/tests/stylesheets/CcmmXslTest.java:125

  • This assertion exceeds the enforced 120-character line limit (checkstyle.xml:47-52). Wrap the expected organization name.
            equalTo("Charles University, Faculty of Mathematics and Physics, Institute of Formal and Applied Linguistics"))));

dspace-oai/src/test/java/org/dspace/xoai/tests/stylesheets/CcmmXslTest.java:182

  • This XPath exceeds the repository's enforced 120-character Java line limit (checkstyle.xml:47-52). Split it into concatenated segments.
            "//ccmm:dataset/ccmm:metadata_identification/ccmm:qualified_relation/ccmm:relation/ccmm:organization/ccmm:name",

<xsl:template name="PrimaryTemporalRepresentation">
<ccmm:temporal_representation>
<xsl:choose>
<xsl:when test="count($approxYears) &gt; 0 and $approxMin != $approxMax">
<!-- dc.title.alternative mapped to alternate_title -->
<xsl:for-each select="doc:metadata/doc:element[@name='dc']/doc:element[@name='title']/doc:element[@name='alternative']/doc:element/doc:field[@name='value']">
<ccmm:alternate_title>
<ccmm:title xml:lang="en">
</xsl:choose>
</ccmm:license>
<xsl:for-each select="doc:metadata/doc:element[@name='dc']/doc:element[@name='rights']/doc:element/doc:field[@name='value']">
<ccmm:description xml:lang="en"><xsl:value-of select="."/></ccmm:description>
<!-- dc.subject -->
<xsl:for-each select="doc:metadata/doc:element[@name='dc']/doc:element[@name='subject']/doc:element/doc:field[@name='value']">
<ccmm:subject>
<ccmm:title xml:lang="en"><xsl:value-of select="."/></ccmm:title>
<!-- dc.subject.* (nested qualifiers) -->
<xsl:for-each select="doc:metadata/doc:element[@name='dc']/doc:element[@name='subject']/doc:element/doc:element/doc:field[@name='value']">
<ccmm:subject>
<ccmm:title xml:lang="en"><xsl:value-of select="."/></ccmm:title>
<!-- dc.description (abstract) -->
<xsl:for-each select="doc:metadata/doc:element[@name='dc']/doc:element[@name='description']/doc:element[@name='abstract']/doc:element/doc:field[@name='value']">
<ccmm:description>
<ccmm:description_text xml:lang="en"><xsl:value-of select="."/></ccmm:description_text>
<!-- dc.description (general, non-qualified) -->
<xsl:for-each select="doc:metadata/doc:element[@name='dc']/doc:element[@name='description']/doc:element/doc:field[@name='value']">
<ccmm:description>
<ccmm:description_text xml:lang="en"><xsl:value-of select="."/></ccmm:description_text>
Comment on lines +30 to +31
String result = apply("ccmm.xsl").to(resource("xoai-ccmm-test.xml"));
assertThat(result, is(ccmm().withXPath("//ccmm:title", equalTo("Czech NLP Dataset v2.0"))));
public void ccmmPublisherUsesPublisherRole() throws Exception {
String result = apply("ccmm.xsl").to(resource("xoai-ccmm-test.xml"));
assertThat(result, is(ccmm().withXPath(
"//ccmm:dataset/ccmm:qualified_relation[ccmm:role/ccmm:label='Publisher']/ccmm:relation/ccmm:organization/ccmm:name",
- original_repository/iri named the item itself on every one of the 2628
  records: it was derived with substring-before(uri, '/handle/'), which
  does not match LINDAT's hdl.handle.net URIs, so the otherwise branch
  emitted the whole item URI. XOAI already carries repository/@url; use it.
- xml:lang was hardcoded "en" on subjects, alternate titles, descriptions
  and the rights wording. XOAI nests every value inside an element named
  after its language qualifier, so take the tag from there and fall back
  to "en" only for the unqualified "none" wrapper.

Both covered by new tests. Re-checked over all 2628 live LINDAT records:
still 2628/2628 valid against CCMM 1.1.0 in XSD 1.1.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

export new DataCite/CCMM metadata over OAI-PMH

2 participants