Subject Matter of the Contract

Implementation and operation of a semantic research data management system (RDMS) that meets the requirements set out below and can be operated on a self-hosted Linux server.

Specification of Services

General Requirements

• License: The research data management system (RDMS), as well as the libraries and frameworks described here, must be available under free and open-source licenses. A list of such licenses is available at https://opensource.org/licenses.
• Support: See also “LinkAhead Services.”
  • Commercial support at standard market rates must be available.
  • The offer includes a fixed allocation of 100 support or development hours, valid for one year.
• Documentation: Complete documentation for users, administrators, and developers, including short reference guides or “cheat sheets” for:
  • Creating and modifying models
  • Search queries
  • Scripting with Python
• System requirements:
  • The RDMS must provide the server service within a client-server architecture.
  • It must be possible to operate the system as a Docker container on a current Linux server, particularly Debian or Ubuntu.

Data Model

• Within the RDMS, data must be represented in the form of entries that can reference one another, creating a graph structure.
• The RDMS must allow the creation of a data schema defining which data classes or data types are managed by the system. Users must be able to adapt and extend the data schema without significant effort. This must make it possible to represent data in accordance with existing domain-specific ontologies.
• The data schema must support specialization and inheritance of data types. For example, the data type “LaserExperiment” could be defined as a specialization of the data type “Experiment.”
• Not all raw data needs to be integrated into the RDMS in a searchable form. Raw data may also be managed in the RDMS as referenced files.
• It must be possible to manage arbitrary files without restrictions to specific file types.
• The RDMS must allow properties to be assigned to entries.
  • These properties must be capable of consisting of a name (or “key”), a value, and a unit.
  • Value data types may be primitive data types, references to other entries, or lists of primitive data types or references. Supported primitive data types must include integer, floating point, Boolean, text, and datetime.
• The data schema must be extendable using YAML files.

File System Integration

• File systems on the host system must be capable of being integrated into the RDMS.
• Files stored on these file systems must be retrievable through the RDMS.
• When files are added, the RDMS must be capable of calculating and storing checksums.
• The RDMS must be capable of regularly verifying the checksums and paths of managed files.

Search

• The RDMS must provide a powerful search function that supports searches by class, file name, key-value combinations, and complex conditions, including reference and metadata filters.
• The search function must also support filtering results based on properties of referenced entries. For example: “Find laser experiments that reference a project with the property Grant=EU-Horizon.”
• Automatically generated metadata, including creation and modification dates and the responsible users, must be usable in searches.
• Search results must be reducible to selected attributes, comparable to the SQL statement “SELECT a, b.” These reduced search results must be downloadable in tabular form as a CSV file.

Versioning

• Every change to an entry must automatically create a new version while retaining the previous version.
• Versions must be uniquely identified and capable of being referenced directly.
• Previous versions must be retrievable at any time and restorable where necessary.

Rights and Identity Management

• The RDMS must include role-based access control. Users are assigned to roles, and each role defines specific permissions.
• Permissions must be definable at the individual-entry level.
• Access control must include at least:
  • Reading, modifying, and deleting entries
  • Retrieving and modifying permission information
  • Modifying the inheritance of entries
• The RDMS must support federated identity management through Keycloak or compatible federated identity services and must optionally permit anonymous access to data marked as public.

API

• The system must provide a network API through which data can be retrieved, created, modified, and deleted.
• The API must support the execution of search queries and the administration of user and role permissions.

Graphical User Interface

• The RDMS must provide a web-based graphical user interface, or GUI.
• The GUI must support viewing, creating, and editing entries; performing searches; managing users and roles; and modifying the data schema.
• The interface must be extensible so that frequently used searches and automated data-processing workflows can be integrated.
• The GUI must provide at least an English-language version and allow the use of custom design elements such as logos and colors.

Client Libraries

• Published Python client libraries must be available for the API and must provide the following functionality:
  • Retrieving data and files
  • Submitting search queries
  • Modifying and deleting entries
  • Managing permissions and roles

Automated Data Processing

• It must be possible to store Python scripts in the RDMS and launch them through the web interface in order to process RDMS entries or generate visualizations, such as time-series visualizations.

Automated Data Integration

• A published programming framework must be available for automatically detecting whether data is already present in the RDMS and for creating or updating entries accordingly.
• Security levels within this framework must determine whether only new data may be added or whether existing data may also be modified.
• Users must be able to define, through configuration files, how information such as the contents of spreadsheet files or file and folder names is translated into properties of RDMS entries.

Subject Matter of the Contract

The subject matter of the contract is the provision of services relating to the introduction, configuration, extension, maintenance, and operation of a semantic research data management system, or RDMS.

The services include, in particular, the following areas:

1. Customer-specific adaptations and configuration of an RDMS.
2. Software extensions, project-specific further development, and support services billed according to hours worked.
3. Software extensions, project-specific further development, and support services provided through a fixed allocation of service hours that may be called off as required.
4. Installation, updating, maintenance, and server administration on the contracting authority’s servers.
5. Consulting relating to data schemas, data integration, interfaces, and workflows.
6. Training and documentation for users, administrators, and developers.

Specification of Services

1. Customer-Specific Configuration and Adaptation

The contractor shall support the contracting authority in configuring an RDMS in accordance with the contracting authority’s functional, organizational, and technical requirements.

The services shall include, in particular, the adaptation of data schemas, role and permission concepts, search functions, workflows, import and export processes, and graphical user interfaces.

The configuration must be documented in a manner that allows the contracting authority to understand, review, and, where necessary, further develop it.

Configuration changes should, wherever possible, be implemented in a way that remains compatible with future updates and is easy to maintain.

2 and 3. Software Extensions and Further Development

The contractor shall develop project-specific extensions for the RDMS or related components.

Development must take account of the existing software architecture, documentation requirements, and maintainability. All developed source code must be transferred to the contracting authority and released under an open-source license. A list of permissible licenses is available at https://opensource.org/licenses.

The contractor shall provide support with testing, troubleshooting, and acceptance of the developed extensions.

The contractor shall make reasonable efforts to ensure that further developments and bug fixes are incorporated into the RDMS standard software, or upstream version.

2 and 3. Support

The contractor shall provide support services. These may include, in particular, error analysis, troubleshooting, consulting, configuration support, assistance with updates, assistance with data schemas and interfaces, and support relating to the use and administration of the RDMS.

4. Installation and Server Maintenance

The contractor shall install and configure the RDMS on a server provided by the contracting authority or in a server environment approved by the contracting authority.

Installation must be possible on current Linux systems, particularly Debian or Ubuntu.

The contractor shall support the contracting authority in setting up the required system services, containers, configuration files, storage locations, access permissions, and security parameters.

Subject to agreement, the contractor shall provide maintenance services. These shall include, in particular, updates, troubleshooting, checks of system availability, and reviews of log files. Maintenance services shall also include the planning, configuration, and monitoring of automated backups and support with recovery scenarios.

The contractor shall document the installation and maintenance activities in a manner that enables the contracting authority to understand the system’s operation and, where necessary, administer it independently.

5. Consulting on Data Schemas, Data Integration, Interfaces, and Workflows

The contractor shall advise the contracting authority on the design, creation, and further development of data schemas for the RDMS.

The consulting services shall include, in particular, the modeling of entities, properties, references, controlled vocabularies, inheritance structures, and project-specific or domain-specific ontologies.

The contractor shall support the development of automated data-integration processes, for example for importing information from files, folder structures, tables, laboratory notebooks, repositories, or other specialist systems.

The contractor shall support the definition of validation rules, quality controls, and troubleshooting processes for data integration.

The contractor shall assist with connecting the RDMS to the contracting authority’s existing systems or to external research-data infrastructures. These may include, in particular, repositories, electronic laboratory notebooks, institutional identity services, file systems, archival systems, analysis tools, or visualization components.

The contractor shall document interfaces, authentication mechanisms, data formats, import and export processes, and operational requirements.

6. Training and Documentation

The contractor shall offer training for different target groups, including end users with varying levels of technical knowledge, administrators, and developers.

Training may cover the use of the user interface, search, data modeling, permission management, data integration, API usage, administration, and troubleshooting.

The contractor shall provide suitable training materials, quick-reference guides, or project-specific documentation. The contractor shall enable the contracting authority, to the greatest extent reasonably possible, to use, administer, and further develop the system independently.

Requirements

The following requirements may, for example, be demonstrated by means of three comparable reference projects. The evidence should, in particular, demonstrate experience with semantic research data management systems, LinkAhead or comparable systems, server operation, software development, data integration, and support services.

Functional and Technical Experience

• The contractor must have demonstrable experience in the introduction, configuration, extension, maintenance, and operation of LinkAhead or comparable semantic RDMS solutions.
• The contractor must be capable of installing, configuring, adapting, and extending LinkAhead or a comparable RDMS for project-specific purposes and of supporting it during ongoing operation.
• The contractor must have demonstrable experience with the requirements of academic institutions, research projects, and public-sector contracting authorities.

Data Modeling, Integration, and Interfaces

• The contractor must have experience with graph-based data models, metadata modeling, ontologies, and controlled vocabularies.
• The contractor must be capable of jointly designing and implementing project-specific data models, entities, properties, references, inheritance structures, and validation rules together with the contracting authority.
• The contractor must have experience with automated data integration, data processing, and interface development using the RDMS to be deployed.

Server Operation, Maintenance, and Support

• The contractor must have experience operating and maintaining server-based applications on Linux systems.
• The contractor must have experience with container technologies, version control, API development, Python-based development, and automated operational or data-processing procedures.