Link Search Menu Expand Document

Setup of NFDI-MatWerk Data Repository

Here is a description of how the NFDI-MatWerk Data Repository can be installed locally with basic settings. A prerequisite for this is an accessible database which needs to be created beforehand.

Check if all dependencies are installed

The NFDI-MatWerk Data Repository is based on the base-repo. Both git and java are needed for installing base-repo. The following sections illustrate how this can be checked.

Java

To check if java is installed, enter the code java -version in the command prompt and press enter. If java is already installed you will get the version number, else you will get an error message. We need atleast openjdk17. If it is an earlier version than openjdk17 or if java is not installed, please install openjdk17 by following the instruction on this website.

If you are working with a server with only command line interface, i.e, without a GUI, then java can be installed using the command sudo apt-get install openjdk-17-jre openjdk-17-jre-headless (for Ubuntu) followed by the enter key.

Make sure to check if the installation is successful by typing java -version on the command prompt and pressing enter.

Git

To check if git is installed, enter the code git -version in the command prompt and press enter. If git is already installed you will get the version number, else you will get an error message. If not already installed, you will have to install git.

For Windows machines, follow the instructions on https://git-scm.com/download/win.

For Mac, install git using Homebrew. If Homebrew is not already installed, install it by following the instructions at https://brew.sh. Then on the command prompt, type brew install git and press enter.

For Ubuntu and other Debian based distributions, run the following commands:

sudo apt update
sudo apt install git

Make sure to check if the installation is successful by typing git --version on the command prompt and pressing enter.

Create Directories for Saving the Data

Create a sub-directory called base-repo under a directory called data. You are free to choose other names, but make sure to use the right names and paths always.

For Unix based systems (eg. Mac, Linux) run the following commands from the command line:

mkdir -p /data/base-repo

For windows, the same can be done by removing the “-p” from the command above.

If you are using a server and have created an unprivileded user called “matwerk” in the beginning, you can use the following commands to give owner rights of the repository to this user and to switch to this user. For this, run the following commands from the command prompt:

chown matwerk /data/base-repo
su - matwerk

Clone base-repo

Next create a folder dem-services/base-repo to store the configuration files of the data repository. Thereafter, create another folder called software. If you have already installed a lab instance of the NFDI-MatWerk Metadata Repository (based on the MetaStore), then the folders dem-services and software would already have been created. In this case it would be enough to just make the new folder base-repo in the folder dem-services. Clone the base-repo in the folder software. For executing these steps, run the following commands from the command prompt for unix based systems and for windows run the same commands without using the “-p”:

mkdir -p dem-services/base-repo
mkdir software
cd software/
git clone https://github.com/kit-data-manager/base-repo

The output would look like this:

Cloning into 'base-repo'...
remote: Enumerating objects: 4433, done.
remote: Counting objects: 100% (1181/1181), done.
remote: Compressing objects: 100% (370/370), done.
remote: Total 4433 (delta 819), reused 1010 (delta 719), pack-reused 3252
Receiving objects: 100% (4433/4433), 1022.24 KiB | 1.52 MiB/s, done.
Resolving deltas: 100% (2027/2027), done.

Build base-repo

Now, change to the cloned base-repo directory and build the base-repo service using gradle.

cd base-repo/
./gradlew  build

The ouput would look similar to this:

Downloading https://services.gradle.org/distributions/gradle-7.6-bin.zip
.........................
Starting a Gradle Daemon (subsequent builds will be faster)
[...]
BUILD SUCCESSFUL in 36s
12 actionable tasks: 6 executed, 6 up-to-date

Configuring base-repo

Now copy the configutation files to dem-service/base-repo (make sure that the path is correct) by running the following commands:

cp ./build/libs/base-repo.jar ~/dem-services/base-repo/
cp ./config/application-default.properties  ~/dem-services/base-repo/application.properties

So base-repo is now built and the required configuration files are copied in place.

Next configure base-repo so that the PostGreSQL database can be used. Also the location where the data is going to be stored on the filesystem (repo.basepath) needs to be configured. For this, change the contents of the application.properties file which was copied in the previous step.

cd ~/dem-services/base-repo/

Edit the file application.properties in the folder base-repo to setup base-repo according to your requirements. In the example below, the nano editor is used. Any other commandline file editor can also be used.

nano application.properties

Go to the section Database and comment default spring settings by putting # before the lines to be commented. Then the file content should look as shown below:

###############################################################################
# Database
#
# Spring Datasource Settings, e.g database type, connection string and login credentials.
# Furthermore, the settings at the end allow to enable logging of SQL queries for
# debugging. By default, SQL logging should be disabled for performance reasons.
###############################################################################
#spring.datasource.driver-class-name: org.h2.Driver
#spring.datasource.url:  jdbc:h2:file:/tmp/base-repo/database;MODE=LEGACY;NON_KEYWORDS=VALUE
#spring.datasource.username: sa
#spring.datasource.password: sa
#spring.jpa.hibernate.ddl-auto: update

Uncomment production settings and provide correct username (spring.datasource.username) and password (spring.datasource.password) for spring boot. Check if all the other values are as shown below:

# For production, PostgreSQL is recommended
spring.datasource.platform: postgres
spring.jpa.database: POSTGRESQL
spring.jpa.database-platform: org.hibernate.dialect.PostgreSQLDialect
spring.datasource.driverClassName: org.postgresql.Driver
spring.datasource.url: jdbc:postgresql://localhost:5432/baserepo
spring.datasource.username: matwerk_db_user
spring.datasource.password: DB_USER_ADMIN_PASSWORD
spring.jpa.hibernate.ddl-auto: update

Next, change the base path where all uploaded data will be stored. The base path must be a URI pointing to a local folder.

repo.basepath:file://data/base-repo

Once all the configurations are made, the base-repo can be manually started by running the following command.

matwerk@matwerklab:~/dem-services/base-repo$ ./base-repo.jar

The output should then look like this:


  .   ____          _            __ _ _
/\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot ::                (v2.7.5)

2023-10-18 10:52:33.998  INFO 6567 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
.
.
.
Spring is running!

If you get the message that Spring is running, without any error messages, the installation seems to be fine.

Testing locally installed base-repo

To test the installation, open a new terminal window. The window in which we received the message “Spring is running” need to be left undisturbed. In the new terminal window, type curl http://localhost:8090 and press enter. The output should look as shown below.

{
  "_links" : {
    "dataResources" : {
      "href" : "http://localhost:8090/dataResources{?page,size,sort}",
      "templated" : true
    },
    "aMQPMessages" : {
      "href" : "http://localhost:8090/aMQPMessages{?page,size,sort}",
      "templated" : true
    },
    "contentInformations" : {
      "href" : "http://localhost:8090/contentInformations{?page,size,sort}",
      "templated" : true
    },
    "allIdentifierses" : {
      "href" : "http://localhost:8090/allIdentifierses{?page,size,sort}",
      "templated" : true
    },
    "profile" : {
      "href" : "http://localhost:8090/profile"
    }
  }
}%

Stopping the base-repo

To stop the base-repo, press Ctrl+C. Please note, this is the control(^) key and not the command key in Mac. It is recommended to stop the base-repo before you shut down your computer.

Configurating the base-repo to start on boot (for linux servers)

First stop the base-repo service by pressing Ctrl+C. To automatically start base-repo on boot, create a file base-repo.service in /etc/systemd/system folder:

  1. In the Terminal window, navigate to the /etc/systemd/system folder using the command cd /etc/systemd/system and press enter

  2. Using a commandline editor like nano, create the file base-repo.service. For this, type nano base-repo.service and press enter.

  3. Write the following content inside the editor and press Ctrl+O and press enter, again followed by Ctrl+X to close the editor and save the file. 
     [Unit]
 Description=base_repo
 After=syslog.target

 [Service]
 User=matwerk
 ExecStart=/home/matwerk/dem-services/base-repo/base-repo.jar --spring.config.location=/home/matwerk/dem-services/base-repo/application.properties
 SuccessExitStatus=143

 [Install]
 WantedBy=multi-user.target
  4. Finally, enable the script and start the service by typing the following commands in the command prompt: 
      systemctl enable base-repo.service
  systemctl start base-repo.service
  systemctl status base-repo.service