Skip to content

ruizmaa/documentation-dotstatsuite-docker-compose

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 

Repository files navigation

Quick Guide: local demo of dot stat suite (docker-compose)

This document explains, step by step, how to set up the dot stat suite demo locally using the included containers, and how to fix the minimal configuration needed for SFS to correctly index dataflows.

1) Clone the repository and set permissions

Run:

git clone https://gitlab.com/sis-cc/.stat-suite/dotstatsuite-docker-compose
cd dotstatsuite-docker-compose
chmod -R a+x */*.sh */scripts/*.sh -v
cd demo

2) Start the demo

Start the containers:

./start.sh localhost

The start.sh script generates the configurations needed for containers to be created and started. However, it leaves some SFS/Solr files misconfigured; specifically:

  • dotstatsuite-docker-compose/demo/config/configs/demo/sfs/settings.json remains empty.
  • dotstatsuite-docker-compose/demo/config/configs/tenants.json needs specific entries in the datasources for the categories you want to index.

If these files are not corrected, SFS starts but cannot communicate with the NSI WS service and, therefore, does not index dataflows.

3) Fix the SFS configuration

Edit dotstatsuite-docker-compose/demo/config/configs/demo/sfs/settings.json and add (or verify) the URLs to the NSI services of the demo. A minimal example:

{
  "spaces": {
    "demo-hpr": {
      "url": "http://nsiws-demo-hpr:8080/rest",
      "urlv3": "http://nsiws-demo-hpr:8080/rest/v2"
    },
    "demo-release": {
      "url": "http://nsiws-demo-release:8080/rest",
      "urlv3": "http://nsiws-demo-release:8080/rest/v2"
    },
    "demo-design": {
      "url": "http://nsiws-demo-design:8080/rest",
      "urlv3": "http://nsiws-demo-design:8080/rest/v2"
    }
  }
}

Additionally, open dotstatsuite-docker-compose/demo/config/configs/tenants.json and make sure that each datasource has the dataqueries (categories) you need to index. If the categories are missing, SFS will not be able to map the dataflows.

4) Restart the configuration and SFS containers

Apply the changes by restarting the relevant containers:

docker restart config sfs

5) Create the Solr collection (SFS)

Inside the sfs container create the collection and load the schema:

docker exec -it sfs sh -lc "yarn dist:schema"

6) Load structures, data and categories from the UI

Open the administration interface (by default):

Demo credentials:

  • User: test-admin
  • Password: admin

Steps in the UI:

  1. Access with the credentials above.
  2. Select the demo-design space.
  3. Upload the data structure and data files (there are examples in sdmx_samples/).
  4. Categorize the dataflow (this is necessary for it to be indexed).

7) Index the dataflows

Once uploaded and categorized, run the indexing script from dotstatsuite-docker-compose/demo:

./scripts/indexation.index-dataflows.sh

8) Verify the indexing

To check the status, use the service's report endpoint (example):

curl -s "http://localhost:3004/admin/report?api-key=secret" -H "x-tenant: demo" | jq .

Expected output:

{
  "indexed": 1,
  "transpiled": 1,
  "loaded": 1
}

9) View the dataset

After indexing, the query interface should display the dataset:


Common issues and quick solutions

  • SFS does not communicate with NSI WS: check dotstatsuite-docker-compose/demo/config/configs/demo/sfs/settings.json and that the hostnames (for example nsiws-demo-hpr) resolve from the containers (use docker network inspect and docker exec to do internal pings or curls).
  • tenants.json without dataqueries: add the required categories or import the demo configuration included in config/configs/demo/.
  • Dataflow not indexed even after fixing configuration: confirm is categorized in the UI.

Relevant files

  • dotstatsuite-docker-compose/demo/config/configs/demo/sfs/settings.json -> configuration of spaces/NSI URLs..
  • dotstatsuite-docker-compose/demo/config/configs/tenants.json -> tenants and datasources configuration.
  • sdmx_samples/ -> examples of data structures/data.
  • dotstatsuite-docker-compose/demo/scripts/ -> useful scripts (indexing, utilities).
  • fixed_config_files -> examples of the fixed configuration files for sds.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors