This document explains, step by step, how to set up the dot stat suite demo locally using the included containers, and how to fix the minimal configuration needed for SFS to correctly index dataflows.
Run:
git clone https://gitlab.com/sis-cc/.stat-suite/dotstatsuite-docker-compose
cd dotstatsuite-docker-compose
chmod -R a+x */*.sh */scripts/*.sh -v
cd demoStart the containers:
./start.sh localhostThe
start.shscript generates the configurations needed for containers to be created and started. However, it leaves some SFS/Solr files misconfigured; specifically:
dotstatsuite-docker-compose/demo/config/configs/demo/sfs/settings.jsonremains empty.dotstatsuite-docker-compose/demo/config/configs/tenants.jsonneeds specific entries in the datasources for the categories you want to index.
If these files are not corrected, SFS starts but cannot communicate with the NSI WS service and, therefore, does not index dataflows.
Edit dotstatsuite-docker-compose/demo/config/configs/demo/sfs/settings.json and add (or verify) the URLs to the NSI services of the demo. A minimal example:
{
"spaces": {
"demo-hpr": {
"url": "http://nsiws-demo-hpr:8080/rest",
"urlv3": "http://nsiws-demo-hpr:8080/rest/v2"
},
"demo-release": {
"url": "http://nsiws-demo-release:8080/rest",
"urlv3": "http://nsiws-demo-release:8080/rest/v2"
},
"demo-design": {
"url": "http://nsiws-demo-design:8080/rest",
"urlv3": "http://nsiws-demo-design:8080/rest/v2"
}
}
}Additionally, open dotstatsuite-docker-compose/demo/config/configs/tenants.json and make sure that each datasource has the dataqueries (categories) you need to index. If the categories are missing, SFS will not be able to map the dataflows.
Apply the changes by restarting the relevant containers:
docker restart config sfsInside the sfs container create the collection and load the schema:
docker exec -it sfs sh -lc "yarn dist:schema"Open the administration interface (by default):
- Web UI: http://localhost:7000
Demo credentials:
- User:
test-admin - Password:
admin
Steps in the UI:
- Access with the credentials above.
- Select the
demo-designspace. - Upload the data structure and data files (there are examples in
sdmx_samples/). - Categorize the dataflow (this is necessary for it to be indexed).
Once uploaded and categorized, run the indexing script from dotstatsuite-docker-compose/demo:
./scripts/indexation.index-dataflows.shTo check the status, use the service's report endpoint (example):
curl -s "http://localhost:3004/admin/report?api-key=secret" -H "x-tenant: demo" | jq .Expected output:
{
"indexed": 1,
"transpiled": 1,
"loaded": 1
}After indexing, the query interface should display the dataset:
- Query frontend: http://localhost:7001
- SFS does not communicate with NSI WS: check
dotstatsuite-docker-compose/demo/config/configs/demo/sfs/settings.jsonand that the hostnames (for examplensiws-demo-hpr) resolve from the containers (usedocker network inspectanddocker execto do internal pings or curls). tenants.jsonwithoutdataqueries: add the required categories or import the demo configuration included inconfig/configs/demo/.- Dataflow not indexed even after fixing configuration: confirm is categorized in the UI.
dotstatsuite-docker-compose/demo/config/configs/demo/sfs/settings.json-> configuration of spaces/NSI URLs..dotstatsuite-docker-compose/demo/config/configs/tenants.json-> tenants and datasources configuration.sdmx_samples/-> examples of data structures/data.dotstatsuite-docker-compose/demo/scripts/-> useful scripts (indexing, utilities).fixed_config_files-> examples of the fixed configuration files for sds.