This repository contains the code for the FRINK Query UI web app, which can be used to query the Theme 1 Proto-OKN knowledge graphs or the FRINK federated knowledge graph.
- Ensure you have an up-to-date version of Node.js installed on your system.
- Clone the repo and navigate to the directory.
- Install all dependencies using the command
npm install. - Create a
.env.localtext file in the root of the project directory and copy the environment variables listed under the "Environment Variables > Required" section below. - When finished, run the development server with
npm run dev. The script should automatically open a browser tab at http://localhost:5173. Any changes you make in the code and save will update the development preview.
Make sure the following variables are set when the site is built. For local development, create a .env.local to have variables automatically loaded by Vite.
VITE_GH_REPO=frink-okn/okn-registry
VITE_GH_SOURCES=/docs/registry/kgs.yaml
VITE_GH_EXAMPLES_DIRECTORY=/docs/registry/queries
These environment variables handle where the site will pull the dynamic data required for the source selector and examples panel. See Fetching Data from OKN registry for more information.
VITE_TANSTACK_BASE_URL: if set to a string, will add a global base url route to all paths. Use this if you're not deploying to a domain at the root route (/)
VITE_GH_TOKEN: All of the Github directory queries (for the examples) are done using the Github API. With no authentication, this endpoint is ratelimited to ~60 req/hr. If you supply your access token (instructions) in this environment variable it will be used instead. This can be useful for development.
- npm: package manager, bundled with Node.js runtime
- Vite: development server, bundling (esbuild + Rollup)
- React: reactive UI library
- Typescript: Javascript superset providing compile-time strong typing
- Tanstack Router: client-side page router
- Tanstack Query: query state manager
- Comunica: SPARQL query engine
- Yasqe: syntax-highlighted SPARQL editor component
- Joy UI: React-based UI component library
- AG Grid: React data table component
- react-resizable-panels: React draggable panels component
- valibot: runtime schema validator
- react-aria: React-based unstyled components for common UI patterns
The actual work of querying the SPARQL and TPF endpoints is mostly handled by Comunica, but is wrapped in a thin useComunicaQuery hook that assists in integrating it with React UI code (loading state, errors, etc). Refer to the Comunica documentation for a deeper understanding of its features and usage.
Additionally, there is a small React Context in /context/query.tsx that provides the values from the useComunicaQuery hook to any component that needs it in the app (both the query editor and results table need access to the same data). The context also adds a timer that is automatically started while the query is running and displayed in the results panel.
A lot of the configuration for this application is fetched at run-time when a user visits the website. This prevents the application from needing to re-built when some information about the knowledge graph changes (i.e. a new graph is added, or an example is modified). All of the metadata about the current state of the OKN graphs are documented in a structured format in the registry repo.
Of particular use in this application are the kgs.yaml and the queries/ directory.
This is an array of all the sources that is used to populate the SourceSelect.tsx dropdown. The code that parses and transforms this file is defined in data/sources.ts. In order for the source to be shown in the dropdown, it must have the following keys:
- title
- shortname
- sparql
- tpf
Furthermore, the sources are grouped together by type in the dropdown, which is hardcoded in a map defined in the sources.ts file. If a source is in the kgs.yaml file but not in the map, it will be listed under the "Other Graphs" group.
The source.ts file exports a fetchSources function. This function is called by the router in the routes/__root.tsx, ensuring the page doesn’t render until the sources have been fetched (this typically happens in less that 50ms). Once that data has been fetched, it can be referenced anywhere else in the app by calling the rootRouteApi.useLoaderData() hook provided by the router.
Important
If you are a maintainer looking to add or update the example queries, follow the step-by-step guide here.
This is a directory that contains arbitrary directories and .rq SPARQL files. It's used to store the sample SPARQL queries. Each .rq file contains a YAML frontmatter section at the start of the file, before the SPARQL code. It's formatted with #+ before each line.
#+ summary: This is an example query
#+ tags:
#+ - biobricks-ice
Every .rq file needs a string summary and an array of tags representing the source(s) to use (matching the shortname tag in kgs.yaml)
The data/examples.ts code contains a fetchExamples function that fetches the all of the .rq files recursively and generates a Javascript object that mirrors the file structure tree. The actual code that manages traversing the queries/ directory on Github's server is defined in data/github-utils.ts. The tree structure is built to support the ExampleTree.tsx component that displays a file tree UI.
Warning
The getFilesFromGithub function has a default maxDepth of 2. The more levels of nesting the queries/ directory contains, the longer the network request waterfall will take. It is recommended to keep queries/ directory fairly flat to keep the load times low.
Because the examples data fetching requires using the Github API once for each folder, it's possible for a user constantly refreshing the page to hit the API limit. To avoid this, the query state management library is set up to cache the return value from fetchExamples for 24hrs. Additionally, the localStorage adapter from @tanstack/query-sync-storage-persister is used as the backing store for the cache so that refreshes to the page pull the examples from localStorage if they haven't expired. For maintainers or developers testing the examples, a "Invalidate example queries cache" button can be found on the settings page in the application.
The above diagram gives a simplified view of the network requests made when the user loads the page. The yellow boxes are using the Github API to fetch the contents of the directory. Once files have been found, the raw.githubusercontent.com url can be used to directly fetch the contents of the file (blue boxes, these are not subject to the rate limit). Note the possibility of network waterfalls for deeply nested folders in queries/. In the UI, the text "Loading..." will be shown until the all the request have finished.
This client interface can be deployed with any web server that can deploy static resources. The static files can be generated using the npm run build command (Node.js is required to build), which will put them in the dist/ folder.
In order to avoid manually building the static files, there is a provided Dockerfile that builds the files and server them from an Nginx server. It is recommended to use Docker for reproducability and simplicity. To begin, ensure Docker is downloaded on your system (for development machines, Docker Desktop provides a nice GUI).
You could create a image using the following command:
docker build . \
-t frink-query-ui \
--build-arg VITE_GH_REPO="frink-okn/okn-registry" \
--build-arg VITE_GH_SOURCES="/docs/registry/kgs.yaml" \
--build-arg VITE_GH_EXAMPLES_DIRECTORY="/docs/registry/queries"
Note that the environment variables must be provided to the website to properly build. Once it has finished building, you can run it using:
docker run --name frink-query-ui -p 8080:8080 frink-query-ui
It can then be visited at localhost:8080. Please see the Docker documentation for more information about how to use Docker.
This project has several Github actions (found in the .github/workflows/ directory) that run automatically in response to various actions.
publish-docker-image.yml: upon a new release, builds a Docker image of the site and publishes it to the Github Container Registry at ghcr.io/frink-okn/frink-query-ui.
deploy.yaml: builds the website and deploys it to Github Pages anytime there is a push to the main branch. The site is viewable at https://frink-okn.github.io/frink-query-ui.
node.js.yaml: ensures the site builds without errors on each pull request to the main branch.
Utilizing the Docker image created from the publish-docker-image.yml workflow, the Helm chart located at /kubernetes can be used to automate the deployment of the query UI within RENCI's Sterling cluster. This deployment can be found at frink.apps.renci.org. Please see the Helm documentation to learn more about updating and deploying releases.