No loader is configured for ".glsl" files when using itk-vtk-viewer in react app with vite

[Tinanuaa]

Hi, I'm trying to use itk-vtk-viewer in my vite react typescript project, node version is v20.3.1, but I got errors like below:

✘ [ERROR] Could not resolve "../itkConfig.js"

    node_modules/itk-vtk-viewer/src/Rendering/VTKJS/Images/fuseImages.js:1:126:
      1 │ ...ComposeImageWorker from'./ComposeImage.worker.js';import itkConfig from'../itkConfig.js';export const fuseImages=async({imageAtScale,//could be array if Congl...
        ╵                                                                           ~~~~~~~~~~~~~~~~~

✘ [ERROR] Could not resolve "../itkConfig.js"

    node_modules/itk-vtk-viewer/src/Rendering/VTKJS/Images/ComposeImage.worker.js:1:625:
      1 │ ...useComponents}from'../../../IO/composeComponents';import itkConfig from'../itkConfig.js';const checkOverlap=(imageA,imageB)=>{const[vtkA,vtkB]=[imageA,imageB]...
        ╵                                                                           ~~~~~~~~~~~~~~~~~

✘ [ERROR] No loader is configured for ".glsl" files: node_modules/vtk.js/Sources/Rendering/OpenGL/glsl/vtkPolyDataVS.glsl

    node_modules/vtk.js/Sources/Rendering/OpenGL/ImageMapper/index.js:1:879:
      1 │ .../ImageProperty/Constants';import vtkPolyDataVS from'vtk.js/Sources/Rendering/OpenGL/glsl/vtkPolyDataVS.glsl';import vtkPolyDataFS from'vtk.js/Sources/Renderin...
        ╵                                                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

✘ [ERROR] No loader is configured for ".glsl" files: node_modules/vtk.js/Sources/Rendering/OpenGL/glsl/vtkPolyDataFS.glsl

    node_modules/vtk.js/Sources/Rendering/OpenGL/ImageMapper/index.js:1:962:
      1 │ .../glsl/vtkPolyDataVS.glsl';import vtkPolyDataFS from'vtk.js/Sources/Rendering/OpenGL/glsl/vtkPolyDataFS.glsl';import vtkReplacementShaderMapper from'vtk.js/Sou...
        ╵                                                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

✘ [ERROR] No loader is configured for ".glsl" files: node_modules/vtk.js/Sources/Rendering/OpenGL/glsl/vtkPolyData2DFS.glsl

    node_modules/vtk.js/Sources/Rendering/OpenGL/PolyDataMapper2D/index.js:2:282:
      2 │ ...es/Common/Core/Points';import vtkPolyData2DFS from'vtk.js/Sources/Rendering/OpenGL/glsl/vtkPolyData2DFS.glsl';import vtkPolyData2DVS from'vtk.js/Sources/Rende...
        ╵                                                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

✘ [ERROR] No loader is configured for ".glsl" files: node_modules/vtk.js/Sources/Rendering/OpenGL/glsl/vtkPolyData2DVS.glsl

    node_modules/vtk.js/Sources/Rendering/OpenGL/PolyDataMapper2D/index.js:2:369:
      2 │ .../vtkPolyData2DFS.glsl';import vtkPolyData2DVS from'vtk.js/Sources/Rendering/OpenGL/glsl/vtkPolyData2DVS.glsl';import vtkReplacementShaderMapper from'vtk.js/So...
        ╵                                                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

✘ [ERROR] No loader is configured for ".glsl" files: node_modules/vtk.js/Sources/Rendering/OpenGL/glsl/vtkVolumeFS.glsl

    node_modules/vtk.js/Sources/Rendering/OpenGL/VolumeMapper/index.js:2:1060:
      2 │ ...penGL/glsl/vtkVolumeVS.glsl';import vtkVolumeFS from'vtk.js/Sources/Rendering/OpenGL/glsl/vtkVolumeFS.glsl';import{registerOverride}from'vtk.js/Sources/Render...
        ╵                                                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

✘ [ERROR] No loader is configured for ".glsl" files: node_modules/vtk.js/Sources/Rendering/OpenGL/glsl/vtkVolumeVS.glsl

    node_modules/vtk.js/Sources/Rendering/OpenGL/VolumeMapper/index.js:2:981:
      2 │ ...Core/VolumeMapper/Constants';import vtkVolumeVS from'vtk.js/Sources/Rendering/OpenGL/glsl/vtkVolumeVS.glsl';import vtkVolumeFS from'vtk.js/Sources/Rendering/O...
        ╵                                                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The steps to reproduce this error is quite simple, I just

npm create vite@latest # project_name as vite-react-app, select react as frameowrk and use typescript
cd vite-react-app
npm install
npm install itk-vtk-viewer
npm run dev

My vite.config.js is like below

import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
import babel from 'vite-plugin-babel';
import glsl from 'vite-plugin-glsl';
// https://vitejs.dev/config/
export default defineConfig({
  
  plugins: [react(),
    glsl(),
    babel({
      babelConfig: {
        babelrc: false,
        configFile: false,
        compact:true,
        plugins: [
          [
            "@babel/plugin-proposal-decorators",
            { loose: true, version: "2022-03" },
          ],
        ],
      },
    }),],
})

my package.json is like

{
  "name": "vite-react-app",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "tsc && vite build",
    "lint": "eslint src --ext ts,tsx --report-unused-disable-directives --max-warnings 0",
    "preview": "vite preview"
  },
  "dependencies": {
    "@babel/plugin-proposal-decorators": "^7.22.7",
    "itk-vtk-viewer": "^14.43.0",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "vite-plugin-babel": "^1.1.3",
    "vite-plugin-glsl": "^1.1.2"
  },
  "devDependencies": {
    "@types/react": "^18.2.14",
    "@types/react-dom": "^18.2.6",
    "@typescript-eslint/eslint-plugin": "^5.61.0",
    "@typescript-eslint/parser": "^5.61.0",
    "@vitejs/plugin-react": "^4.0.1",
    "eslint": "^8.44.0",
    "eslint-plugin-react-hooks": "^4.6.0",
    "eslint-plugin-react-refresh": "^0.4.1",
    "typescript": "^5.0.2",
    "vite": "^4.4.0"
  }
}

May I know why the itkConfig.js file is not imported correctly in those two files? I notice itkConfig.js is three levels up those two files. I've tried to set up the alias in vite, but it doesn't work. And for the glsl loader, I added glsl plugins in the vite plugins, but still got the errors relating to the glsl files.

[Tinanuaa]

I tried to add alias in vite, but still got the itkConfig.js resolving error. so the steps are like below:

1. create tsconfig.paths.json

{
    "compilerOptions": {
      "paths": {
 
        "../itkConfig.js": ["../node_modules/itk-vtk-viewer/src/itkConfig.js"],
      }
    }
  }

2. add baseUrl and extends the above compilerOptions into tsconfig.json.

{
  "compilerOptions": {
    "target": "ES2020",
    "useDefineForClassFields": true,
    "lib": ["ES2020", "DOM", "DOM.Iterable"],
    "module": "ESNext",
    "skipLibCheck": true,

    /* Bundler mode */
    "moduleResolution": "bundler",
    "allowImportingTsExtensions": true,
    "resolveJsonModule": true,
    "isolatedModules": true,
    "noEmit": true,
    "jsx": "react-jsx",

    /* Linting */
    "strict": true,
    "noUnusedLocals": true,
    "noUnusedParameters": true,
    "noFallthroughCasesInSwitch": true,
    **"baseUrl": "."**
  },
  **"include": ["src","node_modules/itk-vtk-viewer" ],**
  "references": [{ "path": "./tsconfig.node.json" }],
  **"extends": "./tsconfig.paths.json"**
}

3. add resove.alias in vite.config.js

import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
import babel from 'vite-plugin-babel';
import glsl from 'vite-plugin-glsl';
import path from 'path'
const itkConfig = path.resolve(__dirname, 'node_modules', 'itk-vtk-viewer', 'src', 'itkConfig.js');

// https://vitejs.dev/config/
export default defineConfig({
  resolve: {
    alias: {
      "../itkConfig.js": itkConfig,
    },
  },
  plugins: [react(),
    glsl(),
    babel({
      babelConfig: {
        babelrc: false,
        configFile: false,
        compact:true,
        plugins: [
          [
            "@babel/plugin-proposal-decorators",
            { loose: true, version: "2022-03" },
          ],
        ],
      },
    }),],
})

[Tinanuaa]

After I add the optimizeDeps and viteStaticCopy to the vite.config.js as following (and I also changed the way I import itk-vtk-viewer to the way you said here,

import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
import babel from 'vite-plugin-babel';
import path from 'path';
import { viteStaticCopy } from 'vite-plugin-static-copy';
const itkConfig = path.resolve(__dirname, 'node_modules', 'itk-vtk-viewer', 'src', 'itkConfig.js');

// https://vitejs.dev/config/
export default defineConfig({
  optimizeDeps: {
    include: ['itk-vtk-viewer'],
  },
  resolve: {
    alias: {
      "../itkConfig.js": itkConfig,
    },
  },
  plugins: [react(),
    // glsl({include:"/vtk\.js[\/\\]Sources/"}),
    babel({
      babelConfig: {
        babelrc: false,
        configFile: false,
        compact:true,
        plugins: [
          [
            "@babel/plugin-proposal-decorators",
            { loose: true, version: "2022-03" },
          ],
        ],
      },
    }),
    viteStaticCopy({
      targets: [
        { src: 'node_modules/itk-vtk-viewer/dist/pipeline.worker.js', dest: './' },
        { src: 'node_modules/itk-vtk-viewer/dist/itk', dest: './' },
      ],
    }),
  ],
})

the glsl errors go away, but the webpage got console errors like below,

And I notice there is no pipeline.worker.js under node_modules/itk-vtk-viewer/dist/pipeline.worker.js. I changed to node_modules/itk-vtk-viewer/dist/itk/web-workers/min-bundles/pipeline.worker.js and still the same console errors. Does it mean the glsl files still not get compiled properly?

[Tinanuaa]

I put the code in github, feel free to play with it if you like https://github.com/Tinanuaa/vite-react-app

[PaulHax]

Simplest way to put itk-vtk-viewer in your app is to use the CDN. This project does that:
https://github.com/InsightSoftwareConsortium/itk-viewer-bootstrap-ui

[Tinanuaa]

I was able to set it up after I change the dest for viteStaticCopy plugin from"." to "./src" like below.

viteStaticCopy({
      targets: [
        { src: 'node_modules/itk-vtk-viewer/dist/itk/web-workers/min-bundles/pipeline.worker.js', dest: './src' },
        { src: 'node_modules/itk-vtk-viewer/dist/itk', dest: './src' },
      ],
    }),

But then I always got error like below:

I'm calling the createView inside App.tsx

import { useState } from 'react';
import reactLogo from './assets/react.svg';
import viteLogo from '/vite.svg';
import './App.css';
import 'itk-vtk-viewer';
import { Box, Typography, Grid, Divider  } from "@mui/material";
import Viewer from './Viewer';

function App() {

  const container =  document.querySelector('#container') as HTMLElement;
  const uiMachineOptions = { href: "https://cdn.jsdelivr.net/npm/itk-viewer-reference-ui-template@0.1.2/dist/referenceUIMachineOptions.js.es.js" }
  const image = new URL(
    "https://data.kitware.com/api/v1/file/564a65d58d777f7522dbfb61/download/data.nrrd",
    window.location.origin
  )  
  itkVtkViewer.createViewer(container,
  {
    image:image,
    rotate: false,
    // config: { uiMachineOptions },
  });

  return (
    <div id="container" ></div>
  )
}

export default App

I also tried to use React ref to get the container, but still doesn't work

import { useState, useRef, useEffect } from 'react';
import reactLogo from './assets/react.svg';
import viteLogo from '/vite.svg';
import './App.css';
import 'itk-vtk-viewer';
import { Box, Typography, Grid, Divider  } from "@mui/material";
import Viewer from './Viewer';

function App() {
  const liveViewerRef = useRef(null);
  const container =  document.querySelector('#container') as HTMLElement;
  const uiMachineOptions = { href: "https://cdn.jsdelivr.net/npm/itk-viewer-reference-ui-template@0.1.2/dist/referenceUIMachineOptions.js.es.js" }
  const image = new URL(
    "https://data.kitware.com/api/v1/file/564a65d58d777f7522dbfb61/download/data.nrrd",
    window.location.origin
  )  
  // itkVtkViewer.createViewer(container,
  // {
  //   image:image,
  //   rotate: false,
  //   // config: { uiMachineOptions },
  // });

  useEffect(() => {
 

    if(liveViewerRef && liveViewerRef.current)
    {
      // itkVtkViewer.createViewer(liveViewerRef.current,
      //   {
      //     image:image,
      //     rotate: true,
      //     // config: { uiMachineOptions },
      //   });
      itkVtkViewer.createViewer(liveViewerRef,
        {
          image:image,
          rotate: false,
          // config: { uiMachineOptions },
        });
    }
    

  }, []);

  return (
    <div id="container"  ref={liveViewerRef}></div>
  )
}

export default App

The error is like

So I want to embed the viewer inside the page, with some statistics information above the viewer, so the viewer doesn't take the whole page, is it possible?

[PaulHax]

Pass the plain HTMLElement?

itkVtkViewer.createViewer(liveViewerRef.current,

[Tinanuaa]

Thanks, it works now!

[Tinanuaa]

Sorry to disturb you again, but I notice sometimes the histogram doesn't show, like below,I'm just displaying the example image(https://data.kitware.com/api/v1/file/564a65d58d777f7522dbfb61/download/data.nrrd), and it works before, it just suddenly doesn't have the histogram

Not sure if there is any reason for this?

Source code is as below

import { useState, useRef, useEffect } from 'react';
import './App.css';
import 'itk-vtk-viewer';
import { Box, Typography, Grid, Divider,Stack  } from "@mui/material";
 
function App() {
  const liveViewerRef = useRef(null);
  const image =  new URL(
    "https://data.kitware.com/api/v1/file/564a65d58d777f7522dbfb61/download/data.nrrd"
  )  
  useEffect(() => {
 
    console.log('itk', !!itkVtkViewer, itkVtkViewer);

    if(liveViewerRef && liveViewerRef.current)
    {
      itkVtkViewer.createViewer(liveViewerRef.current,
        {
          image: image,
          rotate: false,
          // use2D: true,
        });
    }
  }, []);

  return (

    <Stack sx={{width:"100%"}}>
      {/* <Typography sx={{height:"10%"}}>Just a header</Typography> */}
      <Grid id="container"  ref={liveViewerRef}  ></Grid>
    </Stack>
    
  )
}

export default App

[PaulHax]

Strange. I see there are 2 three line expand/collapse menu buttons at the top left of the 2D GUI. That's not right ether.

useEffect getting called twice?

[Tinanuaa]

yes, I notice the two menus too, and from the console, the useEffect does get called twice, I guess it's caused by the RestricMode, I remove that then the useEffect only get called once, but still not showing the histogram.
I commented out these two lines

[PaulHax]

React has some docs on creating a boolean ref to make a useOnMounted like hook.

But regarding the histogram here is where to start debugging:

itk-vtk-viewer/src/UI/reference-ui/src/Images/createTransferFunctionWidget.js

Line 4 in 0a6ea9d

const createTransferFunctionWidget = (context, imagesUIGroup) => {

[Tinanuaa]

The histogram not showing problem hasn't occur since I asked you last time until today, so I look into the issue today and found that somehow the histogram was set to display:none, if I turn it off the histogram shows

Is it possible somehow somewhere in itk-vtk-viewer like ( itk-vtk-viewer/src/UI/reference-ui/src/applyGroupVisibility.js) changed the display of the histogram to None? I guess maybe some special combination will make this happen, cause most of the time the histogram can show, but just sometimes it choose to not display.