HLS doesn't work on many source files

[bladyjoker]

Hi there,
I've been going through the repo code base, generally trying to figure out the build system, setting up the dev environment (emacs, lsp/hls, nix) and reading through code. However, I kept running into issues while using HLS so I wanted to share my findings here just in case they are relevant and actual.

bladyjoker@morphism:~/Desktop/plutus-use-cases/mlabs$ git status
On branch staging
Your branch is up to date with 'origin/staging'.

nothing to commit, working tree clean
bladyjoker@morphism:~/Desktop/plutus-use-cases/mlabs$ nix develop

[nix-develop:~/Desktop/plutus-use-cases/mlabs]$ haskell-language-server-wrapper
Found "/home/bladyjoker/Desktop/plutus-use-cases/mlabs/hie.yaml" for "/home/bladyjoker/Desktop/plutus-use-cases/mlabs/a"
Run entered for haskell-language-server-wrapper(haskell-language-server-wrapper) Version 1.5.0.0, Git revision 311107eabbf0537e0c192b2c377d282505b4eff1 (dirty) x86_64 ghc-8.10.7
Current directory: /home/bladyjoker/Desktop/plutus-use-cases/mlabs
Operating system: linux
Arguments: []
Cradle directory: /home/bladyjoker/Desktop/plutus-use-cases/mlabs
Cradle type: Cabal

Tool versions found on the $PATH
cabal:		3.6.2.0
stack:		2.7.3
ghc:		8.10.4.20210212
...
Files that failed:
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/Setup.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/demo/Main.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/deploy-app/Main.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/governance-demo/Main.hs
2021-12-02 15:22:36.838571564 [ThreadId 3187] INFO hls:	finish: GenerateCore (took 0.00s)
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/lendex-demo/Main.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/nft-demo/Main.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/nft-marketplace/Main.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Demo/Contract/Mint.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Deploy/Governance.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Deploy/Nft.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Governance/Contract.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Governance/Contract/Server.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Governance/Contract/Emulator/Client.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Governance/Contract/Simulator/Handler.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Lending/Contract.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Lending/Contract/Server.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Lending/Contract/Utils.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Lending/Contract/Emulator/Client.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Lending/Contract/Simulator/Handler.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Api.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Contract.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Governance.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Contract/Aux.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Contract/BidAuction.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Contract/Buy.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Contract/CloseAuction.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Contract/Fees.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Contract/Init.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Contract/Mint.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Contract/OpenAuction.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Contract/Query.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Contract/SetPrice.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/PAB/MarketplaceContract.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/PAB/Run.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/PAB/Simulator.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Nft/Contract.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Nft/Contract/Server.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Nft/Contract/StateMachine.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Nft/Contract/Emulator/Client.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Main.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/Demo/Contract/Mint.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/Governance/Contract.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/Governance/Init.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/Lending/Contract.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/Lending/Init.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/NFT/Contract.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/NFT/Init.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/NFT/QuickCheck.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/NFT/Size.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/NFT/Trace.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/NFT/Script/Auction.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/NFT/Script/Dealing.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/NFT/Script/Main.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/NFT/Script/Minting.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/NFT/Script/Values.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/Nft/Contract.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/test/Test/Nft/Init.hs

Completed (44 files worked, 57 files failed)
haskell-language-server-wrapper: callProcess: /nix/store/1f5m9aq39by56hbky1pz7dd52f4ikam8-haskell-language-server-exe-haskell-language-server-1.3.0.0/bin/haskell-language-server (exit 57): failed

In summary the following types of errors occur...

The package mlabs-plutus-use-cases has no file target

Seems like HLS and Cabal can't find under which component the source file should be built.

2021-12-02 14:33:19.533002953 [ThreadId 618] INFO hls:	File:     /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Lending/Contract/Utils.hs
Hidden:   no
Range:    1:1-2:1
Source:   cradle
Severity: DsError
Message: 
  Failed to parse result of calling cabal

  cabal: Unknown target
  '/home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/Lending/Contract/Utils.hs'.
  The package mlabs-plutus-use-cases has no file target
  'src/Mlabs/Lending/Contract/Utils.hs'.

which I tried out manually with:

[nix-develop:~/Desktop/plutus-use-cases/mlabs]$ cabal build src/Mlabs/Lending/Contract/Utils.hs
cabal: Unknown target 'src/Mlabs/Lending/Contract/Utils.hs'.
The package mlabs-plutus-use-cases has no file target
'src/Mlabs/Lending/Contract/Utils.hs'.

The fix seems to be to update the hie.yaml file with appropriate path -> component mappings.

Program error: GHC Core to PLC plugin: E043:Error: Reference to a name which is not a local, a builtin, or an external INLINABLE function

2021-12-02 14:29:43.008452944 [ThreadId 2710] INFO hls:	File:     /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Governance/Validation.hs
Hidden:   no
Range:    1:1-2:1
Source:   compile
Severity: DsError
Message: 
  Program error: GHC Core to PLC plugin: E043:Error: Reference to a name which is not a local, a
  builtin, or an external INLINABLE function: Variable PlutusTx.Base..
  No unfolding
  Context: Compiling expr: PlutusTx.Base..
  Context: Compiling expr: PlutusTx.Base..
  @ (Mlabs.NFT.Governance.Types.GovAct
  -> Plutus.V1.Ledger.Contexts.ScriptContext -> GHC.Types.Bool)
  Context: Compiling expr: PlutusTx.Base..
  @ (Mlabs.NFT.Governance.Types.GovAct
  -> Plutus.V1.Ledger.Contexts.ScriptContext -> GHC.Types.Bool)
  @ Ledger.Typed.Scripts.MonetaryPolicies.WrappedMintingPolicyType
  Context: Compiling expr: PlutusTx.Base..
  @ (Mlabs.NFT.Governance.Types.GovAct
  -> Plutus.V1.Ledger.Contexts.ScriptContext -> GHC.Types.Bool)
  @ Ledger.Typed.Scripts.MonetaryPolicies.WrappedMintingPolicyType
  @ Mlabs.NFT.Types.NftAppInstance
  Context: Compiling expr: PlutusTx.Base..
  @ (Mlabs.NFT.Governance.Types.GovAct
  -> Plutus.V1.Ledger.Contexts.ScriptContext -> GHC.Types.Bool)
  @ Ledger.Typed.Scripts.MonetaryPolicies.WrappedMintingPolicyType
  @ Mlabs.NFT.Types.NftAppInstance
  (Ledger.Typed.Scripts.MonetaryPolicies.wrapMintingPolicy
  @ Mlabs.NFT.Governance.Types.GovAct
  Mlabs.NFT.Governance.Types.$fUnsafeFromDataGovAct)
  Context: Compiling expr: PlutusTx.Base..
  @ (Mlabs.NFT.Governance.Types.GovAct
  -> Plutus.V1.Ledger.Contexts.ScriptContext -> GHC.Types.Bool)
  @ Ledger.Typed.Scripts.MonetaryPolicies.WrappedMintingPolicyType
  @ Mlabs.NFT.Types.NftAppInstance
  (Ledger.Typed.Scripts.MonetaryPolicies.wrapMintingPolicy
  @ Mlabs.NFT.Governance.Types.GovAct
  Mlabs.NFT.Governance.Types.$fUnsafeFromDataGovAct)
  Mlabs.NFT.Governance.Validation.mkGovMintPolicy
  Context: Compiling expr at "main:Mlabs.NFT.Governance.Validation:(59,8)-(59,66)"

The fix seems to be simply add the -fplugin-opt PlutusTx.Plugin:defer-errors GHC flag, however I'm not sure whether it's safe to simply add it unconditionally.

GHC runtime linker: fatal error: I found a duplicate definition for symbol sqlite3_fts3_may_be_corrupt

During linkage this error is raised as the same symbol seems to be defined by different libs.

[nix-develop:~/Desktop/plutus-use-cases/mlabs]$ haskell-language-server-wrapper -d src/Mlabs/NFT/Api.hs

2021-12-02 14:50:54.97629283 [ThreadId 89] DEBUG hls:	hlint:getIdeas:file:NormalizedFilePath "/home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Api.hs"
GHC runtime linker: fatal error: I found a duplicate definition for symbol
   sqlite3_fts3_may_be_corrupt
whilst processing object file
   /nix/store/w43xg9ni6gk28ndl1mgzlinb2dxxpdci-persistent-sqlite-lib-persistent-sqlite-2.13.0.3/lib/x86_64-linux-ghc-8.10.4.20210212/persistent-sqlite-2.13.0.3-8rnIaulr0SMFWDeiU7TZN4/HSpersistent-sqlite-2.13.0.3-8rnIaulr0SMFWDeiU7TZN4.o
The symbol was previously defined in
   /nix/store/ck5zqxqir93dighfhr2rkxr09bwv7qsk-direct-sqlite-lib-direct-sqlite-2.3.26/lib/x86_64-linux-ghc-8.10.4.20210212/direct-sqlite-2.3.26-Gg0YVBPVrMk3LDxZxB2Bzf/HSdirect-sqlite-2.3.26-Gg0YVBPVrMk3LDxZxB2Bzf.o
This could be caused by:
   * Loading two different object files which export the same symbol
   * Specifying the same object file twice on the GHCi command line
   * An incorrect `package.conf' entry, causing some object to be
     loaded twice.
2021-12-02 14:50:57.280727164 [ThreadId 157] INFO hls:	File:     /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Api.hs
Hidden:   no
Range:    1:1-2:1
Source:   compiler
Severity: DsError
Message: 
  loadObj
  "/nix/store/w43xg9ni6gk28ndl1mgzlinb2dxxpdci-persistent-sqlite-lib-persistent-sqlite-2.13.0.3/lib/x86_64-linux-ghc-8.10.4.20210212/persistent-sqlite-2.13.0.3-8rnIaulr0SMFWDeiU7TZN4/HSpersistent-sqlite-2.13.0.3-8rnIaulr0SMFWDeiU7TZN4.o":
  failed
2021-12-02 14:50:57.280972999 [ThreadId 171] INFO hls:	finish: User TypeCheck (took 2.31s)
2021-12-02 14:50:57.281077322 [ThreadId 175] INFO hls:	finish: GetHie (took 0.00s)
2021-12-02 14:50:57.281280027 [ThreadId 177] INFO hls:	finish: GenerateCore (took 0.00s)
Files that failed:
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Api.hs

Completed (0 files worked, 1 file failed)
haskell-language-server-wrapper: callProcess: /nix/store/1f5m9aq39by56hbky1pz7dd52f4ikam8-haskell-language-server-exe-haskell-language-server-1.3.0.0/bin/haskell-language-server "-d" "src/Mlabs/NFT/Api.hs" (exit 1): failed

Not sure what the fix would be for this one, however, it seems to be affecting only this one file.

[bladyjoker]

After the fixes are applied this is the result of hls invocation. I'd push the branch here but I don't have access.

[nix-develop:~/Desktop/plutus-use-cases/mlabs]$ haskell-language-server-wrapper
Files that failed:
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/Setup.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/demo/Main.hs
 * /home/bladyjoker/Desktop/plutus-use-cases/mlabs/src/Mlabs/NFT/Api.hs

Completed (98 files worked, 3 files failed)

[zygomeb]

@bladyjoker can you create a PR from your fork into staging?

[bladyjoker]

Thanks @zygomeb the PR is #308

[peter-mlabs]

Hi Dražen, thanks for the fixes you've provided. I've been working with HLS for a while, and we recently migrated to nix flakes and thought I'd give it another shot. Ironically, I ended up at this issue after independently performing the first two fixes and searching for a solution to the third.

That said, I'd actually like to reopen this.

The

GHC runtime linker: fatal error: I found a duplicate definition for symbol sqlite3_fts3_may_be_corrupt

error is giving us issues over in LiqwidX, and causes HLS to fail on (essentially) the entire project.

I'm not 100% sure of the progeny of our nix setup, but I'd imagine that there's a common ancestor between this repo and LiqwidX, and given that this has been plaguing us for a while, I think it is worthwhile to track down the root cause and fix it where-ever it lies.

[bladyjoker]

Hi Peter, please do reopen, if you're working towards fixing HLS and have it seamlessly run in our nix/flake/* environments all the power to you! Let me know if I can help somehow.

[peter-mlabs]

Sounds good. I'm not seeing an option to re-open on github, so I may not have the correct permissions. But I will be sure to keep you (and this issue) updated if I find a solution :)

[bladyjoker]

I just hit this issue again

GHC runtime linker: fatal error: I found a duplicate definition for symbol
   sqlite3_fts3_may_be_corrupt
whilst processing object file
   /nix/store/w43xg9ni6gk28ndl1mgzlinb2dxxpdci-persistent-sqlite-lib-persistent-sqlite-2.13.0.3/lib/x86_64-linux-ghc-8.10.4.20210212/persistent-sqlite-2.13.0.3-8rnIaulr0SMFWDeiU7TZN4/HSpersistent-sqlite-2.13.0.3-8rnIaulr0SMFWDeiU7TZN4.o
The symbol was previously defined in
   /nix/store/ck5zqxqir93dighfhr2rkxr09bwv7qsk-direct-sqlite-lib-direct-sqlite-2.3.26/lib/x86_64-linux-ghc-8.10.4.20210212/direct-sqlite-2.3.26-Gg0YVBPVrMk3LDxZxB2Bzf/HSdirect-sqlite-2.3.26-Gg0YVBPVrMk3LDxZxB2Bzf.o
This could be caused by:
   * Loading two different object files which export the same symbol
   * Specifying the same object file twice on the GHCi command line
   * An incorrect `package.conf' entry, causing some object to be
     loaded twice.

[zygomeb]

Right, let me reopen this issue. I don't think anyone on our team uses HLS so it does slip under our radar

[AriFordsham]

Is this a duplicate of https://github.com/mlabs-haskell/liqwidx-contracts/issues/102, or is there more to it?

[AriFordsham]

It happens to me wheenever I use the nat quasiquoter.

Unless someone can reproduce elsewhere, I would suggest closing both these issues in favour of an issue on https://github.com/Liqwid-Labs/plutus-extra, at least until we get a bit more insight.

[peter-mlabs]

Is this a duplicate of mlabs-haskell/liqwidx-contracts#102, or is there more to it?

They are duplicates. I didn't think everyone had access to the private Liqwid repos, and stumbled across this issue when I was searching for solutions a few weeks back.

I was also able to trace it to the Nat quasiquoter, alongside the conflicting SQL libraries on the issue you linked.