sonar-devon4j-plugin makes SonarQube server fail to start

[hohwille]

Caused by: java.lang.IllegalStateException: Rule with key 'checkstyle:com.puppycrawl.tools.checkstyle.checks.TranslationCheck' not found
        at com.google.common.base.Preconditions.checkState(Preconditions.java:590)
        at org.sonar.server.qualityprofile.builtin.BuiltInQProfileRepositoryImpl.lambda$updateOrCreateBuilder$6(BuiltInQProfileRepositoryImpl.java:206)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
        at java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1085)
        at org.sonar.server.qualityprofile.builtin.BuiltInQProfileRepositoryImpl.updateOrCreateBuilder(BuiltInQProfileRepositoryImpl.java:203)
        at org.sonar.server.qualityprofile.builtin.BuiltInQProfileRepositoryImpl.lambda$toQualityProfileBuilders$5(BuiltInQProfileRepositoryImpl.java:182)
        at java.base/java.util.HashMap.compute(HashMap.java:1229)
        at org.sonar.server.qualityprofile.builtin.BuiltInQProfileRepositoryImpl.toQualityProfileBuilders(BuiltInQProfileRepositoryImpl.java:180)
        at org.sonar.server.qualityprofile.builtin.BuiltInQProfileRepositoryImpl.lambda$toFlatList$2(BuiltInQProfileRepositoryImpl.java:138)
        at org.sonar.core.util.stream.MoreCollectors.lambda$uniqueIndex$9(MoreCollectors.java:258)
        at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)

[maybeec]

Same as #111?

[hohwille]

Same as #111?

Seems to be related.

[hohwille]

https://community.sonarsource.com/t/marketplace-unstable/67115

[hohwille]

Indeed the problem here is what has already been analyzed in #111:
The problem is that we can not find a way via SonarQube Plugin API to check ...

• which plugins are installed
• which version of a plugin is installed
• which rules are available

We can add a rule to a quality profile that refers to the Rule-ID of a rule from a specific plugin.
However, SonarQube accepts this without error and processes this request asynchronous.
Later during bootstrapping it then fails to startup the entire server if a rule is referenced that does not exist (no plugin is providing it).

See also #110

So without any further progress or insights into SonarQube Plugin API we are unable to create a maintainable way to setup our quality profile.

We could somehow try the following hacks:
Scan the local disc and find the SonarQube installation somehow and check the files in extensions/plugins and derive the plugins and their version from the filenames. Maybe even scan the relevant JAR files and dig for contained rules so even future incompatibilities can be avoided.

[hohwille]

Further investigation:
According to our own installation-guide, we need this plugin installed:
https://github.com/QualInsight/qualinsight-plugins-sonarqube-smell

However, this plugin seems to be discontinued and unmaintained. See also:
https://github.com/SonarSource/sonar-update-center-properties/blob/master/smells.properties

I am not sure, if it has been replaced by this plugin or if that is a different but similar one:
https://github.com/SEPMLAB/CodeHawk
https://github.com/SonarSource/sonar-update-center-properties/blob/master/codehawk.properties

According to the current state, I would propose to remove this unmaintainable dependency from sonar-devon4j-plugin and focus on quality and stability. If users find some exotic plugins and their rules useful, they can copy our quality profile and add whatever rules they like on top.
However, with the current effects, we have to focus on a minimum set of rules that we can guarantee to support and make work also in future releases.

[hohwille]

Others also seem to make this frustrating experience that SonarQube Ecosystem is not designed stable (similar to Eclipse):
CreditMutuelArkea/sonar-apigee-plugin#5

[hohwille]

So my current suggestion is to go to a minimal but reliable set of rules and required plugins via implementing #110 and reducing our quality profile to a set of stable rules (stripping all qualinsight smell rules, consider if we can use Sonar Java plugin rules to address all the major things including to consider dropping checkstyle and PMD).

However, even with Sonar Java Plugin we can observe that nothing is stable and rules come and go over the years.
Still we would need to find a way to check the rules contained in the plugin. Maybe this is also possible:
https://github.com/SonarSource/sonar-java/blob/cbf1e883e99c0c54b499b8663543fc6449aa856e/java-checks/src/main/java/org/sonar/java/checks/naming/BadAbstractClassNameCheck.java#L37
So we could locate the JAR in the extensions/plugins folder on the disc and do a scanning for Java files (e.g. using classgraph, asm, or whatever) to check for all the @rule annotation to collect a Set of available rules.
Then we only add rules from our XML config to the quality profile that are available in the Set.
At least this will solve the problem that the SonarQube server cannot startup what is a very nasty blocker.

It will not help that SonarQube guys start renaming existing rules or deleting them, splitting them into new smaller rules with new keys, etc.
In the end a new Sonar Java version could result in our quality profile being empty in the worst case.
Still better than making the entire server to startup.

[hohwille]

Another investigation shows that Sonar plugins like Java, XML, C#, etc. are already build in.
They can not be installed or uninstalled via marketplace as they always come with the SonarQube release itself.
This makes things easier assuming that we do not have to care about support for older versions of SonarQube without this behavior.

[hohwille]

I again have to give up for today:
I step-wise out-commented rule by rule when it was reported as error on startup.
This process was so annoying as we have so many rules, that I automated it step by step.
After full automation all rules had been disabled.
So to give an obvious example:

Caused by: java.lang.IllegalStateException: Rule with key 'devon4j:N8' not found

So SonarQube can not find a rule we define by our own plugin.
To me this means that SonarQube release is broken or changed in an incompatible way.
Our entire quality profile generation stopped working (at least with SonarQube 9.4.0.54424).