Is there a metric for "architectural complexity"?

[skytreader]

Hello. This is the question I wanted to ask originally before I ran into issue #59. Though I fear is more about the state of the art than this project exclusively, I also want to hear your opinion on whether the project's current parsing can already measure for this (that is, we have the data structures but we lack the algorithm, maybe).

So can the framework measure "architectural complexity" especially with respect to module coupling? The use-case for this is best illustrated by the aforementioned FizzBuzzEnterpriseEdition. The tool ranks its cognitive complexity as follows: 92.4% low, 5.5% medium, 2.1% high. Those are enviable numbers.

And yet that does not reflect the fact that FizzBuzz EE is over-engineered and hardly a good example of software architecture. On personal introspection, the problem is too much abstraction. On one hand this is probably the key to minimizing the cognitive complexity overall but on the other hand, this removes a lot of the code from surrounding context that, on any real-world problem domain, it's hard to see how all of it comes together.

Based on experience, this also affects the testability of the code: it's easy to verify that each individual module conforms to its contract but it's hard to verify that it all fits together properly. Even with 100% coverage, at the top of the abstraction tree you can still be blindsided by a subtle change/edge case one of your dependencies introduced.

Unfortunately I haven't found a study for this metric yet, so I can't suggest an established concept to follow. In fact, the only significant mention I can find of this in my own research is in this blog entry:

Coupling is the connection between one piece of code and another. Keep it to a minimum. If a problem arises with one piece, you’ll need to adjust each piece that’s coupled to it. A developer can fall down the rabbit hole trying to find out which coupled function didn’t get updated.

So what do you think about this? Can the current AST parsing handle this? Is it even possible for plain static code analysis to achieve this? Personally, it would suit my use cases if this is possible with TS but maybe for weaker type systems this would be impossible.

(I'm thinking one ad-hoc way to do this would be to find the depth of the abstraction/dependency tree, but I already foresee some pitfalls with this approach, e.g., if there are unused imports, if there are untyped variables, etc.)

[gillesfabre34]

Hello,

First, thanks again for your interest in @genese/complexity. I will try to give responses as clear as possible.

• "Good" or "bad" scores.

The tool ranks its cognitive complexity as follows: 92.4% low, 5.5% medium, 2.1% high. Those are enviable numbers.

In reality, you may decide what is a "good" score for you, by changing the warning or error thresholds in the geneseconfig.json file. It may be different for React or Angular pass, for example. You may also change these thresholds if you want to have a high or very high maintainability of your code. The pressure on the developers will also be different...

• Other kinds of complexity.

You are right, the architecture and the level of abstraction are important factors which may affect the maintainability of your app. Unfortunately, as far as I know, there is no relevant metric which is able to measure correctly these kinds of complexity. More important: in the article that you shared, there are a lot of metrics which are presented, like the Halstead volume, the cyclomatic complexity, the complexity index, the maintainability index, etc. Unfortunately, all of these metrics are not (or very weakly) correlated to the difficulty to understand the code. Many developers or companies are using them, but in reality there is no scientific proof that these metrics are relevant. I began a PhD in this subject, which is very interesting !

You can read my article about the different metrics of complexity, with all the references of the different scientific articles which are talking about this. You don't need to read the parts related to the dynamic metrics, which were the subject of my thesis.

I hope it will help you !

PS : do not close this comment, it may be interesting for other developers ;)

[skytreader]

Thanks for the detailed reply and the article link. That explains a lot. I think at the moment the most relevant bit to my scenario is at the end, by the future work section

To this end, we will need to analyze the results of more sophisticated experiments, with more complex problematics. For example, we would like to analyze the results of experiments with functions using objects instead of primitives.

It does reflect my experience that the current metrics are better-suited for procedural code specimens but characterize little of the overhead that OOP introduces.

As for my current problem, I think I'll keep playing around with the AST generated and see if there's a loose metric I can come up with anyway, though it would very likely be highly biased to the code base I'm dealing with. 👍