Odd behaviour: auto adding inscope rules while adding domains

[pdelteil]

Hello!

I have this issue on my programs: Some will have a long list of inscope rules that wasn't added to the program.

I think is due to these lines of code (here):

   # not entirely sure this will ever occur, but hey (update: it does occur, as a result of crt.sh)
            # it makes sense to do this here, because it will still check whether it is in scope
            # before extending the existing scope.
            if domain.startswith('*.'):
                domain = domain[2:]
                # if it matches the existing scope definition,
                # add this wildcard to the scope too
                if REGEX_DOMAIN.match(domain) and not self.matches_scope(domain, outscope) and self.matches_scope(domain, inscope):
                    add_inscope.append('*.'+domain)
 

I think that it is triggered when crt.sh gives out domains that include a *.

I would not want my inscope rules to be modified, so I will just comment this section of the code.

Thanks!

[honoki]

Hey @pdelteil - I think this is intentional, to automatically improve the inscope by finding existing subdomains and checking whether or not they are out of scope before adding them as (implicitly) in scope.

Can you give an example when this results in unintended behaviour?

[pdelteil]

Hello there @honoki,

This is an example:

Scope: *.mybbprogram.com

crt.sh gives:

*.test.mybbprogram.com
*.old.test.mybbprogram.com
*.anothertest.mybbprogram.com
...

This rules are redundant since they are already included in *.mybbprogram.com and if you use the inscope rules as input to another tools this might increase the time and resources without providing any benefits. I guess they might be some cases + some specific tool where some new domains can be found. But not my case and I don't want my inscope/outscope rules to be modified automatically.

[honoki]

Hi @pdelteil - gotcha; I'm leaning towards reverting this and making it opt-in behaviour by setting a program tag bbrf program update testprogram -t auto-extend-scope:true.

In that case, only programs with tag auto-extend-scope set to true will see their inscope automatically modified when adding asterisk domains like *.sub.example.com.

This change is lined up for v1.3.3