Skip to content
This repository was archived by the owner on Nov 6, 2023. It is now read-only.
This repository was archived by the owner on Nov 6, 2023. It is now read-only.

How to handle domains with very large subdomains list? #8545

Closed

Description

@Bisaloo

I browsed through the previous issues and couldn't find anything related to my question. I apologize if it was already asked and I simply missed it.

Recently, I came across a lot of potential candidates for a new ruleset but after looking them up with Sublist3r, I found out they have a huge number of subdomains.

A good example here would be mnhn.fr.

Sublist3r finds 744 subdomains for it. I'm betting a lot of them are not functional but still...

I am wondering what should be done in this case:

  • A wildcard rule is out of question because only some subdomains have a valid HTTPS configuration
  • I could check every single subdomain but even then, it would result in a huge ruleset, leading to a painful review and a difficult maintaining. I am not even sure such a ruleset would be accepted due to its size.
  • Don't do anything and wait for the domain to have HSTS on all subdomains or at least valid HTTPS on all of them

What's your opinion on this?

Activity

  1. jeremyn commented on Feb 20, 2017

    @jeremyn
    Contributor

    It depends on the subdomains. In pull request #6899 we covered about 80 subdomains using a wildcard and a special rule, but the subdomains apparently had identical content, so that's fine. On the other end is pull request #5265 for fnal.gov where we want to add mentions of over a hundred subdomains. These are I suppose managed by a variety of groups at a huge government science lab so a wildcard will not work. It's still waiting for review by someone other than me (since I made modifications to it).

    Huge rulesets are unattractive to review. You can put a lot of time into one ruleset and then the contributor doesn't respond quickly. It can be tricky to discuss a multiple problems in a single unthreaded GitHub pull request.

    I see about 240 mnhn.fr subdomains using the recommended Sublist3r settings. A lot of the subdomains are internal/technical like autodiscover, horde5, imap*, test*, pop3, proxy, and webmail. In my opinion these can be ignored entirely.

    At https://mnhn.fr I get an invalid certificate/incomplete certificate chain error, which is not a good sign.

    If I were you, I might submit a small pull request with some of the most important subdomains in a ruleset with (partial) in the name. If that went well then I would submit more pull requests later adding more subdomains. Add a comment in the pull request that you understand there are a lot more subdomains than you included but that you don't want to overwhelm the reviewer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions