httpscreenshot is a tool developed internally over the past year and a half. It has become one of our go to tools for the reconnaissance phase of every penetration test. The tool itself takes a list of addresses, domains, URLs, and visits each in a browser, parses SSL certificates to add new hosts, and captures a screenshot/HTML of the browser instance. Similar tools exist but none met our needs with regards to speed (threaded), features (JavaScript support, SSL auto detection and certificate scraping), and reliability.
The cluster portion of the tool will go through and group "similar" websites together, where "similar" is determined by a fuzzy matching metric.
This tool can be used by both blue and red teams. The blue teams can use this tool to quickly create an inventory of applications and devices they have running in their environments. This inventory will allow them to quickly see if there is anything running in their environment that they may not know about which should be secured or in many cases removed.
The red teams can use this tool to quickly create the same inventory as part of our reconnaissance, which is often very effective in identifying potential target assets.