- The most complete -- having more companies that any other data source.
- The freshest -- checking the data at a higher frequency than everybody else.
- The most accurate -- precise identification and tracking of technologies used.
- The cheapest -- providing more for less than anyone else.
To generate the data you see, the following data sources are used:
- Majestic Million - 1,000,000 domains with the highest number of referring links.
- Wikipedia - company level data.
- Public Suffix List - used to determine the root domain when parsing URLs.
This site uses a set of open source software for infrastructure. These projects are well used, well maintained, robust, reliable, secure, free, and all the benefits of open source done right. Here are some packages being used:
- PostgreSQL -- an SQL database.
- Varnish Cache -- a reverse proxy/HTTP accelerator.
- Flask -- origin server software.
- Grafana -- a set of tools for monitoring, reporting on, and alerting on infrastructure.
- Python 3 -- a very flexible programming lagnuage with lots of high quality modules.
- Postfix -- SMTP for receiving and sending email.
- Dovecot -- IMAP and POP3 support for Postfix.
- Certbot -- TLS certificate generation.
- Ubuntu -- an Operating System.
Here are some diagrams of the architecture:
