Measure keyword signals in daily domain additions

Python example: Market Intelligence

Requires Python 3.9 or later. Uses the standard library only, processes local files, and makes no network requests. Save the script beside your downloaded inputs, or provide full paths.

Download market-intelligence.py

Run the example

python3 market-intelligence.py new.txt.gz --keywords cloud ai solar

Results are printed to the terminal. Redirect standard output to a file if you want to save the report. For the SQLite example, the database is saved at the path you specify.

Complete script

import argparse
import csv
import gzip
import sys


def domains(path):
    with gzip.open(path, "rt", encoding="utf-8") as source:
        for line in source:
            domain = line.strip().lower().rstrip(".")
            if domain:
                yield domain

from collections import Counter

parser = argparse.ArgumentParser(description="Count topic signals in a domain list")
parser.add_argument("file")
parser.add_argument("--keywords", nargs="+", required=True)
args = parser.parse_args()
counts = Counter()
total = 0
for domain in domains(args.file):
    total += 1
    for word in args.keywords:
        if word.lower() in domain.rsplit(".", 1)[0]:
            counts[word] += 1
writer = csv.writer(sys.stdout)
writer.writerow(["input_file", "keyword", "matches", "total", "share_percent"])
for word in args.keywords:
    writer.writerow([args.file, word, counts[word], total,
                     round(100 * counts[word] / total, 4) if total else 0])

To automate input downloads, follow the API documentation for tokens, supported endpoints, and historical dates. Keep API tokens out of shared scripts.