Objective

Build an agent that gathers local news for SEO purposes in a non-English speaking region, using Japan as the target market.

Data Sources

To keep the demo lightweight, a curated set of Japanese and English RSS/XML feeds was selected as the primary data source.

RSS/XML feeds are preferred over web scraping in this context as they are robust, stable, and trivial to parse, making them well-suited for a reliable SEO news pipeline. A blend of Japanese and English sources was chosen to balance local market focus with cross-validation of translation accuracy.

Clustering

Headlines are clustered before analysis to keep the LLM focused on extracting SEO insights per topic, rather than spending context on ad-hoc categorization.

The clustering pipeline runs as follows:

1. News headlines are passed through an embedding model (e.g. gemini-embedding-001)
2. Embeddings are dimensionality-reduced via sklearn PCA (3072 → 50 dimensions) to improve clustering efficiency
3. Reduced embeddings are clustered using sklearn KMeans (k = 6)
4. Cluster labels are mapped back to their corresponding headlines

Analysis

The cleaned, clustered headlines are passed to an LLM to produce a structured report. Each cluster is evaluated as a single entry containing:

• Theme: the overarching topic of the cluster
• Summary: a brief description of the key developments
• SEO Keywords: 3 to 5 relevant search terms extracted from the cluster
• Suggested Posts: content ideas tailored for a B2B tech transformation solutions company