Newsphere Live is a self-updating semantic map of current news. Every six hours, a GitHub Actions pipeline fetches articles from 11 RSS feeds, encodes them into 384-dimensional semantic vectors, reduces those vectors to three dimensions, clusters them by topic, and publishes an interactive 3D visualization to GitHub Pages, with no backend, no database, and no recurring cost.
The core idea is that proximity in the map means semantic similarity. Stories covering the same event from different sources cluster together automatically, independent of phrasing. A user can spin and zoom the visualization, hover to see headlines, click through to full articles, filter by topic cluster, or slide the timeline to see what's newly added.
This project extends an earlier static predecessor, a semantic map of 7,000 archived articles from 2018 built during coursework at Leiden, where clustering parameters were validated against silhouette scores and topic coherence. Newsphere Live uses those empirically-validated settings in a live, continuously-updating system.
11 RSS feeds · 384-dim MiniLM embeddings → 3D UMAP space · HDBSCAN clustering · 6-hour refresh
Zero backend · zero recurring cost · stable layout via Procrustes alignment
Six Python scripts run sequentially on every cron trigger (00:00, 06:00, 12:00, 18:00 UTC):
If validation fails, too few articles or clusters, the pipeline exits without committing, preserving the previous stable state on GitHub Pages. This makes the system robust to transient fetch failures without user intervention.
Multi-source news creates a specific problem: the same story arrives from five different outlets with slightly different phrasing. A purely visual map would show five near-identical points, which collapses the effective information density.
Newsphere handles this in two phases. Phase one runs at fetch time: titles are lowercased, punctuation-stripped, and whitespace-normalised; exact post-normalization matches are dropped immediately. Phase two runs post-embedding: pairs with cosine similarity above 0.90 are merged into a single point, with the longer description becoming canonical and the other sources populating an also_covered_by field visible in the UI on hover.
Each article (title + description) is encoded by all-MiniLM-L6-v2, a sentence-transformer model that produces 384-dimensional dense vectors capturing semantic meaning independent of word choice. Two articles saying the same thing differently end up close in this space; two articles on unrelated topics end up far apart.
UMAP reduces the 384-dimensional embeddings to three dimensions for browser rendering. The reduction prioritises preserving local neighbourhood structure, nearby articles in the original space should remain nearby in the projected space. The parameters used (neighbours, min_dist, metric) were validated against silhouette scores in the static predecessor project.
HDBSCAN identifies dense regions as topic clusters without requiring a pre-specified number of clusters. Articles that don't fit cleanly into any cluster are left as noise rather than forced into the nearest group, which matters in news, where genuinely miscellaneous one-off stories exist alongside coherent topic threads.
Each cluster gets a human-readable label generated by c-TF-IDF, a variant of TF-IDF applied at the cluster level rather than the document level. It identifies terms that are most distinctive within a cluster relative to the rest of the corpus. This produces labels like "Gaza ceasefire talks" or "Fed interest rate" rather than generic topic markers.
UMAP is stochastic: run it twice on similar data and the global orientation of the output can flip or rotate arbitrarily. For a live map that updates every six hours, this would mean returning users see an entirely different spatial layout on each visit.
Procrustes alignment solves this by using URL-matched articles (articles present in both the previous and current run) as reference points. An SVD-based transformation is computed to minimise the distance between their positions across runs, then applied to the full current layout. The macro-level orientation stays consistent; only articles that are genuinely new shift significantly.
The entire system runs on free tiers with no architectural compromises:
| Component | Provider | Cost |
|---|---|---|
| ML pipeline compute | GitHub Actions (public repo) | Free |
| Hosting | GitHub Pages | Free |
| News sources | RSS feeds | Free |
| Email delivery | Resend free tier (100/day) | Free |
| Subscriber database | Resend Audiences | Free |
| Subscription endpoint | Cloudflare Workers free tier | Free |
The key constraint enabling zero cost is the fully static architecture. The frontend is HTML, CSS, and JavaScript loading pre-built JSON files, there are no server-side requests at page load. GitHub Pages hosts static files for public repositories for free. Public GitHub Actions repositories get unmetered compute; each pipeline run takes 6–8 minutes.
The only server-side component is the email subscription endpoint, which cannot be handled client-side because it requires an API key. A Cloudflare Worker receives email addresses, calls the Resend API, and confirms success. The free tier allows 100,000 daily requests.
Bootstrap period. The first pipeline run starts from an empty store, yielding 300–450 articles from immediate feed snapshots. Clustering stabilises over the first few days as the rolling 7-day store accumulates depth.
UMAP drift. Procrustes alignment preserves macro-orientation but individual clusters can shift relative positions as UMAP graph structure changes between runs with significantly different article sets.
Feed sampling gaps. Six-hour fetch intervals mean high-volume publishers' content rotates faster than collection cycles. Articles that briefly exist between fetches are missed.
Subscriber ceiling. Resend's free tier supports 100 subscribers. Beyond that, a paid plan is required.
Full source code, pipeline scripts, and GitHub Actions workflow on GitHub →