Public crawl and LLM discovery surfaces are generated from the same site manifest and content sources.
Page metadata
src/lib/seo.ts builds shared head payloads:
title/descriptioncanonicalURL- Open Graph + Twitter cards
- optional
article:*fields for blog posts noindexfor private or auth surfaces when needed
Public pages should call buildPageHead({ title, description, path, ... }) from their route head() function.
Sitemap and robots
/sitemap.xmllists static public routes, docs pages, and published blog posts when the blog service is reachable/robots.txtallows public paths, blocks auth/dashboard/API, and points to the sitemap + llms index
Implementation: src/lib/server/public-routes.ts plus route handlers under src/routes/.
llms.txt
/llms.txtis a short index of core pages, docs, blog, and legal routes/llms-full.txtexpands the same set with full docs/blog body content when available
These files are designed for LLM crawlers and internal agent tooling. Keep them public and free of secrets.
Content ownership
- Docs catalog:
src/lib/content/docs.ts(mid-term can move to markdown files) - Blog posts:
BLOG_SERVICEAPI (cover, author, date, description, markdown) - Legal pages: locale legal copy modules