Extract
Build a clean training corpus
Raw HTML teaches a model about cookie banners. Extraction returns the article.
Raw HTML is mostly navigation, cookie banners and footers. Feeding that to a model teaches it about cookie banners. Extraction returns the article and nothing else, as text or Markdown, with a word count so you can filter thin pages before they reach your pipeline.
- Each page reports a word count taken before any truncation, so thin pages can be filtered out cheaply.
- Up to 8 URLs per call, and a URL that fails to fetch is never billed.
- Markdown preserves headings, lists and links, which is the structure most chunking strategies depend on.
Send up to 8 URLs per call and pay only for the pages that came back successfully.