AgentStack Docs

Crawl a single page

Add one public URL as knowledge when a full website crawl would be too broad.

Open Sources → Content. Select the Pages tab. Use this tab to index one public URL.

Use this method for a single page, for example a returns policy, a release note, or a help article. As a result, AgentStack does not crawl the rest of the site. If the information is a short, stable answer, a Q&A pair can be more precise.

Add the page

Under Add Page, enter the Page URL. Select Fetch Page. AgentStack adds the HTTPS scheme when the URL has none. AgentStack then queues the source. This does not mean that the page is ready for retrieval yet.

The Crawled Pages table shows the URL, status, and controls for each page. Wait for Completed before you test the page. Pending, Crawling, and Processing mean that AgentStack is still working on the page. Failed means that the page did not become usable.

If a page shows Failed, check that the URL is public. Then retry or replace the page. Delete a page when it no longer belongs in the agent’s knowledge.

Test what was indexed

When the page shows Completed, open Playground. Ask a question with an answer that appears in the page text. Then ask the same question in different words.

If the answer is missing, check the page body. The fact must be in the body text, not only in a navigation label or in content that loads dynamically.

Understand later retraining

AgentStack stores a manually added page under a website group. A later retrain can find new pages from that group’s URL, up to the page limit of your plan. The retrain does not refresh only the one URL that you added.

If you want ongoing discovery across a documentation area, use the website crawl guide instead. Before you enable regular retrains, read Retrain knowledge.

On this page