post https://api.fetchfox.ai/api/crawl
Crawl a site for URLs matching a pattern
Crawl takes a URL pattern, and finds matching URLs on the target site. A URL pattern is a URL that contains at least one wildcard, like https://example.com/category/*.
You can control the crawls maximum depth, how it prioritizes pages, starting URLs, and more.