platform-mcpandClaude Opus 4.8
6b07b83583
E2E Test Suite / Cross-Browser Regression (webkit) (push) Failing after 10m48s
E2E Test Suite / Cross-Browser Regression (firefox) (push) Failing after 11m43s
E2E Test Suite / Cross-Browser Regression (chromium) (push) Failing after 12m35s
E2E Test Suite / Destructive & Chaos Tests (push) Failing after 13m27s
E2E Test Suite / Form Interaction Tests (push) Failing after 14m20s
E2E Test Suite / Security Header Tests (push) Failing after 14m1s
E2E Test Suite / Mobile Device Tests (push) Failing after 14m56s
E2E Test Suite / Smoke Tests (P0) (push) Failing after 21m57s
E2E Test Suite / Critical User Journeys (push) Has been skipped
E2E Test Suite / API Integration Tests (push) Has been skipped
Deploy to Production / Build & Verify (push) Failing after 17m16s
E2E Test Suite / Test Report Summary (push) Failing after 13m13s
Deploy to Production / Pre-Deploy Tests (push) Failing after 12m22s
Deploy to Production / Deploy to Fly.io (push) Failing after 13m13s
Deploy to Production / Deploy to VPS (PM2) (push) Failing after 14m5s
Deploy to Production / Deploy to Render (push) Failing after 14m5s
Deploy to Production / Deploy to Railway (push) Failing after 14m57s
Deploy to Production / Post-Deploy Verification (push) Failing after 13h5m7s
Deploy to Production / Notify on Failure (push) Failing after 15m4s
Repo robots.txt already matched the deployed file (verified via diff against https://workroot.in/robots.txt). Added commonly-cited AI agents that were still missing, each with Allow: /: - OpenAI: OAI-SearchBot - Anthropic: ClaudeBot, Claude-SearchBot, Claude-User - Perplexity: Perplexity-User - Meta: Meta-ExternalAgent / meta-externalagent - Apple: Applebot, Applebot-Extended - Amazon: Amazonbot - ByteDance: Bytespider - Cohere: cohere-ai - DuckDuckGo: DuckAssistBot - You.com: YouBot - Diffbot Disallow list unchanged (/admin/, /private/, /_astro/) so no content directory needed for citation is blocked. Sitemap line preserved. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
135 lines
1.9 KiB
Plaintext
135 lines
1.9 KiB
Plaintext
# Robots.txt for WorkRoot IT Solutions
|
|
# https://workroot.in
|
|
|
|
# Default rules for all bots
|
|
User-agent: *
|
|
Allow: /
|
|
|
|
# Disallow admin and build directories
|
|
Disallow: /admin/
|
|
Disallow: /private/
|
|
Disallow: /_astro/
|
|
|
|
# Allow access to CSS and JS files for proper rendering
|
|
Allow: /*.css$
|
|
Allow: /*.js$
|
|
Allow: /*.webp$
|
|
Allow: /*.jpg$
|
|
Allow: /*.png$
|
|
Allow: /*.svg$
|
|
|
|
# Crawl-delay for polite crawling
|
|
Crawl-delay: 1
|
|
|
|
# Sitemap location
|
|
Sitemap: https://workroot.in/sitemap.xml
|
|
|
|
# Traditional Search Engines
|
|
User-agent: Googlebot
|
|
Allow: /
|
|
|
|
User-agent: Googlebot-Image
|
|
Allow: /
|
|
|
|
User-agent: Bingbot
|
|
Allow: /
|
|
|
|
User-agent: Slurp
|
|
Allow: /
|
|
|
|
User-agent: DuckDuckBot
|
|
Allow: /
|
|
|
|
User-agent: Baiduspider
|
|
Allow: /
|
|
|
|
User-agent: YandexBot
|
|
Allow: /
|
|
|
|
# Google AdSense Bot
|
|
User-agent: Mediapartners-Google
|
|
Allow: /
|
|
|
|
# AI Search Engines & LLM Crawlers (GEO Optimization)
|
|
# OpenAI/ChatGPT
|
|
User-agent: ChatGPT-User
|
|
Allow: /
|
|
|
|
User-agent: GPTBot
|
|
Allow: /
|
|
|
|
User-agent: OAI-SearchBot
|
|
Allow: /
|
|
|
|
# Anthropic/Claude
|
|
User-agent: anthropic-ai
|
|
Allow: /
|
|
|
|
User-agent: Claude-Web
|
|
Allow: /
|
|
|
|
User-agent: ClaudeBot
|
|
Allow: /
|
|
|
|
User-agent: Claude-SearchBot
|
|
Allow: /
|
|
|
|
User-agent: Claude-User
|
|
Allow: /
|
|
|
|
# Perplexity AI
|
|
User-agent: PerplexityBot
|
|
Allow: /
|
|
|
|
User-agent: Perplexity-User
|
|
Allow: /
|
|
|
|
# Google Gemini/Bard
|
|
User-agent: Google-Extended
|
|
Allow: /
|
|
|
|
# Common Crawl (used by many AI models)
|
|
User-agent: CCBot
|
|
Allow: /
|
|
|
|
# Meta AI
|
|
User-agent: FacebookBot
|
|
Allow: /
|
|
|
|
User-agent: Meta-ExternalAgent
|
|
Allow: /
|
|
|
|
User-agent: meta-externalagent
|
|
Allow: /
|
|
|
|
# Apple Intelligence
|
|
User-agent: Applebot
|
|
Allow: /
|
|
|
|
User-agent: Applebot-Extended
|
|
Allow: /
|
|
|
|
# Amazon AI
|
|
User-agent: Amazonbot
|
|
Allow: /
|
|
|
|
# ByteDance / TikTok AI
|
|
User-agent: Bytespider
|
|
Allow: /
|
|
|
|
# Cohere AI
|
|
User-agent: cohere-ai
|
|
Allow: /
|
|
|
|
# DuckDuckGo AI (DuckAssist)
|
|
User-agent: DuckAssistBot
|
|
Allow: /
|
|
|
|
# You.com AI
|
|
User-agent: YouBot
|
|
Allow: /
|
|
|
|
# Diffbot (knowledge graph / AI training)
|
|
User-agent: Diffbot
|
|
Allow: /
|