Pages Websites
Non Indexable Pages MCP
List the pages on a website that search engines cannot index, with the reason. Call the Non-Indexable Pages API via REST or MCP to automate SEO research.
Non Indexable Pages AI Prompts & Use Cases
After connecting this MCP tool to your AI assistant, you can ask the questions below. Replace bracketed placeholders with your own inputs and use the supported values in Arguments.
New to Nexscope MCP? Follow the connection guide, then use the prompts below. The Integration and Parameters & Response tabs contain the connection steps, call example, and technical reference.
1. Find Pages With Indexing Restrictions
Prompt: From my completed website audit [audit reference], show pages classified as non-indexable and the reasons returned for each page. Return up to 20 results in this request. Start with the first result batch.
2. Continue The Audit Review
Prompt: From my completed website audit [audit reference], show pages classified as non-indexable and the reasons returned for each page. Continue after [number already reviewed] and return up to 20 more records. Stop after this batch and preserve the same audit reference.
3. Group Indexing Restrictions For Review
Prompt: From my completed website audit [audit reference], show pages classified as non-indexable and the reasons returned for each page. Return up to 20 results in this request. Start with the first result batch. Group the returned URLs by their reported restriction reason, showing a sample URL for each group. Keep crawl and pagination limits visible; do not claim every restriction is an error.
What You Need to Provide
Provide the existing reference returned by the corresponding source request, from your own account. A placeholder reference cannot retrieve a real result.
Missing values are not zero. Interpret only returned evidence within its observation period and sample scope.
Step 1 · Connect
Connect Your AI Assistant
Connect a compatible MCP client to Nexscope, then select this tool from your assistant. Use the connection settings in your client to complete authentication.
POST https://api.nexscope.ai/mcp
MCP-Protocol-Version: 2025-11-25
Content-Type: application/jsonAuthentication
Use OAuth 2.1 Authorization Code with PKCE and the mcp:tools scope. API keys remain supported as a compatibility option.
Authorization: Bearer nk-xxxxxxxxxxxxxxxxxUsing an API key? Get your key from API Access and include it in the Authorization header shown above.
Call this MCP tool
nexscope_non_indexable_pageshttps://api.nexscope.ai/mcptools/callInitialize an MCP 2025-11-25 Streamable HTTP session at https://api.nexscope.ai/mcp, then use tools/call.
{
"jsonrpc": "2.0",
"id": "non-indexable-pages-1",
"method": "tools/call",
"params": {
"name": "nexscope_non_indexable_pages",
"arguments": {
"limit": 20,
"offset": 0,
"task_id": ""
}
}
}MCP 2025-11-25 · Streamable HTTPProgress is delivered on the original POST SSE response. GET SSE carries notifications; reconnecting must query existing asynchronous work rather than submit again.
Arguments
| Name | Type | Required | Description |
|---|---|---|---|
limit | integer | optional | - |
offset | integer | optional | - |
task_id | string | required | - |
Response fields
Treat structuredContent as the programmatic API response payload.
| Name | Type | Required | Description |
|---|---|---|---|
code | integer | required | 0 is a successful envelope; inspect data.status separately. Allowed: 0, 42008, 42009 |
cost | string | required | Elapsed processing milliseconds; not price. -1 for non-web context. |
data | object | required | Data value used by this API operation. |
data.callId | string | null | optional | Call id value used by this API operation. |
data.completedAt | string | integer | null | optional | Completed at value used by this API operation. |
data.coverage | object | optional | Coverage value used by this API operation. |
data.coverage.missing | array | optional | Missing value used by this API operation. |
data.coverage.missing[] | object | required | Item in data coverage missing. |
data.coverage.missing[].field | string | optional | Field value used by this API operation. |
data.coverage.missing[].reason | string | optional | Reason value used by this API operation. |
data.coverage.status | string | optional | Status value used by this API operation. |
data.requestedAt | string | integer | null | optional | Requested at value used by this API operation. |
data.result | object | null | required | Result value used by this API operation. |
data.result.coverage | object | required | Coverage value used by this API operation. |
data.result.coverage.invalidItemCount | integer | required | Invalid item count value used by this API operation. Minimum: 0 |
data.result.coverage.limit | integer | required | Limit value used by this API operation. Minimum: 0 |
data.result.coverage.maxCrawlPages | integer | null | required | Max crawl pages value used by this API operation. Minimum: 0 |
data.result.coverage.offset | integer | required | Offset value used by this API operation. Minimum: 0 |
data.result.coverage.pagesCrawled | integer | null | required | Pages crawled value used by this API operation. Minimum: 0 |
data.result.coverage.providerTotal | integer | null | required | Provider total value used by this API operation. Minimum: 0 |
data.result.coverage.returnedCount | integer | required | Returned count value used by this API operation. Minimum: 0 |
data.result.coverage.sampled | boolean | required | Sampled value used by this API operation. Constant: true |
data.result.coverage.stopReason | string | null | required | Stop reason value used by this API operation. |
data.result.coverage.truncated | boolean | required | Truncated value used by this API operation. |
data.result.items | array | required | Items value used by this API operation. Maximum items: 100 |
data.result.items[] | object | required | Item in data result items. |
data.result.items[].reason | string | null | required | Reason value used by this API operation. Allowed: ROBOTS_TXT, META_TAG, HTTP_HEADER, ATTRIBUTE, TOO_MANY_REDIRECTS, UNKNOWN, null |
data.result.items[].url | string | required | Url value used by this API operation. Maximum length: 2048 |
data.result.missingFields | array | required | Missing fields value used by this API operation. Maximum items: 10000 |
data.result.missingFields[] | object | required | Item in data result missingFields. |
data.result.missingFields[].field | string | required | Field value used by this API operation. |
data.result.missingFields[].reason | string | required | Reason value used by this API operation. |
data.result.source | object | required | Source value used by this API operation. |
data.result.source.candidateId | string | required | Candidate id value used by this API operation. Constant: MD-PW-10 |
data.result.source.collectedAt | integer | null | required | Collected at value used by this API operation. Minimum: 0 Maximum: 9007199254740991 |
data.result.source.observationKind | string | required | Observation kind value used by this API operation. Constant: CRAWL_SAMPLE |
data.result.source.product | string | required | Product value used by this API operation. Constant: on_page |
data.result.source.provider | string | required | Provider value used by this API operation. Constant: DATAFORSEO |
data.result.source.queriedAt | integer | required | Queried at value used by this API operation. Minimum: 0 Maximum: 9007199254740991 |
data.result.source.sourceCallId | string | null | required | Source call id value used by this API operation. |
data.result.source.sourcePlanVersion | string | null | required | Source plan version value used by this API operation. |
data.result.source.sourceSchemaVersion | string | null | required | Source schema version value used by this API operation. |
data.result.warnings | array | required | Warnings value used by this API operation. Maximum items: 1000 |
data.result.warnings[] | string | required | Item in data result warnings. |
data.schemaVersion | string | null | optional | Schema version value used by this API operation. |
data.source | string | null | optional | Source value used by this API operation. |
data.status | string | optional | Status value used by this API operation. Allowed: SUCCESS, PARTIAL, EMPTY, PROCESSING, UNKNOWN, FAILED |
data.usage | object | null | optional | Usage value used by this API operation. |
data.usage.billingStatus | string | null | optional | Billing status value used by this API operation. |
data.usage.chargedCredits | string | integer | null | optional | Charged credits value used by this API operation. |
msg | string | null | required | Available upstream msg/errmsg/message is preferred for success and failure; Chinese is translated to English. Missing successful message is null. Platform-only errors and task states use platform English. Message text never determines success or retry. |
time | string | required | yyyy-MM-dd HH:mm:ss |
traceId | string | null | required | Trace id value used by this API operation. |
ts | string | required | Epoch milliseconds; global long serializer. |
Format Illustration
Illustrative example with fictional data, not a live tool call.
Illustrative Question
From my completed website audit 00000000-0000-4000-8000-000000000001, show pages classified as non-indexable and the reasons returned for each page. Show up to 20 results.
Illustrative Response Explanation
The illustrative result flags one page with a meta-tag indexing restriction. That restriction may be intentional; the result does not establish that every non-indexable page is an error.
About this sample: Illustrative example with fictional data, not a live tool call. Reserved example domains and references demonstrate the format and cannot be used as proof of current results. Null fields remain unavailable, not zero.
Published sample data is not available for this page. A format illustration or workflow guide is not a substitute for a published result.
View the MCP Request & Response Format
Examples
Illustrative example with fictional data, not a live tool call.
The JSON-RPC envelope below illustrates how the shown arguments and payload fit into MCP. It is not captured network traffic or proof of a successful call. Workflow-only examples do not include a result.
MCP Request
{
"jsonrpc": "2.0",
"id": "non-indexable-pages-example",
"method": "tools/call",
"params": {
"name": "nexscope_non_indexable_pages",
"arguments": {
"limit": 20,
"offset": 0,
"task_id": "00000000-0000-4000-8000-000000000001"
}
}
}MCP Response Format
{
"jsonrpc": "2.0",
"id": "non-indexable-pages-example",
"result": {
"content": [
{
"type": "text",
"text": "{\"code\":0,\"cost\":\"-1\",\"data\":{\"callId\":\"00000000-0000-4000-8000-000000000001\",\"completedAt\":\"1767225601000\",\"coverage\":{\"missing\":[],\"status\":\"PARTIAL\"},\"requestedAt\":\"1767225600000\",\"result\":{\"coverage\":{\"invalidItemCount\":0,\"limit\":0,\"maxCrawlPages\":null,\"offset\":0,\"pagesCrawled\":null,\"providerTotal\":null,\"returnedCount\":0,\"sampled\":true,\"stopReason\":null,\"truncated\":false},\"items\":[{\"reason\":\"META_TAG\",\"url\":\"https://example.com/private\"}],\"missingFields\":[],\"source\":{\"candidateId\":\"MD-PW-10\",\"collectedAt\":null,\"observationKind\":\"CRAWL_SAMPLE\",\"product\":\"on_page\",\"provider\":\"DATAFORSEO\",\"queriedAt\":0,\"sourceCallId\":null,\"sourcePlanVersion\":null,\"sourceSchemaVersion\":null},\"warnings\":[]},\"schemaVersion\":\"1.0\",\"source\":\"dataforseo\",\"status\":\"PARTIAL\",\"usage\":null},\"msg\":null,\"time\":\"2026-01-01 00:00:01\",\"traceId\":null,\"ts\":\"1767225601000\"}"
}
],
"structuredContent": {
"code": 0,
"cost": "-1",
"data": {
"callId": "00000000-0000-4000-8000-000000000001",
"completedAt": "1767225601000",
"coverage": {
"missing": [],
"status": "PARTIAL"
},
"requestedAt": "1767225600000",
"result": {
"coverage": {
"invalidItemCount": 0,
"limit": 0,
"maxCrawlPages": null,
"offset": 0,
"pagesCrawled": null,
"providerTotal": null,
"returnedCount": 0,
"sampled": true,
"stopReason": null,
"truncated": false
},
"items": [
{
"reason": "META_TAG",
"url": "https://example.com/private"
}
],
"missingFields": [],
"source": {
"candidateId": "MD-PW-10",
"collectedAt": null,
"observationKind": "CRAWL_SAMPLE",
"product": "on_page",
"provider": "DATAFORSEO",
"queriedAt": 0,
"sourceCallId": null,
"sourcePlanVersion": null,
"sourceSchemaVersion": null
},
"warnings": []
},
"schemaVersion": "1.0",
"source": "dataforseo",
"status": "PARTIAL",
"usage": null
},
"msg": null,
"time": "2026-01-01 00:00:01",
"traceId": null,
"ts": "1767225601000"
},
"isError": false
}
}