Path Gateway Specification (original) (raw)

IPFS Standards

17 April 2024

History

Commit History

Feedback

GitHub ipfs/specs (inspect source, open issue)

The most versatile form of IPFS Gateway is a Path Gateway.

It exposes namespaces like /ipfs/ and /ipns/ under HTTP server root and provides basic primitives for integrating IPFS resources within existing HTTP stack.

Note: additional Web Gateways aimed for website hosting and web browsers extend the below spec and are defined in [subdomain-gateway] and [dnslink-gateway]. There is also a minimal [trustless-gateway] specification for use cases where client prefers to perform all validation locally.

Table of Contents

1. 1. HTTP API
   1. 1.1 GET /ipfs/{cid}[/{path}][?{params}]
   2. 1.2 HEAD /ipfs/{cid}[/{path}][?{params}]
      1. 1.2.1 only-if-cached HEAD behavior
   3. 1.3 GET /ipns/{name}[/{path}][?{params}]
   4. 1.4 HEAD /ipns/{name}[/{path}][?{params}]
2. 2. HTTP Request
   1. 2.1 Request Headers
      1. 2.1.1 If-None-Match (request header)
      2. 2.1.2 Cache-Control (request header)
         1. 2.1.2.1 only-if-cached
      3. 2.1.3 Accept (request header)
      4. 2.1.4 Range (request header)
      5. 2.1.5 Service-Worker (request header)
   2. 2.2 Request Query Parameters
      1. 2.2.1 filename (request query parameter)
      2. 2.2.2 download (request query parameter)
      3. 2.2.3 format (request query parameter)
      4. 2.2.4 dag-scope (request query parameter)
      5. 2.2.5 entity-bytes (request query parameter)
      6. 2.2.6 car-version (request query parameter)
      7. 2.2.7 car-order (request query parameter)
      8. 2.2.8 car-dups (request query parameter)
3. 3. HTTP Response
   1. 3.1 Response Status Codes
      1. 3.1.1 200 OK
      2. 3.1.2 206 Partial Content
      3. 3.1.3 301 Moved Permanently
      4. 3.1.4 400 Bad Request
      5. 3.1.5 404 Not Found
      6. 3.1.6 410 Gone
      7. 3.1.7 412 Precondition Failed
         1. 3.1.7.1 Use with only-if-cached
      8. 3.1.8 429 Too Many Requests
      9. 3.1.9 451 Unavailable For Legal Reasons
      10. 3.1.10 500 Internal Server Error
      11. 3.1.11 502 Bad Gateway
      12. 3.1.12 504 Gateway Timeout
   2. 3.2 Response Headers
      1. 3.2.1 Etag (response header)
      2. 3.2.2 Cache-Control (response header)
      3. 3.2.3 Last-Modified (response header)
      4. 3.2.4 Content-Type (response header)
      5. 3.2.5 Content-Disposition (response header)
      6. 3.2.6 Content-Location (response header)
      7. 3.2.7 Content-Length (response header)
      8. 3.2.8 Content-Range (response header)
      9. 3.2.9 Accept-Ranges (response header)
      10. 3.2.10 Location (response header)
          1. 3.2.10.1 Use in directory URL normalization
          2. 3.2.10.2 Use in interop with Subdomain Gateway
      11. 3.2.11 X-Ipfs-Path (response header)
      12. 3.2.12 X-Ipfs-Roots (response header)
      13. 3.2.13 X-Content-Type-Options (response header)
      14. 3.2.14 Retry-After (response header)
      15. 3.2.15 Server-Timing (response header)
      16. 3.2.16 Traceparent (response header)
      17. 3.2.17 Tracestate (response header)
   3. 3.3 Response Payload
4. 4. Appendix: notes for implementers
   1. 4.1 Content resolution
      1. 4.1.1 Finding the content root
      2. 4.1.2 Traversing remaining path
      3. 4.1.3 Traversing through UnixFS
      4. 4.1.4 Traversing through DAG-JSON and DAG-CBOR
      5. 4.1.5 Handling traversal errors
   2. 4.2 Best practices for HTTP caching
   3. 4.3 Denylists
   4. 4.4 Generated HTML with directory index
   5. 4.5 Recursive vs non-recursive gateways
5. A. References
6. B. Acknowledgments

1. HTTP API

Path Gateway provides HTTP interface for requesting content-addressed data at specified content path.

1.1 GET /ipfs/{cid}[/{path}][?{params}]

Downloads data at specified immutable content path.

• cid – a valid content identifier (CID)
• path – optional path parameter pointing at a file or a directory under the cid content root
• params – optional query parameters that adjust response behavior

1.2 HEAD /ipfs/{cid}[/{path}][?{params}]

Same as GET, but does not return any payload.

Implementations SHOULD limit the scope of IPFS data transfer triggered byHEAD requests to a minimal DAG subset required for producing response headers such asX-Ipfs-Roots,Content-Lengthand Content-Type.

1.2.1 only-if-cached HEAD behavior

HTTP client can send HEAD request withCache-Control: only-if-cachedto disable IPFS data transfer and inexpensively probe if the gateway has the data cached.

Implementation MUST ensure that handling only-if-cached HEAD response is fast and does not generate any additional I/O such as IPFS data transfer. This allows light clients to probe and prioritize gateways which already have the data.

1.3 GET /ipns/{name}[/{path}][?{params}]

Downloads data at specified mutable content path.

Implementation must resolve the name to a CID, then serve response behind a/ipfs/{resolved-cid}[/{path}][?{params}] content path.

• name may refer to:
  • cryptographic IPNS key hash
  • human-readable DNS name with DNSLink set-up

1.4 HEAD /ipns/{name}[/{path}][?{params}]

Same as GET, but does not return any payload.

2. HTTP Request

2.2 Request Query Parameters

All query parameters are optional.

2.2.1 filename (request query parameter)

Optional, can be used for overriding the filename.

When set, gateway will include it in Content-Disposition header and may use it for Content-Type calculation.

Example:

https://ipfs.io/ipfs/QmfM2r8seH2GiRaC4esTjeraXEachRt8ZsSeGaWTPLyMoG?filename=hello_world.txt

2.2.2 download (request query parameter)

Optional, can be used to request specific Content-Disposition to be set on the response.

Response to HTTP request with download=true MUST includeContent-Disposition: attachment[;filename=...]to indicate that client should not render the response.

The attachment context will force user agents such as web browsers to present a 'Save as' dialog instead (prefilled with the value of the filenameparameter, if present)

2.2.3 format (request query parameter)

Optional, format=<format> can be used to request specific response format.

This is a URL-friendly alternative to sending an Accept header. These are the equivalents:

• format=raw → Accept: application/vnd.ipld.raw
• format=car → Accept: application/vnd.ipld.car
• format=tar → Accept: application/x-tar
• format=dag-json → Accept: application/vnd.ipld.dag-json
• format=dag-cbor → Accept: application/vnd.ipld.dag-cbor
• format=json → Accept: application/json
• format=cbor → Accept: application/cbor
• format=ipns-record → Accept: application/vnd.ipfs.ipns-record

When both Accept HTTP header and format query parameter are present,Accept SHOULD take precedence.

A Client SHOULD include the format query parameter in the request URL, in addition to the Accept header. This provides the best interoperability and ensures consistent HTTP cache behavior across various gateway implementations.

A Gateway SHOULD include theContent-Location header in the response when:

• the request contains an Accept header specifying a well-known response format, but the URL does not include the format query parameter
• the format parameter is present, but does not match the format from Accept

2.2.4 dag-scope (request query parameter)

Optional, can be used to limit the scope of verifiable DAG requests such as CAR, same as dag-scope from [trustless-gateway].

2.2.5 entity-bytes (request query parameter)

Optional, can be used to limit the scope of verifiable DAG requests such as CAR, same as entity-bytes from [trustless-gateway].

2.2.6 car-version (request query parameter)

Optional, specific to CAR requests, same as car-version from [trustless-gateway].

2.2.7 car-order (request query parameter)

Optional, specific to CAR requests, same as car-order from [trustless-gateway].

2.2.8 car-dups (request query parameter)

Optional, specific to CAR requests, same as car-dups from [trustless-gateway].

3. HTTP Response

3.1 Response Status Codes

3.1.1 200 OK

The request succeeded.

If the HTTP method was GET, then data is transmitted in the message body.

If the HTTP method was HEAD, then no body should be sent.

3.1.2 206 Partial Content

Partial Content: range request succeeded.

Returned when requested range of data described by Range header of the request.

3.1.3 301 Moved Permanently

Indicates permanent redirection.

The new, canonical URL is returned in the Location header.

3.1.4 400 Bad Request

A generic client error returned when it is not possible to return a better one. For example, this can be used when the CID is malformed or its codec is unsupported.

3.1.5 404 Not Found

Error to indicate that request was formally correct but either:

• traversal of the requested content path was not possible due to a invalid or missing DAG node, or
• the requested content is not retrievable from this gateway.

Gateways MUST use 404 to signal that content is not available, particularly when the gateway is non recursive, and only provides access to a known dataset, so that it can assess that the requested content is not part of it.

3.1.6 410 Gone

Error to indicate that request was formally correct, but this specific Gateway refuses to return requested data even though it would have normally provided it.

410 is particularly useful to implement deny lists, in order to not serve blocked content. The name of deny list and unique identifier of blocked entries can be provided in the response body.

See: Denylists

See also: 451 Unavailable for Legal Reasons.

3.1.7 412 Precondition Failed

Error to indicate that request was formally correct, but Gateway is unable to return requested data under the additional (usually cache-related) conditions sent by the client.

3.1.7.1 Use with only-if-cached

• Client sends a request with Cache-Control: only-if-cached
• Gateway does not have requested CIDs in local datastore, and is unable to fetch them from other peers due to only-if-cached condition
• Gateway returns status code 412 to the client
  • The code 412 is used instead of 504 because only-if-cached is handled by the gateway itself, moving the error to client error range and avoiding confusing server errors in places like the browser console.

3.1.8 429 Too Many Requests

Error to indicate the client has sent too many requests in a given amount of time.

This error response SHOULD include Retry-After HTTP header to indicate how long the client should wait before making a follow-up request.

3.1.9 451 Unavailable For Legal Reasons

Error to indicate that request was formally correct, but this specific Gateway is unable to return requested data due to legal reasons. Response _SHOULD_include an explanation, as noted in Section 3 of [rfc7725].

See: Denylists

3.1.10 500 Internal Server Error

A generic server error returned when it is not possible to return a better one. An internal server error signals the general unavailability of the gateway.

3.1.11 502 Bad Gateway

Error that indicates that a Gateway was not able to produce response for a known reason: for example, in the case ofrecursive gateways, in the event of failure to find any providers for requested data. 502 indicates that the request can be retried and is not a permanent failure.

This error response SHOULD includeRetry-After HTTP header to indicate how long the client should wait before retrying.

Gateways SHOULD return 404 instead of 502 when the content is known to be unretrievable: for example, when the Gateway isnon-recursive and the content is known to not be available.

3.1.12 504 Gateway Timeout

Error that indicates that the Gateway was not able to produce response under set time limits: for example, when gateway failed to retrieve data from a remote provider. 504 indicates that the request can be retried and is not a permanent failure.

There is no generic timeout, Gateway implementations SHOULD set timeouts based on specific use cases.

This error response SHOULD includeRetry-After HTTP header to indicate how long the client should wait before retrying.

Gateways SHOULD return 404 instead of 504 when the content is known to be unretrievable: for example, when the Gateway isnon-recursive and the content is known to not be available.

3.3 Response Payload

Data sent with HTTP response depends on the type of the requested IPFS resource, and the requested response type.

By default, implicit deserialized response type is based on Accept header and the codec of the resolved CID:

• UnixFS, either dag-pb (0x70) or raw (0x55)
  • File or raw block
    * Bytes representing file/block contents
    * When Range is present, only the requested byte range is returned.
  • Directory
    * Generated HTML with directory index (see additional notes here)
    * When index.html is present, gateway MUST skip generating directory index and return content from index.html instead.
• JSON (0x0200)
  • Bytes representing a JSON file, see application/json.
  • Works exactly the same as raw, but returned Content-Type is application/json
• CBOR (0x51)
  • Bytes representing a CBOR file, see application/cbor
  • Works exactly the same as raw, but returned Content-Type is application/cbor
• DAG-JSON (0x0129)
  • If the Accept header includes text/html, implementation should return a generated HTML with options to download DAG-JSON as-is, or converted to DAG-CBOR.
  • Otherwise, response works exactly the same as raw block, but returned Content-Type is application/vnd.ipld.dag-json
• DAG-CBOR (0x71)
  • If the Accept header includes text/html: implementation should return a generated HTML with options to download DAG-CBOR as-is, or converted to DAG-JSON.
  • Otherwise, response works exactly the same as raw block, but returned Content-Type is application/vnd.ipld.dag-cbor

The following response types require an explicit opt-in, can only be requested with format query parameter or Accept header:

• Raw Block (?format=raw)
  • Opaque bytes, see application/vnd.ipld.raw.
• CAR (?format=car)
  • A CAR file or a stream that contains all blocks required to trustlessly verify the requested content path query, see application/vnd.ipld.car and Section 5 (CAR Responses) at [trustless-gateway].
• TAR (?format=tar)
  • Deserialized UnixFS files and directories as a TAR file or a stream, see [ipip-0288].
• IPNS Record
  • Protobuf bytes representing a verifiable [ipns-record] (multicodec 0x0300)

4. Appendix: notes for implementers

4.1 Content resolution

Content resolution is a process of turning an HTTP request into an IPFS content path, and then traversing it until the content identifier (CID) is found.

4.1.1 Finding the content root

Path Gateway decides what content to serve by taking the path from the URL requested and splitting it into two parts: the CID and the remainder of the path.

The CID provides the starting point, often called content root. The_remainder_ of the path, if present, will be used as instructions to traverse IPLD data, starting from that data which the CID identified.

Note: Other types of gateway may allow for passing CID by other means, such as Host header, changing the rules behind path splitting. (See SUBDOMAIN_GATEWAY.mdand DNSLINK_GATEWAY.md).

4.1.2 Traversing remaining path

After the content root CID is found, the remaining of the path should be traversed and resolved. Depending on the data type, that may occur through UnixFS pathing, or DAG-JSON, and DAG-CBOR pathing.

4.1.3 Traversing through UnixFS

UnixFS is an abstraction over the low level logical DAG-PB pathingfrom IPLD, providing a better user experience:

• Example of UnixFS pathing: /ipfs/cid/dir-name/file-name.txt

For more details regarding DAG-PB pathing, please read the "Path Resolution" section of this document.

4.1.4 Traversing through DAG-JSON and DAG-CBOR

Traversing through DAG-JSON and DAG-CBOR is possible through fields that encode a link:

• DAG-JSON: link are represented as a base encoded CID under the / reserved namespace, see specification.
• DAG-CBOR: links are tagged with CBOR tag 42, indicating that they encode a CID, see specification.

Note: pathing into IPLD Kind other than Link (CID) is not supported at the moment. Implementations should return HTTP 501 Not Implemented when fully resolved content path has any remainder left. This feature may be specified in a future IPIP that introduces data onboarding and IPLD Patch semantics.

4.1.5 Handling traversal errors

Gateway MUST respond with HTTP error when it is not possible to traverse the requested content path:

• 404 Not Found should be returned when the root CID is valid and traversable, but the DAG it represents does not include content path remainder.
  • Error response body should indicate which part of immutable content path (/ipfs/{cid}/path/to/file) is missing
• 400 Bad Request should be returned when the root CID under the ipfs namespace is invalid.
• 500 Internal Server Error can be used for remaining traversal errors, such as domains that cannot be resolved, or IPNS keys that cannot be resolved.

4.2 Best practices for HTTP caching

• Following HTTP Cachingrules around Etag , Cache-Control , If-None-Match and Last-Modifiedshould be produce acceptable cache hits.
• Advanced caching strategies can be built using additional information inX-Ipfs-Path and X-Ipfs-Roots headers.
• Implement support for requests sent withCache-Control: only-if-cached. It allows IPFS-aware HTTP clients to probe and prioritize gateways that already have the data cached, significantly improving retrieval speeds.

4.3 Denylists

Optional, but encouraged.

Implementations are encouraged to support pluggable denylists to allow IPFS node operators to opt into not hosting previously flagged content.

Gateway MUST respond with HTTP error when requested CID is on any of active denylists:

• 410 Gone returned when CID is denied for non-legal reasons, or when the exact reason is unknown
• 451 Unavailable For Legal Reasons returned when denylist indicates that content was blocked on legal basis

Gateway implementation MAY apply some denylists by default as long the gateway operator is able to inspect and modify the list of denylists that are applied.

Examples of public deny lists:

• The Bad Bits Denylist – a list of hashed CIDs that have been flagged for various reasons (copyright violation, malware, etc). Each entry is sha256() hashed so that it can easily be checked given a plaintext CID, but inconvenient to determine otherwise.

4.4 Generated HTML with directory index

While implementations decide on the way HTML directory listing is generated and presented to the user, following below suggestions is advised.

Linking to alternative response types such as CAR and dag-json allows clients to consume directory listings programmatically without the need for parsing HTML.

Directory index response time should not grow with the number of items in a directory. It should be always fast, even when a directory has 10k of items.

The usual optimizations involve:

• Skipping size and type resolution for child UnixFS items, and using Tsizefrom logical format instead, allows gateway to respond much faster, as it no longer need to fetch root nodes of child items.
  • Instead of showing "file size" GUIs should show "IPFS DAG size". This remains useful for quick inspection, but does not require fetching child blocks, making directory listing fast, even with tens of thousands of blocks. Example with 10k items:bafybeiggvykl7skb2ndlmacg2k5modvudocffxjesexlod2pfvg5yhwrqm.
  • Additional information about child nodes, such as exact file size without DAG overhead, can be fetched lazily with JS, but only for items in the browser's viewport.
• Alternative approach is resolving child items, but providing pagination UI.
  • Opening a big directory can return HTTP 302 to the current URL with additional query parameters (?page=0&limit=100), limiting the cost of a single page load.
  • The downside of this approach is that it will always be slower than skipping child block resolution.

4.5 Recursive vs non-recursive gateways

A recursive Gateway is a gateway which generally attempts to fetch content from a third party it does not control by triggering lookups and retrievals. A recursive Gateway may not know in advance whether it can obtain and return a piece of content as the availability of it is out of its control. It may also suggest that clients retry failed requests later via 502 and 504 responses status codes.

A non-recursive Gateway is gateway which accesses a known content-set and, under normal operation conditions, knows with certainty whether content requested can be obtained or not. Non-recursive gateways SHOULD prevent unnecessary retries from clients when the content is known to be unavailable by returning 404.

A. References

[dnslink-gateway]

DNSLink Gateway Specification. Marcin Rataj; Thibault Meunier. 2022-11-09. URL: https://specs.ipfs.tech/http-gateways/dnslink-gateway/

[ipip-0288]

IPIP-0288: TAR Response Format on HTTP Gateways. Henrique Dias; Marcin Rataj. 2022-06-10. URL: https://specs.ipfs.tech/ipips/ipip-0288/

[ipip-0402]

IPIP-0402: Partial CAR Support on Trustless Gateways. Hannah Howard; Adin Schmahmann; Rod Vagg; Marcin Rataj. 2023-04-17. URL: https://specs.ipfs.tech/ipips/ipip-0402/

[ipip-0412]

IPIP-0412: Signaling Block Order in CARs on HTTP Gateways. Marcin Rataj; Jorropo. 2023-05-15. URL: https://specs.ipfs.tech/ipips/ipip-0412/

[ipns-record]

IPNS Record and Protocol. Vasco Santos; Steve Allen; Marcin Rataj; Henrique Dias; Gus Eggert. 2025-02-28. URL: https://specs.ipfs.tech/ipns/ipns-record/

[rfc2119]

Key words for use in RFCs to Indicate Requirement Levels. S. Bradner. IETF. March 1997. Best Current Practice. URL: https://www.rfc-editor.org/rfc/rfc2119

[rfc2181]

Clarifications to the DNS Specification. R. Elz; R. Bush. IETF. July 1997. Proposed Standard. URL: https://www.rfc-editor.org/rfc/rfc2181

[rfc3986]

Uniform Resource Identifier (URI): Generic Syntax. T. Berners-Lee; R. Fielding; L. Masinter. IETF. January 2005. Internet Standard. URL: https://www.rfc-editor.org/rfc/rfc3986

[rfc7725]

An HTTP Status Code to Report Legal Obstacles. T. Bray. IETF. February 2016. Proposed Standard. URL: https://httpwg.org/specs/rfc7725.html

[rfc8187]

Indicating Character Encoding and Language for HTTP Header Field Parameters. J. Reschke. IETF. September 2017. Proposed Standard. URL: https://www.rfc-editor.org/rfc/rfc8187

[rfc9110]

HTTP Semantics. R. Fielding, Ed.; M. Nottingham, Ed.; J. Reschke, Ed.. IETF. June 2022. Internet Standard. URL: https://httpwg.org/specs/rfc9110.html

[rfc9111]

HTTP Caching. R. Fielding, Ed.; M. Nottingham, Ed.; J. Reschke, Ed.. IETF. June 2022. Internet Standard. URL: https://httpwg.org/specs/rfc9111.html

[subdomain-gateway]

Subdomain Gateway Specification. Marcin Rataj; Adrian Lanzafame; Vasco Santos; Oli Evans; Thibault Meunier; Steve Loeppky. 2023-01-28. URL: https://specs.ipfs.tech/http-gateways/subdomain-gateway/

[trustless-gateway]

Trustless Gateway Specification. Marcin Rataj; Henrique Dias; Héctor Sanjuán. 2025-03-06. URL: https://specs.ipfs.tech/http-gateways/trustless-gateway/

B. Acknowledgments

We gratefully acknowledge the following individuals for their valuable contributions, ranging from minor suggestions to major insights, which have shaped and improved this specification.

Editors

Marcin Rataj (Protocol Labs)

Adrian Lanzafame (Protocol Labs)

Vasco Santos (Protocol Labs)

Oli Evans (Protocol Labs)

Henrique Dias (Protocol Labs)

1

1. No definition for term "url".