make-fetch-happen (original) (raw)

make-fetch-happen

npm version license Travis Coverage Status

make-fetch-happen is a Node.js library that wraps minipass-fetch with additional features minipass-fetch doesn't intend to include, including HTTP Cache support, request pooling, proxies, retries, and more!

Install

$ npm install --save make-fetch-happen

Table of Contents

Example

const fetch = require('make-fetch-happen').defaults({ cachePath: './my-cache' // path where cache will be written (and read) })

fetch('https://registry.npmjs.org/make-fetch-happen').then(res => { return res.json() // download the body as JSON }).then(body => { console.log(got ${body.name} from web) return fetch('https://registry.npmjs.org/make-fetch-happen', { cache: 'no-cache' // forces a conditional request }) }).then(res => { console.log(res.status) // 304! cache validated! return res.json().then(body => { console.log(got ${body.name} from cache) }) })

Features

> fetch(uriOrRequest, [opts]) -> Promise<Response>

This function implements most of the fetch API: given a uri string or a Request instance, it will fire off an http request and return a Promise containing the relevant response.

If opts is provided, the minipass-fetch-specific options will be passed to that library. There are also additional options specific to make-fetch-happen that add various features, such as HTTP caching, integrity verification, proxy support, and more.

Example

fetch('https://google.com').then(res => res.buffer())

> fetch.defaults([defaultUrl], [defaultOpts])

Returns a new fetch function that will call make-fetch-happen using defaultUrl and defaultOpts as default values to any calls.

A defaulted fetch will also have a .defaults() method, so they can be chained.

Example

const fetch = require('make-fetch-happen').defaults({ cachePath: './my-local-cache' })

fetch('https://registry.npmjs.org/make-fetch-happen') // will always use the cache

> minipass-fetch options

The following options for minipass-fetch are used as-is:

These other options are modified or augmented by make-fetch-happen:

For more details, see the documentation for minipass-fetch itself.

> make-fetch-happen options

make-fetch-happen augments the minipass-fetch API with additional features available through extra options. The following extra options are available:

> opts.cachePath

A string Path to be used as the cache root for cacache.

NOTE: Requests will not be cached unless their response bodies are consumed. You will need to use one of the res.json(), res.buffer(), etc methods on the response, or drain the res.body stream, in order for it to be written.

The default cache manager also adds the following headers to cached responses:

Using cacache, a call like this may be used to manually fetch the cached entry:

const h = response.headers cacache.get(h.get('x-local-cache'), h.get('x-local-cache-key'))

// grab content only, directly: cacache.get.byDigest(h.get('x-local-cache'), h.get('x-local-cache-hash'))

Example

fetch('https://registry.npmjs.org/make-fetch-happen', { cachePath: './my-local-cache' }) // -> 200-level response will be written to disk

> opts.cache

This option follows the standard fetch API cache option. This option will do nothing if opts.cachePath is null. The following values are accepted (as strings):

(Note: option descriptions are taken from https://fetch.spec.whatwg.org/#http-network-or-cache-fetch)

Example

const fetch = require('make-fetch-happen').defaults({ cachePath: './my-cache' })

// Will error with ENOTCACHED if we haven't already cached this url fetch('https://registry.npmjs.org/make-fetch-happen', { cache: 'only-if-cached' })

// Will refresh any local content and cache the new response fetch('https://registry.npmjs.org/make-fetch-happen', { cache: 'reload' })

// Will use any local data, even if stale. Otherwise, will hit network. fetch('https://registry.npmjs.org/make-fetch-happen', { cache: 'force-cache' })

> opts.cacheAdditionalHeaders

The following headers are always stored in the cache when present:

This option allows a user to store additional custom headers in the cache.

Example

fetch('https://registry.npmjs.org/make-fetch-happen', { cacheAdditionalHeaders: ['my-custom-header'], })

> opts.proxy

A string or new url.URL()-d URI to proxy through. Different Proxy handlers will be used depending on the proxy's protocol.

Additionally, process.env.HTTP_PROXY, process.env.HTTPS_PROXY, andprocess.env.PROXY are used if present and no opts.proxy value is provided.

(Pending) process.env.NO_PROXY may also be configured to skip proxying requests for all, or specific domains.

Example

fetch('https://registry.npmjs.org/make-fetch-happen', { proxy: 'https://corporate.yourcompany.proxy:4445' })

fetch('https://registry.npmjs.org/make-fetch-happen', { proxy: { protocol: 'https:', hostname: 'corporate.yourcompany.proxy', port: 4445 } })

> opts.noProxy

If present, should be a comma-separated string or an array of domain extensions that a proxy should not be used for.

This option may also be provided through process.env.NO_PROXY.

> opts.ca, opts.cert, opts.key, opts.strictSSL

These values are passed in directly to the HTTPS agent and will be used for both proxied and unproxied outgoing HTTPS requests. They mostly correspond to the same options the https module accepts, which will be themselves passed totls.connect(). opts.strictSSL corresponds to rejectUnauthorized.

> opts.localAddress

Passed directly to http and https request calls. Determines the local address to bind to.

> opts.maxSockets

Default: 15

Maximum number of active concurrent sockets to use for the underlying Http/Https/Proxy agents. This setting applies once per spawned agent.

15 is probably a pretty good value for most use-cases, and balances speed with, uh, not knocking out people's routers. 🤓

> opts.retry

An object that can be used to tune request retry settings. Retries will only be attempted on the following conditions:

The following are worth noting as explicitly not retried:

If opts.retry is false, it is equivalent to {retries: 0}

If opts.retry is a number, it is equivalent to {retries: num}

The following retry options are available if you want more control over it:

For details on what each of these do, refer to the retry documentation.

Example

fetch('https://flaky.site.com', { retry: { retries: 10, randomize: true } })

fetch('http://reliable.site.com', { retry: false })

fetch('http://one-more.site.com', { retry: 3 })

> opts.onRetry

A function called with the response or error which caused the retry whenever one is attempted.

Example

fetch('https://flaky.site.com', { onRetry(cause) { console.log('we will retry because of', cause) } })

> opts.integrity

Matches the response body against the given Subresource Integrity metadata. If verification fails, the request will fail with an EINTEGRITY error.

integrity may either be a string or an ssri Integrity-like.

Example

fetch('https://registry.npmjs.org/make-fetch-happen/-/make-fetch-happen-1.0.0.tgz', { integrity: 'sha1-o47j7zAYnedYFn1dF/fR9OV3z8Q=' }) // -> ok

fetch('https://malicious-registry.org/make-fetch-happen/-/make-fetch-happen-1.0.0.tgz', { integrity: 'sha1-o47j7zAYnedYFn1dF/fR9OV3z8Q=' }) // Error: EINTEGRITY