Undici

An HTTP/1.1 client, written from scratch for Node.js

HTTP client • Fetch • AJAX
nodejs/undicinodejs.github.io/undiciundici

README

undici


Node CI  js-standard-style npm version codecov

A HTTP/1.1 client, written from scratch for Node.js.

Undici means eleven in Italian. 1.1 -> 11 -> Eleven -> Undici.

It is also a Stranger Things reference.

<!--
Picture of Eleven
-->

Install


  1. ```
  2. npm i undici
  3. ```

Benchmarks


Machine: AMD EPYC 7502P

Node 15
  1. ```
  2. http - keepalive x 12,028 ops/sec ±2.60% (265 runs sampled)
  3. undici - pipeline x 31,321 ops/sec ±0.77% (276 runs sampled)
  4. undici - request x 36,612 ops/sec ±0.71% (277 runs sampled)
  5. undici - stream x 41,291 ops/sec ±0.90% (268 runs sampled)
  6. undici - dispatch x 47,319 ops/sec ±1.17% (263 runs sampled)
  7. ```

The benchmark is a simple hello world example using a
single unix socket with pipelining.

Quick Start


  1. ``` js
  2. import { request } from 'undici'

  4. const {
  5.   statusCode,
  6.   headers,
  7.   trailers,
  8.   body
  9. } = await request('http://localhost:3000/foo')

  11. console.log('response received', statusCode)
  12. console.log('headers', headers)

  14. for await (const data of body) {
  15.   console.log('data', data)
  16. }

  18. console.log('trailers', trailers)
  19. ```

API


new undici.Client(url, opts)


A basic HTTP/1.1 client, mapped on top of a single TCP/TLS connection. Pipelining is disabled
by default.

Requests are not guaranteeed to be dispatched in order of invocation.

url can be a string or a [URL](https://nodejs.org/api/url.html#url_class_url) object.
It should only include the protocol, hostname, and port.

Options:

- socketPath: String|Null, an IPC endpoint, either Unix domain socket or Windows named pipe.
  Default: null.

- keepAliveTimeout: Number, the timeout after which a socket without active requests
  will time out. Monitors time between activity on a connected socket.
  This value may be overridden by keep-alive hints from the server.
  Default: 4e3 milliseconds (4s).

- keepAliveMaxTimeout: Number, the maximum allowed keepAliveTimeout when overridden by
  keep-alive hints from the server.
  Default: 600e3 milliseconds (10min).

- keepAliveTimeoutThreshold: Number, a number subtracted from server keep-alive hints
  when overriding keepAliveTimeout to account for timing inaccuracies caused by e.g.
  transport latency.
  Default: 1e3 milliseconds (1s).

- headersTimeout: Number, the timeout after which a request will time out, in
  milliseconds. Monitors time between receiving complete headers.
  Use 0 to disable it entirely. Default: 30e3 milliseconds (30s).

- bodyTimeout: Number, the timeout after which a request will time out, in
  milliseconds. Monitors time between receiving body data.
  Use 0 to disable it entirely. Default: 30e3 milliseconds (30s).

- pipelining: Number, the amount of concurrent requests to be sent over the
  single TCP/TLS connection according to RFC7230.
  Carefully consider your workload and environment before enabling concurrent requests
  as pipelining may reduce performance if used incorrectly. Pipelining is sensitive
  to network stack settings as well as head of line blocking caused by e.g. long running requests.
  Set to 0 to disable keep-alive connections.
  Default: 1.

- tls: Object|Null, an options object which in the case of https will be passed to
  [tls.connect](https://nodejs.org/api/tls.html#tls_tls_connect_options_callback).
  Default: null.

- maxHeaderSize: Number, the maximum length of request headers in bytes.
  Default: 16384 (16KiB).

client.request(opts[, callback(err, data)]): Promise|Void


Performs a HTTP request.

Options:

path: String
method: String
opaque: Any
body: String|Buffer|Uint8Array|stream.Readable|Null
  Default: null.
headers: Object|Array|Null, an object with header-value pairs or an array with header-value pairs bi-indexed (['header1', 'value1', 'header2', 'value2']).
  Default: null.
signal: AbortSignal|EventEmitter|Null
  Default: null.
idempotent: Boolean, whether the requests can be safely retried or not.
  If false the request won't be sent until all preceding
  requests in the pipeline has completed.
  Default: true if method is HEAD or GET.

Headers are represented by an object like this:

  1. ``` js
  2. {
  3.   'content-length': '123',
  4.   'content-type': 'text/plain',
  5.   connection: 'keep-alive',
  6.   host: 'mysite.com',
  7.   accept: '*/*'
  8. }
  9. ```

Or an array like this:

  1. ``` js
  2. [
  3.   'content-length', '123',
  4.   'content-type', 'text/plain',
  5.   'connection', 'keep-alive',
  6.   'host', 'mysite.com',
  7.   'accept', '*/*'
  8. ]
  9. ```

Keys are lowercased. Values are not modified.
If you don't specify a host header, it will be derived from the url of the client instance.

The data parameter in callback is defined as follow:

statusCode: Number
opaque: Any
headers: Object, an object where all keys have been lowercased.
trailers: Object, an object where all keys have been lowercased. This object start out
  as empty and will be mutated to contain trailers after body has emitted 'end'.
body: stream.Readable response payload. A user must
  either fully consume or destroy the body unless there is an error, or no further requests
  will be processed.

Returns a promise if no callback is provided.

Example:

  1. ``` js
  2. const { Client } = require('undici')
  3. const client = new Client(`http://localhost:3000`)

  5. client.request({
  6.   path: '/',
  7.   method: 'GET'
  8. }, function (err, data) {
  9.   if (err) {
  10.     // handle this in some way!
  11.     return
  12.   }

  14.   const {
  15.     statusCode,
  16.     headers,
  17.     trailers,
  18.     body
  19.   } = data

  21.   console.log('response received', statusCode)
  22.   console.log('headers', headers)

  24.   body.setEncoding('utf8')
  25.   body.on('data', console.log)
  26.   body.on('end', () => {
  27.     console.log('trailers', trailers)
  28.   })

  30.   client.close()
  31. })
  32. ```

Non-idempotent requests will not be pipelined in order
to avoid indirect failures.

Idempotent requests will be automatically retried if
they fail due to indirect failure from the request
at the head of the pipeline. This does not apply to
idempotent requests with a stream request body.

Aborting a request

A request can be aborted using either an AbortController or an EventEmitter.
To use AbortController in Node.js versions earlier than 15, you will need to
install a shim - npm i abort-controller.

  1. ``` js
  2. const { Client } = require('undici')

  4. const client = new Client('http://localhost:3000')
  5. const abortController = new AbortController()

  7. client.request({
  8.   path: '/',
  9.   method: 'GET',
  10.   signal: abortController.signal
  11. }, function (err, data) {
  12.   console.log(err) // RequestAbortedError
  13.   client.close()
  14. })

  16. abortController.abort()
  17. ```

Alternatively, any EventEmitter that emits an 'abort' event may be used as an abort controller:

  1. ``` js
  2. const EventEmitter = require('events')
  3. const { Client } = require('undici')

  5. const client = new Client('http://localhost:3000')
  6. const ee = new EventEmitter()

  8. client.request({
  9.   path: '/',
  10.   method: 'GET',
  11.   signal: ee
  12. }, function (err, data) {
  13.   console.log(err) // RequestAbortedError
  14.   client.close()
  15. })

  17. ee.emit('abort')
  18. ```

Destroying the request or response body will have the same effect.

client.stream(opts, factory(data)[, callback(err)]): Promise|Void


A faster version of [request][request].

Unlike [request][request] this method expects factory
to return a [Writable](https://nodejs.org/api/stream.html#stream_class_stream_writable) which the response will be
written to. This improves performance by avoiding
creating an intermediate [Readable](https://nodejs.org/api/stream.html#stream_readable_streams) when the user
expects to directly pipe the response body to a
[Writable](https://nodejs.org/api/stream.html#stream_class_stream_writable).

Options:

... same as [client.request(opts[, callback])][request].

The data parameter in factory is defined as follow:

statusCode: Number
headers: Object, an object where all keys have been lowercased.
opaque: Any

The data parameter in callback is defined as follow:

opaque: Any
trailers: Object, an object where all keys have been lowercased.

Returns a promise if no callback is provided.

  1. ``` js
  2. const { Client } = require('undici')
  3. const client = new Client(`http://localhost:3000`)
  4. const fs = require('fs')

  6. client.stream({
  7.   path: '/',
  8.   method: 'GET',
  9.   opaque: filename
  10. }, ({ statusCode, headers, opaque: filename }) => {
  11.   console.log('response received', statusCode)
  12.   console.log('headers', headers)
  13.   return fs.createWriteStream(filename)
  14. }, (err) => {
  15.   if (err) {
  16.     console.error('failure', err)
  17.   } else {
  18.     console.log('success')
  19.   }
  20. })
  21. ```

opaque makes it possible to avoid creating a closure
for the factory method:

  1. ``` js
  2. function (req, res) {
  3.    return client.stream({ ...opts, opaque: res }, proxy)
  4. }
  5. ```

Instead of:

  1. ``` js
  2. function (req, res) {
  3.    return client.stream(opts, (data) => {
  4.      // Creates closure to capture `res`.
  5.      proxy({ ...data, opaque: res })
  6.    }
  7. }
  8. ```

client.pipeline(opts, handler(data)): Duplex


For easy use with [stream.pipeline](https://nodejs.org/api/stream.html#stream_stream_pipeline_source_transforms_destination_callback).

Options:

... same as [client.request(opts, callback)][request].
objectMode: Boolean, true if the handler will return an object stream.
  Default: false

The data parameter in handler is defined as follow:

statusCode: Number
headers: Object, an object where all keys have been lowercased.
opaque: Any
body: stream.Readable response payload. A user must
  either fully consume or destroy the body unless there is an error, or no further requests
  will be processed.

handler should return a [Readable](https://nodejs.org/api/stream.html#stream_class_stream_readable) from which the result will be
read. Usually it should just return the body argument unless
some kind of transformation needs to be performed based on e.g.
headers or statusCode.

The handler should validate the response and save any
required state. If there is an error it should be thrown.

Returns a Duplex which writes to the request and reads from
the response.

  1. ``` js
  2. const { Client } = require('undici')
  3. const client = new Client(`http://localhost:3000`)
  4. const fs = require('fs')
  5. const stream = require('stream')

  7. stream.pipeline(
  8.   fs.createReadStream('source.raw'),
  9.   client.pipeline({
  10.     path: '/',
  11.     method: 'PUT',
  12.   }, ({ statusCode, headers, body }) => {
  13.     if (statusCode !== 201) {
  14.       throw new Error('invalid response')
  15.     }

  17.     if (isZipped(headers)) {
  18.       return pipeline(body, unzip(), () => {})
  19.     }

  21.     return body
  22.   }),
  23.   fs.createWriteStream('response.raw'),
  24.   (err) => {
  25.     if (err) {
  26.       console.error('failed')
  27.     } else {
  28.       console.log('succeeded')
  29.     }
  30.   }
  31. )
  32. ```

client.upgrade(opts[, callback(err, data)]): Promise|Void


Upgrade to a different protocol.

Options:

path: String
opaque: Any
method: String
  Default: GET
headers: Object|Null, an object with header-value pairs.
  Default: null
signal: AbortSignal|EventEmitter|Null.
  Default: null
protocol: String, a string of comma separated protocols, in descending preference order.
  Default: Websocket.

The data parameter in callback is defined as follow:

headers: Object, an object where all keys have been lowercased.
socket: Duplex
opaque

Returns a promise if no callback is provided.

client.connect(opts[, callback(err, data)]): Promise|Void


Starts two-way communications with the requested resource.

Options:

path: String
opaque: Any
headers: Object|Null, an object with header-value pairs.
  Default: null
signal: AbortSignal|EventEmitter|Null.
  Default: null

The data parameter in callback is defined as follow:

statusCode: Number
headers: Object, an object where all keys have been lowercased.
socket: Duplex
opaque: Any

Returns a promise if no callback is provided.

client.dispatch(opts, handler): Void


This is the low level API which all the preceding APIs are implemented on top of.

This API is expected to evolve through semver-major versions and is less stable
than the preceding higher level APIs. It is primarily intended for library developers
who implement higher level APIs on top of this.

Multiple handler methods may be invoked in the same tick.

Options:

path: String
method: String
body: String|Buffer|Uint8Array|stream.Readable|Null
  Default: null.
headers: Object|Null, an object with header-value pairs.
  Default: null.
idempotent: Boolean, whether the requests can be safely retried or not.
  If false the request won't be sent until all preceding
  requests in the pipeline has completed.
  Default: true if method is HEAD or GET.

The handler parameter is defined as follow:

onConnect(abort), invoked before request is dispatched on socket.
  May be invoked multiple times when a request is retried when the request at the head of the pipeline fails.
  abort(): Void, abort request.
onUpgrade(statusCode, headers, socket): Void, invoked when request is upgraded  either due to a Upgrade header or CONNECT method.
  statusCode: Number
  headers: Array|Null
  socket: Duplex
onHeaders(statusCode, headers, resume): Boolean, invoked when statusCode and headers have been received.
  May be invoked multiple times due to 1xx informational headers.
  statusCode: Number
  headers: Array|Null, an array of key-value pairs. Keys are not automatically lowercased.
  resume(): Void, resume onData after returning false.
onData(chunk): Boolean, invoked when response payload data is received.
  chunk: Buffer
onComplete(trailers): Void, invoked when response payload and trailers have been received and the request has completed.
  trailers: Array|Null
onError(err): Void, invoked when an error has occurred.
  err: Error

The caller is responsible for handling the body argument, in terms of 'error' events and destroy():ing up until
the onConnect handler has been invoked.

client.close([callback]): Promise|Void


Closes the client and gracefully waits for enqueued requests to
complete before invoking the callback.

Returns a promise if no callback is provided.

client.destroy([err][, callback]): Promise|Void


Destroy the client abruptly with the given err. All the pending and running
requests will be asynchronously aborted and error. Waits until socket is closed
before invoking the callback. Since this operation is asynchronously dispatched
there might still be some progress on dispatched requests.

Returns a promise if no callback is provided.

client.url: URL


Returns url passed to undici.Pool(url, opts).

client.pipelining: Number


Property to get and set the pipelining factor.

client.pending: Number


Number of queued requests.

client.running: Number


Number of inflight requests.

client.size: Number


Number of pending and running requests.

client.connected: Number


Thruthy if the client has an active connection. The client will lazily
create a connection when it receives a request and will destroy it
if there is no activity for the duration of the timeout value.

client.busy: Boolean


True if pipeline is saturated or blocked. Indicates whether dispatching
further requests is meaningful.

client.closed: Boolean


True after client.close() has been called.

client.destroyed: Boolean


True after client.destroyed() has been called or client.close() has been
called and the client shutdown has completed.

Events


'drain', emitted when pipeline is no longer fully
  saturated.

'connect', emitted when a socket has been created and
  connected. The first argument is the  Client instance.
  The client will connect once client.size > 0.

'disconnect', emitted when socket has disconnected. The
  first argument of the event is the error which caused the
  socket to disconnect. The second argument is the
   Client instance. The client will reconnect if or once
  client.size > 0.

new undici.Pool(url, opts)


A pool of [Client][] connected to the same upstream target.
Implements the same api as [Client][].

Requests are not guaranteeed to be dispatched in order of invocation.

Options:

... same as [Client][].
connections, the number of clients to create.
  Default 10.

new undici.Agent(opts)


opts undici.Pool.options - options passed through to Pool constructor

Returns: Agent

Returns a new Agent instance for use with pool based requests or the following top-level methods request, pipeline, and stream.

agent.get(origin): Pool


origin string - A pool origin to be retrieved from the Agent.

This method retrieves Pool instances from the Agent. If the pool does not exist it is automatically added. You do not need to manually close these pools as they are automatically removed using a WeakCache based on WeakRef and FinalizationRegistry.

The following methods request, pipeline, and stream utilize this feature.

agent.close(): Promise


Returns a Promise.all operation closing all of the pool instances in the Agent instance. This calls pool.close under the hood.

agent.destroy(): Promise


Returns a Promise.all operation destroying all of the pool instances in the Agent instance. This calls pool.destroy under the hood.

undici.setGlobalAgent(agent)


agent Agent

Sets the global agent used by request, pipeline, and stream methods.

The default global agent creates undici.Pools with no max number of
connections.

The agent must only implement the Agent API; not necessary extend from it.

undici.request(url[, opts]): Promise


url string | URL | object
opts { agent: Agent } & client.request.opts

url may contain path. opts may not contain path. opts.method is GET by default.

Calls pool.request(opts) on the pool returned from either the globalAgent (see setGlobalAgent) or the agent passed to theopts argument.

Returns a promise with the result of the request method.

undici.stream(url, opts, factory): Promise


url string | URL | object
opts { agent: Agent } & client.stream.opts
factory client.stream.factory

url may contain path. opts may not contain path.

See client.stream for details on theopts and factory arguments.

Calls pool.stream(opts, factory) on the pool returned from either the globalAgent (see setGlobalAgent) or the agent passed to theopts argument.

Result is returned in the factory function. See client.stream for more details.

undici.pipeline(url, opts, handler): Duplex


url string | URL | object
opts { agent: Agent } & client.pipeline.opts
handler client.pipeline.handler

url may contain path. opts may not contain path.

See client.pipeline for details on theopts and handler arguments.

Calls pool.pipeline(opts, factory) on the pool returned from either the globalAgent (see setGlobalAgent) or the agent passed to theopts argument.

See client.pipeline for more details.

client.upgrade(opts[, callback(err, data)]): Promise|Void


url string | URL | object
opts { agent: Agent } & client.upgrade.opts

url may contain path. opts may not contain path.

client.connect(opts[, callback(err, data)]): Promise|Void


url string | URL | object
opts { agent: Agent } & client.connect.opts

url may contain path. opts may not contain path.

undici.errors


Undici exposes a variety of error objects that you can use to enhance your error handling.
You can find all the error objects inside the errors key.

  1. ``` js
  2. const { errors } = require('undici')
  3. ```

ErrorErrorDescription
-----------------------------|-----------------------------------|------------------------------------------------|
`InvalidArgumentError``UND_ERR_INVALID_ARG`passed
`InvalidReturnValueError``UND_ERR_INVALID_RETURN_VALUE`returned
`RequestAbortedError``UND_ERR_ABORTED`the
`ClientDestroyedError``UND_ERR_DESTROYED`trying
`ClientClosedError``UND_ERR_CLOSED`trying
`SocketError``UND_ERR_SOCKET`there
`NotSupportedError``UND_ERR_NOT_SUPPORTED`encountered
`ContentLengthMismatchError``UND_ERR_CONTENT_LENGTH_MISMATCH`|
`InformationalError``UND_ERR_INFO`expected
`TrailerMismatchError``UND_ERR_TRAILER_MISMATCH`trailers

Specification Compliance


This section documents parts of the HTTP/1.1 specification which Undici does
not support or does not fully implement.

Expect


Undici does not support the Expect request header field. The request
body is  always immediately sent and the 100 Continue response will be
ignored.

Refs: https://tools.ietf.org/html/rfc7231#section-5.1.1

Pipelining


Uncidi will only use pipelining if configured with a pipelining factor
greater than 1.

Undici always assumes that connections are persistent and will immediately
pipeline requests, without checking whether the connection is persistent.
Hence, automatic fallback to HTTP/1.0 or HTTP/1.1 without pipelining is
not supported.

Undici will immediately pipeline when retrying requests afters a failed
connection. However, Undici will not retry the first remaining requests in
the prior pipeline and instead error the corresponding callback/promise/stream.

Refs: https://tools.ietf.org/html/rfc2616#section-8.1.2.2
Refs: https://tools.ietf.org/html/rfc7230#section-6.3.2

Collaborators


* [__Robert Nagy__](https://github.com/ronag),

License


MIT

[Client]: #client
[request]: #request
[stream]: #stream
[pipeline]: #pipeline
[upgrade]: #upgrade
[connect]: #connect
[dispatch]: #dispatch