Retrieving a website
Using HTTP POST
Progress reporting
SSL/TLS support
Timeouts
Proxy
Redirects
Imports
Types
Consts
Procs
Exports

This module implements a simple HTTP client that can be used to retrieve webpages and other data.

Warning: Validate untrusted inputs: URI parsers and getters are not detecting malicious URIs.

Retrieving a website

This example uses HTTP GET to retrieve http://google.com:

import std/httpclient
var client = newHttpClient()
try:
  echo client.getContent("http://google.com")
finally:
  client.close()

The same action can also be performed asynchronously, simply use the AsyncHttpClient:

import std/[asyncdispatch, httpclient]
proc asyncProc(): Future[string] {.async.} =
  var client = newAsyncHttpClient()
  try:
    return await client.getContent("http://google.com")
  finally:
    client.close()
echo waitFor asyncProc()

The functionality implemented by HttpClient and AsyncHttpClient is the same, so you can use whichever one suits you best in the examples shown here.

Note: You need to run asynchronous examples in an async proc otherwise you will get an Undeclared identifier: ‘await’ error.

Note: An asynchronous client instance can only deal with one request at a time. To send multiple requests in parallel, use multiple client instances.

Using HTTP POST

This example demonstrates the usage of the W3 HTML Validator, it uses multipart/form-data as the Content-Type to send the HTML to be validated to the server.

var client = newHttpClient()
var data = newMultipartData()
data["output"] = "soap12"
data["uploaded_file"] = ("test.html", "text/html",
  "<html><head></head><body><p>test</p></body></html>")
try:
  echo client.postContent("http://validator.w3.org/check", multipart=data)
finally:
  client.close()

To stream files from disk when performing the request, use addFiles.

Note: This will allocate a new Mimetypes database every time you call it, you can pass your own via the mimeDb parameter to avoid this.

let mimes = newMimetypes()
var client = newHttpClient()
var data = newMultipartData()
data.addFiles({"uploaded_file": "test.html"}, mimeDb = mimes)
try:
  echo client.postContent("http://validator.w3.org/check", multipart=data)
finally:
  client.close()

You can also make post requests with custom headers. This example sets Content-Type to application/json and uses a json object for the body

import std/[httpclient, json]
let client = newHttpClient()
client.headers = newHttpHeaders({ "Content-Type": "application/json" })
let body = %*{
    "data": "some text"
}
try:
  let response = client.request("http://some.api", httpMethod = HttpPost, body = $body)
  echo response.status
finally:
  client.close()

Progress reporting

You may specify a callback procedure to be called during an HTTP request. This callback will be executed every second with information about the progress of the HTTP request.

import std/[asyncdispatch, httpclient]
proc onProgressChanged(total, progress, speed: BiggestInt) {.async.} =
  echo("Downloaded ", progress, " of ", total)
  echo("Current rate: ", speed div 1000, "kb/s")
proc asyncProc() {.async.} =
  var client = newAsyncHttpClient()
  client.onProgressChanged = onProgressChanged
  try:
    discard await client.getContent("http://speedtest-ams2.digitalocean.com/100mb.test")
  finally:
    client.close()
waitFor asyncProc()

If you would like to remove the callback simply set it to nil.

client.onProgressChanged = nil

Warning: The total reported by httpclient may be 0 in some cases.

SSL/TLS support

This requires the OpenSSL library. Fortunately it’s widely used and installed on many operating systems. httpclient will use SSL automatically if you give any of the functions a url with the https schema, for example: https://github.com/.

You will also have to compile with ssl defined like so: nim c -d:ssl ….

Certificate validation is performed by default.

A set of directories and files from the ssl_certs module are scanned to locate CA certificates.

Example of setting SSL verification parameters in a new client:

import httpclient
var client = newHttpClient(sslContext=newContext(verifyMode=CVerifyPeer))

There are three options for verify mode:

CVerifyNone: certificates are not verified;
CVerifyPeer: certificates are verified;
CVerifyPeerUseEnvVars: certificates are verified and the optional environment variables SSL_CERT_FILE and SSL_CERT_DIR are also used to locate certificates

See newContext to tweak or disable certificate validation.

Timeouts

Currently only the synchronous functions support a timeout. The timeout is measured in milliseconds, once it is set any call on a socket which may block will be susceptible to this timeout.

It may be surprising but the function as a whole can take longer than the specified timeout, only individual internal calls on the socket are affected. In practice this means that as long as the server is sending data an exception will not be raised, if however data does not reach the client within the specified timeout a TimeoutError exception will be raised.

Here is how to set a timeout when creating an HttpClient instance:

import std/httpclient
let client = newHttpClient(timeout = 42)

Proxy

A proxy can be specified as a param to any of the procedures defined in this module. To do this, use the newProxy constructor. Unfortunately, only basic authentication is supported at the moment.

Some examples on how to configure a Proxy for HttpClient:

import std/httpclient
let myProxy = newProxy("http://myproxy.network")
let client = newHttpClient(proxy = myProxy)

Use proxies with basic authentication:

import std/httpclient
let myProxy = newProxy("http://myproxy.network", auth="user:password")
let client = newHttpClient(proxy = myProxy)

Get Proxy URL from environment variables:

import std/httpclient
var url = ""
try:
  if existsEnv("http_proxy"):
    url = getEnv("http_proxy")
  elif existsEnv("https_proxy"):
    url = getEnv("https_proxy")
except ValueError:
  echo "Unable to parse proxy from environment variables."
let myProxy = newProxy(url = url)
let client = newHttpClient(proxy = myProxy)

Redirects

The maximum redirects can be set with the maxRedirects of int type, it specifies the maximum amount of redirects to follow, it defaults to 5, you can set it to 0 to disable redirects.

Here you can see an example about how to set the maxRedirects of HttpClient:

import std/httpclient
let client = newHttpClient(maxRedirects = 0)

Imports

since, net, strutils, uri, parseutils, base64, os, mimetypes, math, random, httpcore, times, tables, streams, monotimes, asyncnet, asyncdispatch, asyncfile, nativesockets

Types

AsyncHttpClient = HttpClientBase[AsyncSocket]