Making REST API Requests in Golang using the HTTP Client

In most web applications, we need to make HTTP requests to external services. For example, we might need to make a request to a third-party API to fetch some data. Or we might need to make a request to our own API to create, update, or delete some data.

In this post, we will learn how to make REST API requests (like GET, POST, etc) in Golang using the HTTP client. We will also learn about some advanced features of the HTTP client, like setting headers, and timeouts.

The HTTP Client Library

Most of the functionality for making HTTP requests is provided by the net/http package. This package provides a Client type that we can use to make HTTP requests.

Requests and Responses

Whenever we make an HTTP request, we need to specify the HTTP method (GET, POST, etc), the URL, and the request body (if any).

In return, we get an HTTP response. The response contains the response body, the status code, and some other metadata. In Go, the response is represented by the Response type.

request response

Making a GET Request

For certain common request methods, like GET, and POST, the http package provides helper methods that make it easier to make requests.

For example, to make a GET request, we can use the http.Get method.

func main() {
	url := "http://www.example.com"
	resp, err := http.Get(url)
	if err != nil {
		// we will get an error at this stage if the request fails, such as if the
		// requested URL is not found, or if the server is not reachable.
		log.Fatal(err)
	}
	defer resp.Body.Close()

	// if we want to check for a specific status code, we can do so here
	// for example, a successful request should return a 200 OK status
	if resp.StatusCode != http.StatusOK {
		// if the status code is not 200, we should log the status code and the
		// status string, then exit with a fatal error
		log.Fatalf("status code error: %d %s", resp.StatusCode, resp.Status)
	}

	// print the response
	data, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(string(data))
}

The output of this is the HTML body from the example.com webpage (Try it here):

<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
# ...
# ... #remaining output truncated for brevity
# ...

Making a POST Request

To make a POST request, we can use the http.Post method. This method takes in the URL, the content type, and the request body as parameters.

The request body allows us to send data to the server, which is not possible with a GET request. Let’s see how we can send plain text data as the request body:

func main() {
	// we will run an HTTP server locally to test the POST request
	url := "http://localhost:3000/"

	// create post body
	body := strings.NewReader("This is the request body.")

	resp, err := http.Post(url, "text/plain", body)
	if err != nil {
		// we will get an error at this stage if the request fails, such as if the
		// requested URL is not found, or if the server is not reachable.
		log.Fatal(err)
	}
	defer resp.Body.Close()

	// print the status code
	fmt.Println("Status:", resp.Status)
}

To test this request, we will need to run an HTTP server locally. We will do this using the http-echo-server package.
After installation, we can run the command http-echo-server 3000 on a new terminal to start the server on port 3000.
This echo server will return a 200 (OK) response for any request sent to http://localhost:3000 and print the request details, like the method, path, headers and body.

Output (Try it here):

Status: 200 OK

Output from the echo server, showing details of the request:

--> POST / HTTP/1.1
--> Host: localhost:3000
--> User-Agent: Go-http-client/1.1
--> Content-Length: 25
--> Content-Type: text/plain
--> Accept-Encoding: gzip
-->
--> This is the request body.

Sending JSON Data

Sending JSON data is a common use case for POST requests. To send JSON data, we can use the http.Post method, but we need to make some changes:

We need to set the content type header to application/json - this tells the server that the request body is a JSON string
The request body needs to be a JSON string - we can use the JSON standard library to convert a Go struct to a JSON string

Let’s look at the previous example, but this time, we will send a JSON string as the request body:

// Person is a struct that represents the data we will send in the request body
type Person struct {
	Name string
	Age  int
}

func main() {
	url := "http://localhost:3000"

	// create post body using an instance of the Person struct
	p := Person{
		Name: "John Doe",
		Age:  25,
	}
	// convert p to JSON data
	jsonData, err := json.Marshal(p)
	if err != nil {
		log.Fatal(err)
	}

	// We can set the content type here
	resp, err := http.Post(url, "application/json", bytes.NewReader(jsonData))
	if err != nil {
		log.Fatal(err)
	}
	defer resp.Body.Close()

	fmt.Println("Status:", resp.Status)
}

Output (Try it here):

Status: 200 OK

Output from the echo server:

--> POST / HTTP/1.1
--> Host: localhost:3000
--> User-Agent: Go-http-client/1.1
--> Content-Length: 28
--> Content-Type: application/json
--> Accept-Encoding: gzip
-->
--> {"Name":"John Doe","Age":25}

As we can see, the server received the stringified JSON data as the request body, and the Content-Type header value is set to application/json.

Parsing JSON Responses

In the previous example, we sent JSON data as the request body. But what if we want to get JSON data from the response?

We can use the JSON standard library to parse JSON data from the response body into a Go struct. Let’s look at an example:

// parse the response
responsePerson := Person{}
err = json.NewDecoder(resp.Body).Decode(&responsePerson)
if err != nil {
	log.Fatal(err)
}

Making PUT, PATCH, and DELETE Requests

Unlike the http.Get and http.Post methods, there are no helper methods for PUT, PATCH, and DELETE requests.

Instead, we need to use the http.NewRequest method to create a new request, and then use the http.Client.Do method to send the request:

func main() {
	// declare the request url and body
	url := "http://localhost:3000/some/path"
	body := strings.NewReader("This is the request body.")

	// we can set a custom method here, like http.MethodPut
	// or http.MethodDelete, http.MethodPatch, etc.
	req, err := http.NewRequest(http.MethodPut, url, body)
	if err != nil {
		log.Fatal(err)
	}

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		// we will get an error at this stage if the request fails, such as if the
		// requested URL is not found, or if the server is not reachable.
		log.Fatal(err)
	}
	defer resp.Body.Close()

	// print the status code
	fmt.Println("Status:", resp.Status)
}

Output (Try it here):

Status: 200 OK

Output from the echo server:

--> PUT /some/path HTTP/1.1
--> Host: localhost:3000
--> User-Agent: Go-http-client/1.1
--> Transfer-Encoding: chunked
--> Accept-Encoding: gzip
-->
--> 19
--> This is the request body.
--> 0

The numbers 19 and 0 are a part of the chunked transfer encoding, which is set by default for an http.Request instance. The server uses this to know when the request body has ended.

Setting Headers

Headers are a way to send metadata along with the request, as well as the response. They are used for a wide variety of purposes. For example:

The Content-Type header that was set in the previous examples tells the server what type of data the request body contains.
The Authorization header is used to send authentication information to the server.
The User-Agent header is used to identify the client making the request.
We can even set custom headers for application-specific purposes.

These are just some examples, but there are many more standardized headers used for web applications.

We can use the http.Header type to set headers for our requests, and read headers from the response. The underlying data structure is a map of strings to slices of strings, where each key is the header name, and the value is a slice of strings containing the header values.

Let’s look at an example where we set a custom header that is sent with our request, and also read the headers from the response:

func main() {
	url := "http://localhost:3000"
	req, err := http.NewRequest("GET", url, nil)
	if err != nil {
		log.Fatal(err)
	}
	// add a custom header to the request
	// here we specify the header name and value as arguments
	req.Header.Add("X-Custom-Header", "custom-value")

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		log.Fatal(err)
	}
	defer resp.Body.Close()

	// print all the response headers
	fmt.Println("Response headers:")
	for k, v := range resp.Header {
		fmt.Printf("%s: %s\n", k, v)
	}
}

Output (Try it here):

Response headers:
Date: [Sun Jul 16 2023 19:54:59 GMT+0530 (India Standard Time)]
Content-Type: [text/plain]
Access-Control-Allow-Origin: [*]

Output from the echo server:

--> GET / HTTP/1.1
--> Host: localhost:3000
--> User-Agent: Go-http-client/1.1
--> X-Custom-Header: custom-value
--> Accept-Encoding: gzip

We can see that the X-Custom-Header header was sent with the request.

Setting Timeouts

When making HTTP requests, we should always set a timeout. This ensures that our application does not hang indefinitely if the server is not reachable, or if the server takes too long to respond.

For example, if we make a request to example.com, but the server is non-responsive, the goroutine that is making the request will go on forever. This can cause a memory leaks, since the resources associated with the goroutine will never be freed.

We can set the Timeout field of the http.Client type to set a timeout for all requests made using that client.

In our examples, we’ve been using the http.DefaultClient instance, which is a package level variable that is initialized with default values. We can set the timeout for this client, and all requests made using this client will use the timeout.

func main() {
	url := "http://www.example.com"

	// set a timeout of 50 milliseconds. This means that if the server does not
	// respond within 50 milliseconds, the request will fail with a timeout error
	http.DefaultClient.Timeout = 50 * time.Millisecond
	resp, err := http.Get(url)
	if err != nil {
		log.Fatal(err)
	}
	defer resp.Body.Close()

	// print the status code
	fmt.Println("Status:", resp.Status)
}

In our case, the server at example.com does indeed take longer than 50ms to respond, so the request fails with a timeout error.

Output (Try it here):

2023/07/16 20:54:10 Get "http://www.example.com": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Setting a Timeout for a Single Request

Sometimes, we might want to set a timeout for a single request, instead of setting a timeout for all requests made using a client.

We can do this by using a context with a timeout, and bundle it with the request.

func main() {
	url := "http://www.example.com"

	// create a new context instance with a timeout of 50 milliseconds
	ctx, _ := context.WithTimeout(context.Background(), 50*time.Millisecond)

	// create a new request using the context instance
	// now the context timeout will be applied to this request as well
	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
	if err != nil {
		log.Fatal(err)
	}

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println("Status:", resp.Status)
}

Output (Try it here:

2023/07/16 21:09:36 Get "http://www.example.com": context deadline exceeded

Conclusion

The net/http package provides a lot of functionality for making HTTP requests. In this post, we learned how to make GET, POST, PUT, PATCH, and DELETE requests, as well as how to set headers and timeouts.

There are more configuration options that you can tweak, like setting the maximum number of idle connections, or setting cookies. You can read more about these in the http package documentation.

You can see the working code for all examples shown here on Github

The HTTP Client Library#

Requests and Responses#

Making a GET Request#

Making a POST Request#

Sending JSON Data#

Parsing JSON Responses#

Making PUT, PATCH, and DELETE Requests#

Setting Headers#

Setting Timeouts#

Setting a Timeout for a Single Request#

Conclusion#

The HTTP Client Library

Requests and Responses

Making a GET Request

Making a POST Request

Sending JSON Data

Parsing JSON Responses

Making PUT, PATCH, and DELETE Requests

Setting Headers

Setting Timeouts

Setting a Timeout for a Single Request

Conclusion