Why You Should Use the WordPress HTTP functions to Make API Requests

#

Sometimes your WordPress site needs to talk to other services around the web. This almost exclusively happens using the HTTP protocol. A common example of this is when your WordPress installation contacts the wordpress.org servers to check for new versions of plugins, themes, and WordPress core itself.

It’s also a very common thing to do in a WordPress plugin or theme. Any plugin that interacts with an external service is bound to make some HTTP requests here and there. Things like adding subscribers to your Mailchimp list, sending emails via Amazon SES, or offloading images to Amazon S3 all involve making a bunch of HTTP requests.

In this post, we’re going to look closer at the built-in HTTP functions that WordPress offers to make these requests and why it’s almost always a good idea to use those functions.

But first, some background.

Doing It Wrong – Using PHP Functions

PHP itself offers a couple of different ways to make HTTP requests. You could work directly with the Network functions for your HTTP requests using fsockopen(), fread() and fwrite(), but you would end up both wasting time and reinventing a lot of wheels for no good reason.

Slightly better would be to use one of the file reading functions that can handle PHP’s HTTP protocol wrapper. This includes the slightly cumbersome fopen() as well as the much easier file() and file_get_contents() functions. A simple one-liner will actually get you quite far:

// Example code only, don’t do this!
$response = file_get_contents('https://www.google.com/');

But before you stick this code into your WordPress plugin or child theme, please be aware that many hosting providers explicitly disable opening URLs this way. So what works in your local development environment may not work in your production environment.

And besides, just because it’s possible to get content from an Internet URL using the PHP functions doesn’t mean that’s the best approach in day to day development. Just adding the seemingly innocent requirement of specifying a header in the HTTP request makes the above example much more complicated (example from Stack Overflow):

// Example code only, don’t do this!
// Create a stream
$opts = array(
  "http" => array(
    "method" => "GET",
    "header" => "Accept-language: en\r\n",
  )
);
 ​
// DOCS: https://www.php.net/manual/en/function.stream-context-create.php
$context = stream_context_create( $opts );
 ​
// Open the file using the HTTP headers set above
// DOCS: https://www.php.net/manual/en/function.file-get-contents.php
$file = file_get_contents( 'https://www.google.com/', false, $context );

Not so easy any longer. And then consider that this example doesn’t have any error handling in place. So unless you are 100% sure that the remote server will politely reply back (hint: you’re not), you have to add more code to wrap that elusively simple file_get_contents() before it’s usable in production code.

The Abundance of PHP HTTP Libraries

Since making HTTP requests using raw PHP is a bit clumsy, it should be no surprise that there are dozens of helper libraries that make creating HTTP requests easier.

In the broader PHP world, the one outside of WordPress that relies on Composer for dependency management, Guzzle and Httpful are among the most popular libraries for handling HTTP requests but there are literally dozens of others to choose from. All of them with their own strengths and weaknesses.

In the world of WordPress we need to be a bit careful before using Composer libraries, especially if you are going to ship this code to other WordPress users. There are two primary reasons for this.

First and foremost there’s a real risk that you or your users end up in some version of the Composer Dependency Hell. In short, that is when another plugin on the same WordPress installation uses the same dependency that your plugin does, but that plugin uses a different version of that dependency.

WordPress has a deterministic way of loading plugins. While it’s technically possible to hack the plugin order, it’s very much discouraged. Besides, whatever tricks you play to force your plugin to load first, other developers can play too. For these reasons, you will never know if your plugin is going to use the versions of the dependency that you shipped, or some other, perhaps incompatible, version.

There are ways to work around this using the Imposter plugin that can wrap all Composer dependencies in your own PHP namespace. You can go down that route but the seemingly simple idea to use a Composer package to manage HTTP requests suddenly turned into a much bigger problem that is probably also altering your build process.

On our own product WP Offload Media, we use third party libraries from the supported storage providers (Amazon AWS, Digital Ocean etc.) that in turn use the Guzzle HTTP library. To avoid compatibility issues with other plugins, we are using Imposter to wrap all of those dependencies in a separate namespace.

The other reason we should think twice before using a third party library for HTTP requests is that it will short circuit the WordPress filtering process.

One of the fine qualities of WordPress is that so many things can be customized via actions and filters and making HTTP requests is no exception. We’ll go into more detail in the next section of this post, but giving site owners the opportunity to modify HTTP requests to suit their special needs is inherently good. You shouldn’t remove this feature unless you have a really good reason.

Say Hello to wp_remote_get, wp_remote_post and Friends

For the past 12 years WordPress has shipped with it’s own built-in functions for dealing with HTTP requests, namely wp_remote_get(), wp_remote_post() and wp_remote_head(), one function for each of the HTTP verbs GET, POST and HEAD. These three global functions all work as wrappers around the WP_Http class that in turn uses the Requests library. We’ve actually written a little bit about the Requests library before in our post about how WordPress uses cURL to make HTTP requests.

Using wp_remote_get() can be as simple as:

$response = wp_remote_get( 'https://www.google.com/' );

There are plenty of reasons why you should prefer using the functions in the wp_remote_* family, here are the ones I think are most important.

They Are Always There

The wp_remote_* functions have been around in WordPress almost since the beginning so there’s absolutely no need to worry about them not being present.

Several important functions in WordPress core, such as plugin installations and automatic updates, rely on these functions to be there and just work, which they do. So there is no real reason to add third party libraries for HTTP requests, you already have access to a good and reliable library.

They Are Very Capable

You can access pretty much all features of the HTTP protocol by passing in additional arguments via the second parameter of these functions.

You can add headers, change the user agent string, force a specific version of the HTTP protocol, etc. You can also specify your own location for SSL certificates, disable SSL certificate verification, modify request waiting time (time out), or even add HTTP verbs if you need to. We’ll actually look at adding wrappers for two HTTP verbs further down in this blog post.

Here’s an example where I modify the HTTP request a bit before sending it off to the remote server:

$args = array(
  // Increase the timeout from the default of 5 to 10 seconds
  'timeout'    => 10,

  // Overwrite the default: "WordPress/5.8;www.mysite.tld" header:
  'user-agent' => 'My special WordPress installation',

  // Add a couple of custom HTTP headers
  'headers'    => array(
     'X-Custom-Id' => 'ABC123',
     'X-Secret-Thing' => 'secret',
  ),

  // Skip validating the HTTP servers SSL cert;
  'sslverify' => false,
);

$response = wp_remote_get( 'https://www.example.com/', $args );

The complete list of valid arguments aren’t completely documented anywhere but can be found in the source file for the HTTP class itself or in this helpful comment

In short, wp_remote_get(), wp_remote_post() and wp_remote_head() are capable of handling most use cases for HTTP requests that I can think of. So in reality there are very few situations where you absolutely need to use a third party HTTP library in WordPress.

Using the underlying Requests library, it’s even possible to use built-in functions to do multiple requests at the same time, but that’s a little bit out of scope for this blog post.

They Are Easy to Customize

During the execution of an HTTP request all wp_remote_* functions will call a lot of different WordPress filters that allow you and others to modify the exact behavior or even block external URLs from ever being requested.

This means that not only can you control your own HTTP requests in great detail, you can also interfere with HTTP requests made by other plugins, or even WordPress core itself.

Imagine you are responsible for a couple of internal WordPress installations at a big company. To comply with internal IT security rules, you need to hide the internal site URL that is automatically added to the user agent string in HTTP requests. How would you accomplish this?

Well, wp_remote_get() and all its friends will trigger the http_request_args filter before the request is sent off, so we can change this using a simple WordPress filter:

function my_http_request_args($args) {
  $args['user-agent'] = 'WordPress';

  return $args;
}
add_filter('http_request_args', 'my_http_request_args');

Some site owners may have the need to block HTTP traffic to external URLs altogether. All requests that that are made using wp_remote_get(), wp_remote_post() or wp_remote_head() will respect the WP_HTTP_BLOCK_EXTERNAL and the WP_ACCESSIBLE_HOSTS constants. We’ve written a bit about these in our documentation for WP Migrate DB Pro.

If defined, these two constants will allow a site owner to stop all outgoing traffic but optionally also allow a set of exceptions:

define( 'WP_HTTP_BLOCK_EXTERNAL', true );
define( 'WP_ACCESSIBLE_HOSTS', 'api.example.com,www.example.com' );

A request to any other host will now return a ´WP_Error´:

$response = wp_remote_get( 'https://www.google.com' );

/* Returns:
WP_Error Object
(
    [errors] => Array
    (
      [http_request_not_executed] => Array
      (
        [0] => User has blocked requests through HTTP.
      )
    )
  …
)*/

By using the built-in HTTP functions in your code, your users get all this low level control of HTTP requests without you having to write a single line of extra code. How great is that!!?

They Are Transport Agnostic

A neat feature of the Requests library that sets it apart from a few of the those linked to above is that it is transport agnostic. More specifically in the case of HTTP requests, it’s not dependent on the cURL module being installed on the web server.

The Requests library prefers to use cURL if it is installed. But in the unlikely event that cURL isn’t present, the Requests library can fall back to using fsockopen() instead.

Either way, the Requests library is going to get the job done so you will have one less dependency to test, document or worry about.

What About wp_remote_put and wp_remote_delete?

If you’ve done a bit of work with remote APIs and know your way around the HTTP verbs you may have noticed that so far I haven’t mentioned wp_remote_put() or wp_remote_delete(). Since we have built-in implementations for GET, POST and HEAD, clearly there should be an implementation for PUT and DELETE as well. Right?

Well no. The WordPress Core team hasn’t implemented all the standard HTTP verbs just yet. But there’s an ongoing discussion about adding them in the WordPress issue tracker so it just might happen in a future release.

If you need those functions sooner, the modular nature of the WP_Http class, combined with the wp_remote_request wrapper function, makes it very easy to add these two in your own code:

if ( ! function_exists( 'wp_remote_put' ) ) {
  function wp_remote_put($url, $args) {
     $defaults = array('method' => 'PUT');
     $r = wp_parse_args( $args, $defaults );
     return wp_remote_request($url, $r);
  }
}

if ( ! function_exists( 'wp_remote_delete' ) ) {
  function wp_remote_delete($url, $args) {
     $defaults = array('method' => 'DELETE');
     $r = wp_parse_args( $args, $defaults );
     return wp_remote_request($url, $r);
  }
}

Note that wrapping these function declarations in ´function_exists()´ is important because there’s a very real possibility that another plugin on your site may already have implemented them. It’s also a good safeguard if the WordPress Core team actually does add them in a future version of WordPress.

Conclusion

I hope this has been a good introduction to how and why you should use the built-in HTTP functions in WordPress rather than using a third party library or rolling your own.

As we’ve seen above, WordPress comes with a great HTTP API baked in, which is well documented and maintained. There are multiple advantages to using it over any alternatives. These advantages extend not only to you as a developer but also to WordPress site administrators, and the entire WordPress ecosystem.

Do you have questions about the HTTP functions? Did I miss any important aspect or feature, or do you have any tips? Let me know in the comments below.

About the Author

Erik Torsner

Well experienced software developer from Stockholm, Sweden with a specific taste for WordPress plugins, software quality and automations. Loves any technology that turns out to be the right solution to the problem.