Introducing the All-New Assets Pull Addon for WP Offload S3

We’re very excited to introduce you to the all-new Assets Pull addon for WP Offload S3. It’s a completely new plugin, written from scratch using a different approach, and intended to replace the original Assets addon.

The Assets addon, you might recall, allows you to serve your site’s assets (CSS, JS, images, fonts, etc) from S3, CloudFront, or another CDN in just a few clicks. Making it possible to speed up delivery of these assets and reduce the demand on your web server. This adds onto WP Offload S3’s ability to offload your WordPress Media Library to S3 and serve it from S3, CloudFront, or another CDN speeding up delivery of your Media Library and reducing demand on your server.

So why did we need to build a new Assets addon?

Push vs Pull

The original Assets addon pushes files to S3. It scans your site’s filesystem for assets and uploads them to S3. We went with a push solution because at the time the team felt it would be better than pull.

With a pull CDN, it retrieves the asset from the origin server on the first request, caches it, and serves it. It then serves that asset from the cache for subsequent requests (until it expires). This first request can therefore be a bit slower. With a push CDN, we push all the assets to the CDN, and now that the asset is already there it can be served on the very first request.

This seems like reason enough to favor push over pull, but we’ve since realized there are some significant downsides to the push solution:

When we update a theme, plugin, or WordPress core we need to serve its assets from the web server until all the new and updated assets have been uploaded. Otherwise the site is likely to break using old assets.
We have to scan the filesystem regularly for updated assets because files can be updated via FTP, SSH, etc. We can’t rely on WordPress hooks to tell us when something has been updated.

So not a great situation to have periods of time where your assets are being served from your web server again instead of your CDN. However, it’s the filesystem scanning which has been the bigger problem. There are major issues with scanning the file system from a WordPress plugin:

Slow and consumes server resources. Recursively iterating through folders, getting directory listings, and checking for file changes is a lot of I/O, CPU, and memory.
Unreliable. WordPress doesn’t have a background processing system. We had to bake our own, work out the bugs, and struggled to make it reliable for all web hosting configurations.

After debugging hundreds of support requests with our customers and coding dozens of fixes and improvements to our background processing library, the push solution looked like a bad choice.

S3 is not a CDN

We also noticed that many of our customers were serving their assets directly from S3. They weren’t going through the extra step of configuring CloudFront or another CDN to point at their assets on S3. They were setting up the Assets addon to push their assets to S3 and serving them from there. As we’ve written in the past, you’re better off serving your assets from your web server than S3.

This is our fault of course. After you configured the Assets addon to push your assets to S3 it definitely seemed like you were done. If you read our documentation you would know that you should configure CloudFront or another CDN, but we provided very little guidance in the Assets addon itself. We really should have been encouraging this more strongly.

For all these reasons, the original Assets addon was not the best solution. We decided to fix that.

The All-New Assets Pull Addon

The Assets Pull addon is a new plugin (not an upgrade) designed to make properly serving your assets as easy as possible using Amazon CloudFront (or just about any other CDN). The addon includes a CloudFront setup wizard to walk you through the process of creating and properly configuring a new CloudFront distribution to “pull” assets directly from your web server rather than from S3.

Without the need to upload assets to S3, assets are now immediately available to be served via CloudFront once your distribution has been configured. Another benefit is that the plugin is more compatible with third-party extensions which you might use for minification and concatenation, features not part of the Assets Pull addon. Serving compressed (gzipped) assets is now handled automatically by CloudFront.

Perhaps the feature that really sets the Assets Pull addon apart from other similar solutions is the ability to run a comprehensive test on your configuration before you enable URL rewriting. This greatly reduces the risk of experiencing problems due to DNS propagation delays or CDN misconfiguration.

Sunsetting the Assets Addon

With the release of the Assets Pull addon, we are deprecating the original Assets addon. The deprecated addon will still be supported for some time (security releases only) but will eventually be discontinued.

We’re encouraging everyone to install and activate the new Assets Pull addon to replace it. We’re confident you’ll love the new addon so much that you won’t miss the old one. 😉

Can the Two Addons Coexist?

Both the new Assets Pull and old Assets addons can be installed at the same time, but to avoid URL rewriting conflicts only one of them can be active at any given time. Activating one will automatically deactivate the other with a nice notice about what happened.

Transitioning from the Old Addon to the New Addon

If you have the original Assets addon running on your site, just install and activate the new Assets Pull addon and go through the setup wizard. Your assets will be served from your web server while you’re going through setup. Serving your assets from the web server temporarily should be completely fine as that’s exactly what happened with the original Assets addon any time you updated your plugins, themes, or WordPress core.

Updates to WP Offload S3

Along with the release of the new addon, our WP Offload S3 and Amazon Web Services plugins have been updated as well, fixing up a few things here and there.

Perhaps the biggest change is that we’ve added notices for sites running a PHP version less than 5.5 (hopefully you never see those yourself). While running a stable version of PHP is obviously recommended, we’ve never required anything higher than what was needed by the AWS SDK v2 (PHP 5.3). But we’re planning to change that soon.

Version 3 of the SDK has been available for quite some time and offers some features not available in v2. But it requires PHP 5.5+. This is why we’re warning users about old PHP versions now in this release.

What’s Coming Next

We get quite a few requests to upgrade the AWS SDK in our Amazon Web Services plugin to v3. Usually the customer wants to use the SDK in their own development and they require some features that were added to v3 and are missing in v2.

Updating to v3 of the SDK is tricky though. We’re already addressing the issue of increasing the PHP version requirements (as mentioned above) but what about those who’ve built on top of the existing Amazon Web Services plugin? It’s been coded for v2 of the SDK. If we update to v3, it’s likely to bork anything they’ve built and could take down some sites.

We’ve realized we can’t update the Amazon Web Services plugin to v3 of the SDK. And so we won’t. We’re leaving it alone.

The next release of WP Offload S3 will no longer require the Amazon Web Services plugin as a dependency. Instead it will include its own namespaced versions of the components from the AWS SDK that it needs. This means that you can safely load and use whatever version of the AWS SDK you want for your own needs, without conflicting with the SDK code bundled into WP Offload S3!

This will also eliminate one of the most common sources of problems our customers experience with WP Offload S3: another plugin loading its own version of the SDK. This causes WP Offload S3 to malfunction and so customers think there’s something wrong with it and reach out to us for help.

Congratulations to the rest of the WP Offload S3 team for all their work on this release.

Does this new addon sound like something you’d use? Any questions? Let us know in the comments.

Not a WP Offload S3 customer yet but like what you see? Get WP Offload S3 with the Assets Pull Addon at the Gold License and above.

This entry was posted in WP Offload Media and tagged New Release, WP Offload S3, Assets Addon, Assets Pull Addon, WP Offload Media.

.replybox-fallback, .replybox-fallback ul { list-style: none; }

Carl Myers says:

November 20, 2017 at 1:29 pm

Thanks – we just bought the old version – does that mean we should replace it with this – how do we do that?

Evan says:

November 20, 2017 at 2:34 pm

Yeah, pretty much. There’s nothing transferrable between the two plugins settings-wise. You can download the new addon from your My Account page. Once you install and activate it, the old addon will be deactivated automatically if present. From there you just follow the set up wizard. The whole process is designed to be really easy.

Reply

Wilson Revehl says:

November 20, 2017 at 1:35 pm

Neat! So then, fundamentally, does this work by way of the asset being requested via cloudfront, it checks if the asset exists, if it doesn’t, it copies it to cloudfront for next time and then expires at some point, rinse and repeat?

Lloyd says:

November 20, 2017 at 1:56 pm

Will there be an auto-update feature, to automatically transition our setup with the old plugin to the new one?

Evan says:

November 20, 2017 at 2:42 pm

No, because there isn’t really anything to transition. You’ll likely need to create a new CloudFront distribution. Other than that, you might need to add a new DNS record for your domain, or you could change the one you were using previously if you had a dedicated subdomain for your assets before. Not to worry though as the whole set up is really easy and the wizard will walk you through it.

Reply
- Lloyd says:
  
  November 20, 2017 at 3:16 pm
  
  Apologies if I misunderstood, but, my reading of the page implied to me that this was to replace the Assets addon. If that’s the case, won’t there need to be settings either manually entered into this new addon from the old addon, or automatically ported if that functionality exists? Either way, looks cool!
  
  Reply
  - Ian says:
    
    November 20, 2017 at 3:33 pm
    
    Yes, it will eventually replace the now deprecated Assets addon, but it works in such a different way that there aren’t any settings that can be ported over. If you follow the set up wizard you’ll see it requires a different kind of CloudFront distribution, so most customers will create a new one while some may want to remove their old distribution and create it anew as a pull distribution. Either way, confirming the new distributions domain by entering it manually is a must.
    
    Reply
- Brad says:
  
  November 20, 2017 at 3:41 pm
  
  We can’t automatically transfer settings over because the settings from the original addon don’t translate to the new addon necessarily. When you activate the new addon, the old one will be deactivated and your assets will be served from your web server until you complete set up of the new addon.
  
  Reply

Jeremy says:

November 20, 2017 at 3:24 pm

Is the new Assets plugin compatible with WP Rocket? If so, are there any settings we need to be aware of? Thanks

Ian says:

November 20, 2017 at 3:37 pm

We’re not currently able to specify any page caching plugin as being fully tested and supported with the new Assets Pull addon yet, but a big reason for this plugin existing is to be able to support plugins like WP Rocket much much better than the old plugin did.

Reply
- Jeremy says:
  
  November 20, 2017 at 5:10 pm
  
  Thanks for the quick response. Does this mean that we should try to use it with WP Rocket now, or should we wait until further testing has been completed?
  
  Reply

Bill says:

November 20, 2017 at 4:22 pm

Sorry, but I got bored reading this and couldn’t work out exactly what it was. Can it offload images to an FTP server yet and update the URLs to match? Thanks.

Dale Reardon says:

November 20, 2017 at 9:22 pm

Hi, I use Cloudflare as my CEDN service and upload all my images to S3 using your plugin of course but I haven’t been using the assets addon. Will your new assets addon work with Cloudflare? Dale. https://mydisabilitymatters.club

Ian says:

November 24, 2017 at 12:58 pm

Yes, while the wizard shows how to set up CloudFront as the pull CDN, you should be able to translate the basic ideas to CloudFlare and simply enter the correct URL in the Assets Pull addon’s settings page.

Reply

jan7 says:

November 21, 2017 at 7:17 am

Don’t forget to update Amazon Web Services to 1.0.4 to avoid permission issues.

Scott says:

November 21, 2017 at 2:05 pm

Quick question, my site already has an SSL certificate for my domain, do I need to get an new certificate from Amazon to handle serving my assets from AWS CloudFront? Or is the Amazon SSL only used for assets being linked from CloudFront to my website? Will these two certificates cause conflicts with each other?

Ian says:

November 24, 2017 at 1:01 pm

The separate certificate you set up for the CloudFront distribution will not conflict with your site’s certificate.

Reply

Grant Bivens says:

November 21, 2017 at 6:51 pm

If we already have CloudFront setup with an SSL cert is it possible to use the same distribution and SSL cert or do we have to create a secondary distribution and SSL cert?

Ian says:

November 24, 2017 at 1:09 pm

If your site is fronted by a CloudFront distribution then you may not get much benefit from using the Assets Pull addon unless that existing distribution doesn’t cache on params due to the site being pretty dynamic. If your current distribution isn’t caching on params then setting up a separate CloudFront distribution for the Assets Pull addon to use that does cache on params (see the wizard for details) will reduce pulls and cache usage for the versioned static assets such as CSS, JS, fonts and referenced images.

Reply

pubg says:

November 22, 2017 at 11:15 am

This article was very useful, thanks a lot for doing such a great job 🙂 open free PUBG crates

Dale Reardon says:

November 26, 2017 at 1:15 am

Slightly different problem – I am using Offload S3 to move my media files to S3 but also using Cloudflare as my firewall and CDN. No matter what I try I cannot get Cloudflare to work with Amazon S3 through a Cname record to then have Cloudflare act as CDN for the S3 files. Cloudflare says I have setup the Cname correctly but still no luck. I am hosted with Kinsta but don’t think that would matter. Can you recommend someone I can pay to fix it for me? Please email me : dale AT mydisabilitymatters.com Thanks, Dale.