Mergebot is in Beta (Screencasts)

#

It’s been a few months since my last article about Mergebot and people have been asking for an update. For the past month or so I’ve been onboarding people manually, doing a one-on-one screen sharing session, guiding them through set up and helping them take their first steps with Mergebot. I learned a lot during these walkthroughs and it gave me the insight needed to shape the documentation and screencasts for the onboarding process.

The Beta is Open

Three weeks ago (Nov 29) I finished producing the onboarding process and we opened up the first public seats for the Mergebot beta. I sent an email to our beta email list and the first 75 seats went very fast. We immediately had a little feedback coming in from customers via email and the private Slack chat (everyone who signs up automatically gets invited to our Mergebot Slack chat). I say “customers” because we decided to charge for the beta. People tend to take free accounts for granted and I felt charging for it would help with that. I want people to ask themselves “Am I getting value out of Mergebot?” every time they receive an invoice.

We decided to charge a special beta rate of $9/month for 10 sites (you can deactivate sites and development/staging sites don’t count toward the limit). Those who sign up at this rate are grandfathered in and won’t have their price change when we launch.

We now have 112 customers using the beta and providing feedback, which is super exciting. We’ve already fixed a bunch of bugs and are working on fixing some more as we continue to chip away at our roadmap at the same time.

What is Mergebot?

Mergebot aims to ease the pain of merging WordPress databases at deploy time. It has two parts: a cloud app and a WordPress plugin.

mergebot-app-diagram

As you work on your Development site, the Mergebot plugin sends your database changes to our cloud app. When you want the latest data from the Production site, you simply replace the Development database with the Production database (using WP Migrate DB Pro or by another means) and then apply the changes saved to Mergebot. And you can do this as often as you like during development. When you’re ready to deploy to Production, you simply apply the changes saved to Mergebot on your Production site.

As you can see, we never send Production changes to Mergebot. We designed Mergebot around this because monitoring changes on high-traffic sites has a devastating impact on server resources and performance.

Here’s a more detailed look at the Mergebot workflow:

Current Limitations

In order to get the beta out to you sooner than later, we limited its scope. So although the following things are currently not supported, we do plan to support them in the future (some very soon, others when we’re out of beta):

Third-Party Plugins and Themes

Many third-party plugins and themes create their own custom database tables and shortcodes. WooCommerce and Gravity Forms are good examples of popular plugins that do this. You could also create your own database tables for your own custom plugin or theme.

Although Mergebot currently only understands WordPress core’s tables and shortcodes out-of-the-box, you can describe custom tables and shortcodes with what we call schemas, giving Mergebot the information it needs to handle them. In the future, we will be supporting popular plugins out-of-the box, so you won’t have to do this for WooCommerce, Gravity Forms, etc. In fact, we are currently putting together a GitHub repository so we can collaborate on schemas and start to roll them into the app.

To be clear, this doesn’t mean that Mergebot won’t work with those plugins installed. For example, if you have WooCommerce installed and only care about deploying changes for some blog posts and pages, Mergebot should work fine. If you wanted to deploy some changes to WooCommerce tax settings however, you would need to define a schema so that Mergebot understands the woocommerce_tax_rates table.

Multisite

WordPress multisite is not currently supported, but we plan to launch support for it early next year.

Duplicate Values

WordPress currently does not enforce UNIQUE column constraints at the database level. It relies on PHP to ensure that duplicate values aren’t entered into a column. For example, if you create a post with the same post_slug on both Development and Production then deploy to Production, two posts with the same slug will exist without conflict. We plan to start handling this shortly after multisite support is launched.

1,000 Queries per Changeset

We have a hard stop limitation of 1,000 queries per changeset for the beta. Once that limit is reached, no more queries will be saved to the changeset. It’s a good idea to deploy before you get close to that limit. Currently, generating a deployment can take several minutes when there are more than 1,000 queries, so we will be working to optimize the process, reduce the generation time, and increase the query limit.

Staging and Other Environments

Most people don’t just have two environments: Development and Production. It’s common to have at least one Staging environment (in fact, we recommend that you do). It’s also fairly common for multiple developers to be working on the same project and each have their own Development environment. We definitely recognize this and will be working to support multiple environments, but probably not until after we’re out of beta.

Media Files

If you upload a file to the Media Library while recording changes in Development, Mergebot will take note of it. It will also take note of any files deleted from the Media Library. You can then get a list of all the files uploaded and deleted on the Changesets page of the app.

Currently you will need to manually upload the files in the list to Production when you deploy. You’ll also need to manually delete files that need to be deleted. You’ll need to use FTP, SSH or another method. We plan to automate this in the future so you won’t need to fuss with any file uploading or deleting. Definitely an after-beta thing though.

Operations Requiring Deploy-Time Data

To illustrate the problem here, let’s look at WooCommerce as an example. You’re running a store, selling some digital products. You turn on recording changes to Mergebot and update one of the products, adding a new file download. WooCommerce finds each customer who has purchased that product and adds a new row to the woocommerce_downloadable_product_permissions table granting them access to download the new file. A week later, you deploy to Production and some of your newest customers report that they can’t download the new file.

Because you updated the product a week ago, WooCommerce only granted permissions to all customers who purchased the product up to that time. Any customers who purchased the product since then don’t have permission to download.

Another example of an operation causing this problem might involve going through all the posts of type shop_order and adding a row to the postmeta table. Obviously if any new posts of type shop_order are added to Production after the operation is run in Development, they won’t have the row added to the postmeta table after deployment to Production.

These types of operations are very common in install/uninstall/update/activate/deactivate routines in plugins and themes and when upgrading WordPress core.

Unfortunately we don’t have an elegant solution for this yet. When you identify one of these types of operations, we recommend working around it with the following process:

  1. If you already have a changeset on the go, you’ll have to deploy it to Production or discard it
  2. Refresh Development with the Production database
  3. Perform the operation in Development (you can do as many of these actions on as many plugins and themes as you like at this one time, though you should probably write down what you do as you’ll need to do it again when deploying to Production)
  4. Start recording changes
  5. Refresh your Development database at any time, then perform all the actions you performed in #3 above before applying changes from Mergebot
  6. When you deploy to Production, you need to first perform all the actions you performed in #3 above before applying changes from Mergebot

We realize this is far from ideal and are trying to come up with a better solution. We may need to work with plugin authors to expose these types of operations in an API and allow Mergebot to run them again at deploy time.

Phew, that was a lot of limitations. Time for some fun again. Let’s take a look at Mergebot in action.

Merging Databases

Conflict Resolution

Join the Beta!

If you’ve read this far, congratulations, you’ve already made it through the first few sections of our onboarding process! If you’re interested in continuing the fun and testing out Mergebot, visit app.mergebot.com/register and snag yourself an account. There are currently 60 seats available. If for any reason you want a refund within 14 days, by all means just ask and we’ll return your $9.

If you’d rather wait until Mergebot is out of beta, just sign up to the general news email list and we won’t bug you with updates about the beta.

About the Author

Brad Touesnard

As founder of Delicious Brains Inc, Brad has worn many hats. He now spends most of his time managing the product teams and growing the business. Before starting this company, Brad was a freelance web developer, specializing in front-end development.