Development Journal: Unit Testing the Entire Website

I’m working on writing a unit testing framework for the entire Bangor Daily News family of websites, and it’s something that I’m struggling with. For that reason I’ve decided to journal my frustrations and the steps in the hopes that I can remember what happened on the other side of this.

What is Unit Testing?

Unit testing is basically writing code that checks other code to make sure that the code it is testing performs as expected and intended. Unit testing is something that every large scale project should be doing, and something that we as a company hadn’t been doing. That made it my problem.

May 27, 2015

The goal of the project is pretty straight forward: WordPress has a rather robust unit testing setup for their core files, and there are examples for how to unit test Plugins and Theme files. I’m going to combine all of these tests, install the website files over the WordPress core, and run unit testing on all of the files.

Right from the start I’m realizing it’s not as simple as copy pasta over core. Our site has heavily customized things, and we’re using many plugins and drop-ins that the Core unit tests don’t account for, so I’ve forked the Unit test framework, and I’m making the required changes.

Issue #1: Database

Our website is a Multisite install, which the core files are setup to handle, but unfortunately the default setup isn’t working. Connection is refused, the unit tests aren’t working, etc. To see what the problem is, I’m going to run a simple test.

Steps:

  1. Clone the WordPress development files.
  2. Create a fresh database table.
  3. Update wp-tests-config.php with corresponding DB info.
  4. Run phpunit

If the issue really is the difference between our website files and WP core files, PHP Unit should run without issue.

Result:

After doing the steps above, the result was a clean running unit test. That tells me that our database isn’t working with the WordPress core unit test setup. I’m going to test this further with multisite setup, but I think I’m on the right track.

WordPress let’s you create custom database error handlers, so I created one to print a backtrace for me whenever the DB isn’t working, and low and behold it worked and let me know that HyperDB, which we use because multisite, isn’t loading just the way it should, for one reason or other.

We heavily customized our config file, I mean a lot, so I’m taking a play from WordPress’ handbook here and creating a config file strictly for our unit testing. This means I’ll be cutting out a lot of the clutter, and the things that aren’t being tested currently, and hopefully that’ll get us to the point we can at least run the tests. I don’t care if they error out, I just want a fully running PHP Unit.

Because I really don’t care about HyperDB, and it’s just causing me heartache, I got around the errors by just not using HyperDB during unit testing. This is obviously a hacky way to handle it, but once I have a working unit test setup, I can work on forking HyperDB and fixing it’s implementation.

Next up to ruin my day is Batcache! That said, I also don’t really care about cacheing when I’m running unit tests, so I’m just going to shut that off for this as well.

Shut off caching and it found new errors. Sunrise was causing errors, but being wise to their tricks, I created a blank sunrise file. There were a few more errors, but I realized during the process that I was running Master tests against the 4.1 branch. So I switched to the 4.1 branch tests and ran again. PHP Unit started running.

It’s randomly throwing errors, but that’s fine. I track each error down and either skip it, or fix it.

June 2nd, 2015

The last time I ducked into it we were getting a lot of terminal issues where a rogue UTF-8 character was changing the character structure of the terminal. I spent a few days trying to figure out how to remove the data sets from the PHPUnit debug output. Figured that out today, so now I’m back tracking down individual tests.

After skipping all of the tests that caused PHP errors, or took more than a few seconds to complete, I finally reached the point where the tests would finish and the PHP Unit report would display. This is good. It’s filled with fails, but it completes.

Now I’m working on a BASH Script to automate the setup and tear down of the main repository code, so I don’t have to keep a duplicate copy of our codebase in the testing repository.

June 17th, 2015

I finished this a week or two ago, shortly after my last entry, but forgot to update. Basically what I ended up doing is looking at each fail, and if it was something that would take more than a couple minutes or a line of code change, I skipped the test, and left myself a note in the code as to why. I now have a passing Unit Test.

I’m going to publish this as this entire brain dump, and revisit the topic in a future post where I’ll explain the whys and the whats a lot better. For now, this is what you get.

Going back to an old project

I started making WordPress plugins about 5 years ago, and one of my first was Auto More Tag, which essentially just adds a “Read More” button automatically into your content. Now, this seems like an easy thing to do, but at the time I had no idea how to go about it. I ended up coding it to insert the more tag into the content during the saving of the content. Once it was working, I left it to do other things.

Let me make something very clear, something that this very plugin taught me.

It is never a good idea to change the content of someone’s website with a plugin.

Read those words. They’re in a blockquote because they’re important. Do not do this. Ever. Do not do it. You might think it’s a good idea, but it’s not. At the time I was naive, and thought I could get away with doing this. I couldn’t. It’s bad. So very bad.

And the support forum taught me that, with many people cursing my name because of deleted content, and rightly so. Turns out that I hadn’t put in support for multibyte strings, so our non-English speaking friends were getting their content chopped. This isn’t a good thing, but I had no idea how to fix it.

Then a developer offered to add in MB String support, and I let him fork it, and take over. Then I stepped away for a couple years. I’ve just recently opened the code back up after getting a few emails that the plugin wasn’t working.

Now that I’m a bit more seasoned, I realize exactly how stupid it was for me to edit content. I realized this before, but now I actually have an idea of how to fix that stupidity. In release 4.0.0, which I pushed out to the WordPress plugin repo this weekend, I’ve converted the codebase from inserting on save, and editing the content (read the quote above) to now using a filter on page load to insert the more tag.

This is a much better way to handle things, and I’m happy to do it.

I’m also happy to see that one of my very first plugins, and in fact the very first plugin I ever released on the WordPress repository, is still being used today. 4 years later.

Future Plans

I plan to improve the settings page, and clean up the code a bit, make it easier to understand. But a lot of what I plan to edit has to do with the development environment. I’m hoping to encourage people to help me to improve the plugin.

Aside from that, I’m looking forward to improving the user experience, and making the plugin even simpler to use. Possibly adding in additional ways to customize where and when the More tag is added.

“Generally WordPress plugins work really well … when they’re small. When you build something that solves one particular problem, or improves one specific WordPress feature, or does something very … purposeful.” – John James Jacoby, LoopConf Vegas, 2015