A Call to Arms

WordPress 2.9 was just released, And several users have run into a bug. Surprising? Not really. Theres one simple reason for this, While thousands of people Test each and every WordPress release, These users are not You.

I’d like to use this as an example to all here, Why WordPress needs your help. No, I’m not talking about Coding help specifically, I’m talking about Testing. WordPress requires that users test the product throughout the development period.

WordPress is an open source application written by hundreds of contributors. While those hundreds probably use the Development version of WordPress every day, They do not use the same webhost as you, nor do they have the same theme, nor do they have the same requirements, They use different functions of WordPress than you.

During The beta and RC stages, thousands of people download and test, These testers are end users like you, In order to prevent these bugs getting into a released product, It requires that users actually take part in the developement of WordPress and report the bugs encountered.

Testing WordPress is not just something that Developers should do. If you use WordPress and enjoy using it, Please take some time once every few months to test WordPress, Its announced on the Dev blog when Beta’s are available, On a default install of WordPress, The Dashboard should have a RSS feed mentioning the releases too.

So please, For 3.0, When a Beta is released (Theres generally ~2-3 weeks of beta, with 2-3 beta’s from my quick recolection) Install it on your website, It doesnt have to be your main one, it can be in a subdirectory (ie. your usual one at http://my-site.com/ and the test at http://my-site.com/testing/), and test that things works ok for you, and your plugins, This does take an hour out of your time, and i realise not everyone can afford it, But it may spare you 2-3 hours of your time, when after a release, a bug that affects you is found, that a Developer had not noticed.

WordPress requires your input, Whilst I agree there are some downfalls in reporting bugs sometimes (Please do not flame me with related messages here, I’m not after that, This is mearly a request for more contributions) overall, your contributions would be greatly appreciated by all.

In order to make it easier to test Beta’s and nightly versions, Westi wrote this great plugin It allows you to use the inbuilt upgrader to upgrade to a beta, to make it easier for you, to be involved with the project you love using.

21 thoughts on “A Call to Arms

    1. A similar problem happens every release. Not enough people test on their host, and the result is, that a untested server configuration is hit. When you consider that Developers probably make up 0.05% of total installs, and beta testers maybe 0.1% Its impossible to cover every branch.

      As an example, So far there have been 320,000 downloads of WordPress 2.9. There were 140 identified contributors to WordPress 2.9, Lets assume that out of those 320k downloads, 10% were on unique server setups, thats 32,000 server configurations, Thats 230, Two Hundred and Thirty different Server configurations PER CONTRIBUTOR, Its impossible for developers to do that testing.. Testers are required.

      The more testers there are, The more server configurations are tested, and functions, Its impossible for them to all be covered…

      1. Thats just based on Server Configuration, If you take into account that different features work different on different servers, the testing list can grow even larger.

  1. Hi DD32. I agree with the body of the message. I must point out, however, that the bug you referenced at the top did not exist in any of the beta or RC versions of WordPress. This bug was introduced *without testing* immediately before release as described in Ticket #11468

    1. I do agree that particular changeset was pushed in late, due to a known issue with another transport.. In retrospect, The changes to the other transports should’ve been delayed but at the time, seemed to make sense according to the docs available.. and testing on the servers that were tried.

      This is not the -only- issue in this release, just as it is every release, There’s always some kind of bug which affects some which never surfaces until a few hours after release.. My point was more of a generic one, even though it was spured from the pointed out item. Its something i hear every release, and if people were to test more, it’d be less likely to happen – I’d throw a comment in here about ‘damn php bugs’ but i know someone will just flame me for passing the buck..

      1. > changes to the other transports shouldve been delayed

        I must respectfully disagree. This is a situation that demands a second release candidate. Any non-trivial change to something like the HTTP API should be given weeks of testing.

  2. As proposed in dev talks a clone solution to run a clone of online site on a test install will help to find those bugs testers dont find.

    If I remember correct Matt`s respons was that its not any lack of testers. I there was 10.000 + installs of beta/RC.

    John Myrstad

    1. I personally dont give a crap about what Matt thinks. Its all opinion.

      And in my opinion, Due to the vast number of servers out there and the configurations in which people use wordpress, More testers will -always- be needed.

      I’m not sure what you mean about a Clone solution? By running a copy of a production site, you are infact, a tester.

    2. Number of testers as counted by the number of people who’ve downloaded the beta or RC is very different from the number of people who actually report bugs, which is even still different from the number of people who report bugs and provide enough information to make it reproducible and thus fixable. I’m with Dion here, we need more testers, and more specifically, we need more testers who are willing to follow a published QA process. (Which, of course, means we’d need to have a published QA process!)

      Seriously, though, they’re both right. We do have a lot of people who download early trunk versions and give them a spin, but very few who run automated tests or even manually review each change and report back on the performance on their setups. I think creating a section on WordPress.org with instructions for testing, maybe with a simple QA form that can be submitted, would help us get more high-value testers. The easier we can make it, the better off we’ll be.

      1. I’m repeating myself, but the number of testers doesn’t mean a damn thing unless someone fixes the bugs that are reported. I put so much effort into writing patches and screaming about known bugs during this beta cycle, that I didn’t even get around to reporting all the bugs I found, let alone test everything.

        1. And the patches need to be tested by more than one or two people before they are evaluated for commit. You’re making the case for more testers without apparently realizing it. If more things were tested at the level they are meant to be before being reviewed for commit, the core commit team could get through patch reviews faster, and patches wouldn’t languish.

          1. This article had nothing to do with testing patches before commit. If you’re saying there aren’t enough people involved in Trac, I think that’s a whole separate topic.

  3. If bugs are found earlier, it will be more time to test, debug and patch.

    If someone could collaborate on a Clone WP Plugin, which makes it easy to set up a local or subdomain clone of a live site, more medium skilled people will run RC`s to prepare their site for the upgrade which may lead to more bugs found early.

    Some code which may be used in such plugin is made by Dave Coveney: http://www.davesgonemental.com/mysql-database-search-replace-with-serialized-php/

  4. I cant understand why they cant have automated tests and put them through the grind. Manually reviewing each change and then reporting back on the performance on their setups is fraught with too many whims/

  5. I have been patient but really these bugs need to be tested more thoroughly with new updates, I know its impossible to get everyone’s bugs resolved but I have a wish list for the next wordpress update:
    1) Please fix the “spelling correction button” on Writing Posts I often am in a big hurry. I have to spell check each post and you know it picks up false positives, and then freezes up on my and doesn’t work further. I wish you would have a better spell checker on WP, and a grammar fix would be nice too.
    Yes some people will say why don’t you write it first in word then spell check then bla bla bla….C&P….well that defeats the whole purpose of writing a post freely within WP posting area, and the editing function. I want it to work properly when I am writing an article -that’s about it.

    Wish Number Two: would be to have an Archive that gives you the option of displaying post titles instead of just categories, this is a great SEO function as well that I would like to use, have each post displayed as a link – not just the months of the year.
    If I were a reader – How would I know what subject title was written 3 months ago> its makes it harder for people to find posts.

    If anyone has a plug or two that does the above mentioned -it would be appreciated to find it -where ever it is hidden. I don’t do codes, and PHP…I am a writer….and the easier these ” Techey” things are – the less stress in my life.
    Sam

    1. Hi Sam. I recommend After the Deadline as a plugin for better spell checking and grammar checking. It’s in the repo, so you can install it right from your “Add New Plugins” screen.

  6. Actually, no, the problem isn’t that WordPress needs more testers, it’s that WP keeps upgrading for no reason at all. Most bloggers don’t need all these new features that developers keep dreaming up. We just want something that works. I’m really pissed off that I have to keep upgrading WP every week or so (seems like) and all I want is something stable to write on. I have no time for all these upgrade problems. Developers are selfish, they think everyone is interested in code like they are, and everyone is NOT. Writers just want to writer, not ditz around with all this code crap. I don’t want any more upgrades, ever.

    1. Well, without upgrades you would be using some old version without the features you appreciate and you probably would have been hacked too. Maybe you would be beter suited with a wordpress.com solution.

Comments are closed.