Google Forms, Checkboxes and PHP 5.4 or 5.5

WordPress Google Form v0.63 introduced a change  which allowed the plugin to work on newer versions of PHP, notably 5.4 and 5.5.  Unfortunately that change has broken support for checkboxes.

For those who are new to the plugin or never needed to know how it worked, what the plugin does is retrieve the HTML for the form from Google and render it within the context of WordPress.  When the form is submitted, it is actually submitted within the context of WordPress.   The data is collected by the plugin and then submitted to Google.  The retrieval from and submission to Google is done with the WordPress HTTP API.  In particular, the wp_remote_get() and wp_remote_post() functions are used to retrieve and submit the form.

To complicate the problem further, Google uses Python as the backend for their form processor where as WordPress uses PHP.  For the most part, the fact that they are based on different scripting languages isn’t a big deal.  Until you get to checkboxes.  Checkboxes in Python are handled differently than they are in PHP.

I had solved the compatibility problem a couple years ago (see this thread on the wp-hackers mailing list) using a small jQuery script which fixed the form variables on the WordPress side and manual construction of the body parameter for wp_remote_post() when submitting the data to Google.  This solution worked fine until I received a bug report that nothing was being submitted on a site which was running PHP 5.4.x.

Fortunately the user who encountered the problem provided me with a patch that I was able to fold in which changed the way the body parameter was constructed (array instead of a string) which worked with PHP 5.4.x and also worked with older versions.  However, I didn’t test it thoroughly as I have had several reports that checkbox content was not being submitted correctly.  Uh-oh.  I was able to verify the problem fairly quickly and was able to push out a version which essentially reverts how the body parameter is constructed (string instead of an array).  The problem is, this build doesn’t work with newer versions of PHP.

I have PHP 5.5 running on an Ubuntu virtual machines for testing and so far, I have not found a solution which (a) works with PHP 5.5. and (b) submits checkboxes correctly.

Stay tuned.

WordPress Google Forms v0.64-beta-2

This evening I released beta-2 of the upcoming v0.64 release of WordPress Google Forms.  This build addresses a problem recently reported on the Support Forum where the responses for checkboxes wasn’t being recorded properly.

I was able to verify this problem using one of my test forms and began looking into it.  It turns out, the changes made in v0.63 to allow the plugin to work with later versions of PHP 5.4 breaks the ability to uses checkboxes.  This build employs a similar method of submitting parameters to Google that versions prior to v0.63 did.

Handling checkboxes has long been a challenge due to the different ways which Python (used by Google) and PHP (used by WordPress) handle the assignment of checkbox values to a named parameter.

Please download this build and test and let me know if you run into any issues.

Google Forms Beta (7742 downloads )

wp-SwimTeam v1.41 released

This morning I released v1.41 of wp-SwimTeam. You can find the update on your WordPress Dashboard or in the WordPress plugin repository. This release addresses two bugs, one of which is pretty significant but would not affect a site where the age groups were already set up.

  1. Fixed bug which resulted in warning regarding extra output upon plugin activation.  This was introduced in the WIP role code and as the result of a reference to a non-existing role.
  2. Fixed a serious bug which prevented creation or update of standard age groups.  If age groups were already set up, this bug was unlikely to be encountered but for new installations, it was a significant issue.

Email Users v4.6.3-beta-9 available

Update:  Around noon EST I updated the build to beta-10 to fix another bug.

This morning I posed beta-9 of the upcoming Email Users v4.6.3 release.  This build includes an updated Spanish translation as well as several new options for the BCC Limit setting targeted at sites with very large numbers of users (e.g. 4k, 13k, etc.).

This build also moves some debug code I have been using to chase down the memory issue under the control of the Email Users Debug setting (which is on the plugin settings page – Dashboard->Settings->Email Users).  I have found this code to be really useful chasing down this odd memory problem however I don’t want it running all the time so putting it under the control of the debug setting makes sense.  When debug is enabled, if you choose the “Send to Users” menu (Dashboard->Email Users->Send to Users) and examine the source HTML code for the page, you will find a section of code which looks similar to this:

<pre id="line1">				<select id="<a>send_users</a>" name="<a>send_users[]</a>" size="<a>8</a>" style="<a>width: 654px; height: 250px;</a>" multiple="<a>multiple</a>" >
				<!-- email-users.php::1010 -->
<!-- email-users.php::1058 -->
<!-- email-users.php::1059 -->
<!--
Array
(
    [total_users] => 805
    [avail_roles] => Array
        (
            [administrator] => 1
            [contributor] => 4
            [editor] => 2
            [subscriber] => 798
        )

)
 -->
<!-- email-users.php::1061 -->
<!--
Array
(
    [exclude] => 1
    [fields] => Array
        (
            [0] => ID
            [1] => display_name
        )

    [offset] => 0
    [number] => 500
)
 -->
<!-- email-users.php::1063 -->
<!-- email-users.php::1082  Query #1  Memory Usage:  20.5M -->
<!-- email-users.php::1082  Query #2  Memory Usage:  20.5M -->
<!--
805 -->
<!-- email-users.php::1116 -->
					<option value="<a>308</a>" ></pre>

This sort of information is tremendously helpful when chasing down odd behavior.

I am hoping to get a couple more translation updates before I do the v4.6.3 release along with some testing from a couple of sites with large user counts.  In the meantime, you can download the latest beta version here.

Email Users Beta (4829 downloads )

Email-Users v4.6.3-beta-8 now available

I have just uploaded beta-8 of Email Users v4.6.3.  This build addresses an unusual out of memory situations which affected sites with very large numbers of users.  You can find a detailed write up of the problem here.

The change s required to address this problem are pretty substantial so I would appreciate any testing people can do, I don’t want to break anyone’s email system!

Email Users Beta (4829 downloads )

Chasing down Email-Users out of memory issues

For the past couple of days I have been looking at two sites which were experiencing an issue with Email Users where any of the pages on the Dashboard which presents a list of recipients was incomplete.  Looking closer at the pages, PHP was crashing which resulted in an incomplete page.  One of the sites had another clue, it reported a request for memory which could not be granted.

Both users provided me access where I could upload a debug version of Email-Users.  While not an ideal debug environment, I am grateful for the trust and access both users provided as I would have not been able to chase this bug down in my own development environment.

I was able to narrow the problem down to a call to get_users() which is a standard WordPress API function.  Because Email Users has been around a while, it still contains some code which was necessary in older versions of WordPress and the arguments passed to get_users() included the ‘all_with_meta’ parameter which was the only way to retrieve the first and last name of a user prior to the magic methods which were introduced for get_users() in WordPress 3.x.  The magic methods remove the need for the ‘all_with_meta’ parameter however the plugin was never updated because it wasn’t broken.

In the process of chasing down this memory problem I added some code to partition the get_users() query into blocks of 500 users and I could watch the memory usage increase with each query.  This would continue until memory was exhausted at which time PHP would terminate ungracefully and the partial page would be rendered.

An email to the wp-hackers mailing list helped me understand how much data was being cached by calling get_users() with the ‘all_with_meta’ parameter and I realized I needed to find a different solution as what I was doing wouldn’t scale.

I had encountered the get_users() magic methods previously and I realized that I no longer needed to call get_users() with the ‘all_with_meta’ parameter so I removed it and did some testing and sure enough, I was able to successfully run on both sites without any issues.  Memory usage on the site with 13K users topped out at 47M, well under the 256M maximum defined by WordPress.

In the current beta version (4.6.3-beta-8) there is still some debug code in place to monitor memory usage.  If you look at the page source you will find something like this:

<pre id="line1"><!-- email-users.php::1091  Query #1  Memory Usage:  34.5M -->
<!-- email-users.php::1091  Query #2  Memory Usage:  34.75M -->
<!-- email-users.php::1091  Query #3  Memory Usage:  35.25M -->
<!-- email-users.php::1091  Query #4  Memory Usage:  35.75M -->
<!-- email-users.php::1091  Query #5  Memory Usage:  36.5M -->
<!-- email-users.php::1091  Query #6  Memory Usage:  37M -->
<!-- email-users.php::1091  Query #7  Memory Usage:  37.5M -->
<!-- email-users.php::1091  Query #8  Memory Usage:  37.75M -->
<!-- email-users.php::1091  Query #9  Memory Usage:  38.5M -->
<!-- email-users.php::1091  Query #10  Memory Usage:  39M -->
<!-- email-users.php::1091  Query #11  Memory Usage:  39.5M -->
<!-- email-users.php::1091  Query #12  Memory Usage:  40M -->
<!-- email-users.php::1091  Query #13  Memory Usage:  40.25M -->
<!-- email-users.php::1091  Query #14  Memory Usage:  40.75M -->
<!-- email-users.php::1091  Query #15  Memory Usage:  41.25M -->
<!-- email-users.php::1091  Query #16  Memory Usage:  41.5M -->
<!-- email-users.php::1091  Query #17  Memory Usage:  42.5M -->
<!-- email-users.php::1091  Query #18  Memory Usage:  43M -->
<!-- email-users.php::1091  Query #19  Memory Usage:  43.5M -->
<!-- email-users.php::1091  Query #20  Memory Usage:  44M -->
<!-- email-users.php::1091  Query #21  Memory Usage:  44.25M -->
<!-- email-users.php::1091  Query #22  Memory Usage:  44.75M -->
<!-- email-users.php::1091  Query #23  Memory Usage:  45.25M -->
<!-- email-users.php::1091  Query #24  Memory Usage:  45.75M -->
<!-- email-users.php::1091  Query #25  Memory Usage:  46M -->
<!-- email-users.php::1091  Query #26  Memory Usage:  46.5M -->
<!-- email-users.php::1091  Query #27  Memory Usage:  47M --></pre>

I need to do some additional testing but based on these two sites now working, I am bullish on this solution to this unusual problem.

Email Users v4.6.3-beta-2 available

In the process of working with a user providing a Dutch translation, I found a number of strings which were not properly set up for language translation.  This build provides no new functionality, only changes to support translation.

This build also replaces the French translation files with new ones as the files in the release could not be opened with Poedit nor would they be loaded by WordPress.  Not sure what happened but the files were corrupt.

Note:  Updated to beta-3 late in the afternoon on 12/30.

Email Users Beta (4829 downloads )

WordPress Google Form v0.64-beta-1 available

This morning I posted the first beta of WordPress Google Form v0.64.  There is no new functionality in this version, it just addresses a bunch of strings which were not properly set up for language translation.

If you want to provide a language pack for WordPress Google Form, this is the best version to work from.  I am currently working with a user who is working on a French translation.

Google Forms Beta (7742 downloads )

Poor Response Time

I’ve been out of pocket for the past few weeks, for anyone who has submitted a bug or help request, I apologize.  I am back in North Carolina and have some time to address some of the questions and help requests which have queued up since before Christmas.  I have answered a few Email Users and WordPress Google Form questions in the WordPress Support Forum but if you haven’t heard back from me, don’t hesitate to ping me again.

WordPress Google Form v0.63-beta-2

What? Another beta release? Yes. A few weeks ago I had a report from a user who ran into a problem with WordPress Google Form running under PHP 5.4.4-14+deb7u2. None of the values in the form were being submitted to the target spreadsheet. The user was able to resolve the problem by changing how I was calling wp_remote_post() and sent me a DIFF file.

Unfortunately I had already started working on the changes I made to support multiple form instances (v0.61, v0.62) and the diff file couldn’t be applied to my source tree. I then forgot about it in my effort to get v0.61 out which had a number of other fixes in it that were affecting quite a few people.

Over the weekend I received a couple more help requests from people who had forms submitting that didn’t result in any data in the spreadsheet OR if the fields were required, messages indicating they had unanswered questions.

In looking at one of these sites in detail, everything looked correct on the WordPress side.  This site has the Suhosin security module installed.  I’ve run into Suhosin before but I couldn’t find any comments in my code nor any references to it on my web site.  I am guessing that I was never able to fix the problem when I ran into it previously.  In reading about Suhosin again, I recalled the patch which was sent to me a couple weeks ago.

My suspicion is that the way I was constructing the arguments to wp_remote_post() isn’t palatable to some versions of PHP.  I decided to implement the changes that the patch had incorporated and see if it would address the problem.  The change involves constructing the body argument as an array instead of a long URL formatted string.  This is the right way to do it and it makes the code a little cleaner when needing to look at the parameters passed to wp_remote_post().

I was able to incorporate the patch by manually finding the corresponding source code lines in my current version and accounting for the changes I have made in the past month and get a beta release together fairly quickly.  This fix allows the test form on the sites (one running PHP 5.4.6, one running PHP 5.3.10-1ubuntu3.8 with Suhosin-Patch).

If you want to try out this latest version, I would appreciate any feedback.

Google Forms Beta (7742 downloads )