Chasing Checkbox Support

For the past couple of days I have continued to look at the problem I wrote about with Google Forms and Checkboxes with PHP 5.4 and PHP 5.5.  It turns out, that it really isn’t PHP version related, at least I don’t think so.  It is WordPress related as near as I can tell.

As I’ve written before, I love VMware Workstation as it allows you to create virtual machines for very specific purposes, use them as long as needed, then put them away until needed again.

I created an Unbuntu VM to play with Ruby on Rails a couple months ago so I decided to check what version of PHP it had running.  Lo and behold, it had PHP 5.5.x running so it was a good platform to further test potential solutions for my checkbox problem.  My Windows environment is running PHP 5.3.3 and is still running WordPress 3.6.1 (don’t ask why but it proved to be very useful that it was).

I quickly set up WordPress 3.8 on Ubuntu and sure enough, submitting the form failed.  The exact same plugin code on my Windows VM submitted correctly.  I continued to dig through my code and eventually into WordPress itself trying to see what was different.

I eventually started looking at the source to wp_remote_post() and decided to identify which transport was being used.  On Windows WordPress was using cURL but on Ubuntu it was using Streams.  Ah-ha.  Since the WordPress HTTP API abstracts the details of the transports away from the application, it shouldn’t matter but it seems to.  I continued digging.

Using an advanced feature of WordPress Google Form to control transports, I disabled the cURL transport on the Windows machine and much to my surprise, the form still submitted correctly.  Now this is odd.  So I then installed cURL on Ubuntu and the form submitted correctly.  The good news is there at least appears to be a solution although I’d prefer to not have cURL be a requirement.

Now what was different?  I started looking into WordPress’ class-http.php file (which defines the WP_Http class) and noticed that the file on Windows was very different than the file on Ubuntu.  Looks like something changed between 3.6.1 and 3.8.1.

I decided to download the 3.6.1 and 3.7.1 releases from the WordPress archive and do some tests on Ubuntu where it is trivial to switch between WordPress releases.

To add a little more information, I did some testing with older versions of WordPress in combination with the http_api_transports to force a specific transport (‘streams’, ‘fsockopen’, and ‘curl’).  I found the following results:

WordPress 3.6.1 WordPress 3.7.1 WordPress 3.8.1
cURL Success Success Success
fsockopen Success Fail Fail
streams Success Fail Fail

Something clearly changed with the streams and fsockopen transports between 3.6.1 and 3.7.1.  A diff of the class-http.php file shows the change was substantial as the files are significantly different.

At this point I have concluded that my plugin will only work with sites where cURL is available.  I will probably release a version which displays a warning on the Dashboard if cURL is not available and that usage of the plugin is not recommended.

Google Forms, Checkboxes and PHP 5.4 or 5.5

WordPress Google Form v0.63 introduced a change  which allowed the plugin to work on newer versions of PHP, notably 5.4 and 5.5.  Unfortunately that change has broken support for checkboxes.

For those who are new to the plugin or never needed to know how it worked, what the plugin does is retrieve the HTML for the form from Google and render it within the context of WordPress.  When the form is submitted, it is actually submitted within the context of WordPress.   The data is collected by the plugin and then submitted to Google.  The retrieval from and submission to Google is done with the WordPress HTTP API.  In particular, the wp_remote_get() and wp_remote_post() functions are used to retrieve and submit the form.

To complicate the problem further, Google uses Python as the backend for their form processor where as WordPress uses PHP.  For the most part, the fact that they are based on different scripting languages isn’t a big deal.  Until you get to checkboxes.  Checkboxes in Python are handled differently than they are in PHP.

I had solved the compatibility problem a couple years ago (see this thread on the wp-hackers mailing list) using a small jQuery script which fixed the form variables on the WordPress side and manual construction of the body parameter for wp_remote_post() when submitting the data to Google.  This solution worked fine until I received a bug report that nothing was being submitted on a site which was running PHP 5.4.x.

Fortunately the user who encountered the problem provided me with a patch that I was able to fold in which changed the way the body parameter was constructed (array instead of a string) which worked with PHP 5.4.x and also worked with older versions.  However, I didn’t test it thoroughly as I have had several reports that checkbox content was not being submitted correctly.  Uh-oh.  I was able to verify the problem fairly quickly and was able to push out a version which essentially reverts how the body parameter is constructed (string instead of an array).  The problem is, this build doesn’t work with newer versions of PHP.

I have PHP 5.5 running on an Ubuntu virtual machines for testing and so far, I have not found a solution which (a) works with PHP 5.5. and (b) submits checkboxes correctly.

Stay tuned.

wp-SwimTeam v1.41 released

This morning I released v1.41 of wp-SwimTeam. You can find the update on your WordPress Dashboard or in the WordPress plugin repository. This release addresses two bugs, one of which is pretty significant but would not affect a site where the age groups were already set up.

  1. Fixed bug which resulted in warning regarding extra output upon plugin activation.  This was introduced in the WIP role code and as the result of a reference to a non-existing role.
  2. Fixed a serious bug which prevented creation or update of standard age groups.  If age groups were already set up, this bug was unlikely to be encountered but for new installations, it was a significant issue.

Email-Users v4.6.3-beta-8 now available

I have just uploaded beta-8 of Email Users v4.6.3.  This build addresses an unusual out of memory situations which affected sites with very large numbers of users.  You can find a detailed write up of the problem here.

The change s required to address this problem are pretty substantial so I would appreciate any testing people can do, I don’t want to break anyone’s email system!

Email Users Beta (71 downloads)

Chasing down Email-Users out of memory issues

For the past couple of days I have been looking at two sites which were experiencing an issue with Email Users where any of the pages on the Dashboard which presents a list of recipients was incomplete.  Looking closer at the pages, PHP was crashing which resulted in an incomplete page.  One of the sites had another clue, it reported a request for memory which could not be granted.

Both users provided me access where I could upload a debug version of Email-Users.  While not an ideal debug environment, I am grateful for the trust and access both users provided as I would have not been able to chase this bug down in my own development environment.

I was able to narrow the problem down to a call to get_users() which is a standard WordPress API function.  Because Email Users has been around a while, it still contains some code which was necessary in older versions of WordPress and the arguments passed to get_users() included the ‘all_with_meta’ parameter which was the only way to retrieve the first and last name of a user prior to the magic methods which were introduced for get_users() in WordPress 3.x.  The magic methods remove the need for the ‘all_with_meta’ parameter however the plugin was never updated because it wasn’t broken.

In the process of chasing down this memory problem I added some code to partition the get_users() query into blocks of 500 users and I could watch the memory usage increase with each query.  This would continue until memory was exhausted at which time PHP would terminate ungracefully and the partial page would be rendered.

An email to the wp-hackers mailing list helped me understand how much data was being cached by calling get_users() with the ‘all_with_meta’ parameter and I realized I needed to find a different solution as what I was doing wouldn’t scale.

I had encountered the get_users() magic methods previously and I realized that I no longer needed to call get_users() with the ‘all_with_meta’ parameter so I removed it and did some testing and sure enough, I was able to successfully run on both sites without any issues.  Memory usage on the site with 13K users topped out at 47M, well under the 256M maximum defined by WordPress.

In the current beta version (4.6.3-beta-8) there is still some debug code in place to monitor memory usage.  If you look at the page source you will find something like this:

<pre id="line1"><!-- email-users.php::1091  Query #1  Memory Usage:  34.5M -->
<!-- email-users.php::1091  Query #2  Memory Usage:  34.75M -->
<!-- email-users.php::1091  Query #3  Memory Usage:  35.25M -->
<!-- email-users.php::1091  Query #4  Memory Usage:  35.75M -->
<!-- email-users.php::1091  Query #5  Memory Usage:  36.5M -->
<!-- email-users.php::1091  Query #6  Memory Usage:  37M -->
<!-- email-users.php::1091  Query #7  Memory Usage:  37.5M -->
<!-- email-users.php::1091  Query #8  Memory Usage:  37.75M -->
<!-- email-users.php::1091  Query #9  Memory Usage:  38.5M -->
<!-- email-users.php::1091  Query #10  Memory Usage:  39M -->
<!-- email-users.php::1091  Query #11  Memory Usage:  39.5M -->
<!-- email-users.php::1091  Query #12  Memory Usage:  40M -->
<!-- email-users.php::1091  Query #13  Memory Usage:  40.25M -->
<!-- email-users.php::1091  Query #14  Memory Usage:  40.75M -->
<!-- email-users.php::1091  Query #15  Memory Usage:  41.25M -->
<!-- email-users.php::1091  Query #16  Memory Usage:  41.5M -->
<!-- email-users.php::1091  Query #17  Memory Usage:  42.5M -->
<!-- email-users.php::1091  Query #18  Memory Usage:  43M -->
<!-- email-users.php::1091  Query #19  Memory Usage:  43.5M -->
<!-- email-users.php::1091  Query #20  Memory Usage:  44M -->
<!-- email-users.php::1091  Query #21  Memory Usage:  44.25M -->
<!-- email-users.php::1091  Query #22  Memory Usage:  44.75M -->
<!-- email-users.php::1091  Query #23  Memory Usage:  45.25M -->
<!-- email-users.php::1091  Query #24  Memory Usage:  45.75M -->
<!-- email-users.php::1091  Query #25  Memory Usage:  46M -->
<!-- email-users.php::1091  Query #26  Memory Usage:  46.5M -->
<!-- email-users.php::1091  Query #27  Memory Usage:  47M --></pre>

I need to do some additional testing but based on these two sites now working, I am bullish on this solution to this unusual problem.

WordPress Google Form v0.61-beta-5 available

Another beta update, #5, for WordPress Google Form v0.61.  This update fixes a couple more issues with validation which resulted in some extra jQuery content that was malformed.  It only happened IF validation was enabled but no custom validation rules were defined.

Google Forms Beta (157 downloads)

WordPress Google Form v0.61-beta-2 available

The second beta build of WordPress Google Form v0.61 is now available.  This beta build addresses some jQuery syntax errors introduced in the process of supporting multiple form instances.  It also addresses a bug where the CAPTCHA would not be output as part of the form under certain circumstances.

Please download and test this new build if you have time and report anything which isn’t working.

Google Forms Beta (157 downloads)

WordPress Google Form v0.59 released

After I have not had any reports of issues since releasing beta-3 so I have pushed out the formal release.  You can find v0.59 in the WordPress plugin repository or on your Dashboard.

Enhancements and Bug Fixes in v0.59:

  • Added ability to preset values for Google form as part of WordPress URL.
  • Added new CSS declarations to default plugin CSS to account for recent changes by Google to Forms.
  • Added ability to define fields as “hidden” and preset with a user defined or system defined value.
  • Fixed validation limitation which only allowed one validation rule per input.
  • Added basic support (CSS, jQuery) to use WordPress Google Form to view a Google Spreadsheet within WordPress.

WordPress Google Form v0.59-beta-2 now available

This morning I released beta-2 of WordPress Google Form v0.59.  This build introduces one new feature (hidden fields) and fixes one limitation (validation rules).

Much like validation rules and placeholders, an input field can now be defined as hidden.  When a field is defined as hidden, it does not appear to the user when they view the form and the value is set to a fixed value (e.g. a static string) or to something WordPress derives (e.g. the user’s IP address).

GForm_SS_64

The format of the field name is exactly the same as used for validation and placeholders.

Validation has been improved and the limitation of one validation rule per field has been lifted.  You can now define multiple validation rules for a single field.  Simply enter the field name for each separate type of validation.

Google Forms Beta (157 downloads)

WordPress Google Form v0.58-beta-3 now available

This morning I released beta-3 of WordPress Google Form v0.58.  This update addresses one problem, properly handling newline characters in TEXTAREA entry boxes.  This issue was reported on the WordPress Support Forum.

If there are no more reported issues, I will likely release v0.58 in the next day or so.

Google Forms Beta (157 downloads)