Chasing down Email-Users out of memory issues

For the past couple of days I have been looking at two sites which were experiencing an issue with Email Users where any of the pages on the Dashboard which presents a list of recipients was incomplete.  Looking closer at the pages, PHP was crashing which resulted in an incomplete page.  One of the sites had another clue, it reported a request for memory which could not be granted.

Both users provided me access where I could upload a debug version of Email-Users.  While not an ideal debug environment, I am grateful for the trust and access both users provided as I would have not been able to chase this bug down in my own development environment.

I was able to narrow the problem down to a call to get_users() which is a standard WordPress API function.  Because Email Users has been around a while, it still contains some code which was necessary in older versions of WordPress and the arguments passed to get_users() included the ‘all_with_meta’ parameter which was the only way to retrieve the first and last name of a user prior to the magic methods which were introduced for get_users() in WordPress 3.x.  The magic methods remove the need for the ‘all_with_meta’ parameter however the plugin was never updated because it wasn’t broken.

In the process of chasing down this memory problem I added some code to partition the get_users() query into blocks of 500 users and I could watch the memory usage increase with each query.  This would continue until memory was exhausted at which time PHP would terminate ungracefully and the partial page would be rendered.

An email to the wp-hackers mailing list helped me understand how much data was being cached by calling get_users() with the ‘all_with_meta’ parameter and I realized I needed to find a different solution as what I was doing wouldn’t scale.

I had encountered the get_users() magic methods previously and I realized that I no longer needed to call get_users() with the ‘all_with_meta’ parameter so I removed it and did some testing and sure enough, I was able to successfully run on both sites without any issues.  Memory usage on the site with 13K users topped out at 47M, well under the 256M maximum defined by WordPress.

In the current beta version (4.6.3-beta-8) there is still some debug code in place to monitor memory usage.  If you look at the page source you will find something like this:

<pre id="line1"><!-- email-users.php::1091  Query #1  Memory Usage:  34.5M -->
<!-- email-users.php::1091  Query #2  Memory Usage:  34.75M -->
<!-- email-users.php::1091  Query #3  Memory Usage:  35.25M -->
<!-- email-users.php::1091  Query #4  Memory Usage:  35.75M -->
<!-- email-users.php::1091  Query #5  Memory Usage:  36.5M -->
<!-- email-users.php::1091  Query #6  Memory Usage:  37M -->
<!-- email-users.php::1091  Query #7  Memory Usage:  37.5M -->
<!-- email-users.php::1091  Query #8  Memory Usage:  37.75M -->
<!-- email-users.php::1091  Query #9  Memory Usage:  38.5M -->
<!-- email-users.php::1091  Query #10  Memory Usage:  39M -->
<!-- email-users.php::1091  Query #11  Memory Usage:  39.5M -->
<!-- email-users.php::1091  Query #12  Memory Usage:  40M -->
<!-- email-users.php::1091  Query #13  Memory Usage:  40.25M -->
<!-- email-users.php::1091  Query #14  Memory Usage:  40.75M -->
<!-- email-users.php::1091  Query #15  Memory Usage:  41.25M -->
<!-- email-users.php::1091  Query #16  Memory Usage:  41.5M -->
<!-- email-users.php::1091  Query #17  Memory Usage:  42.5M -->
<!-- email-users.php::1091  Query #18  Memory Usage:  43M -->
<!-- email-users.php::1091  Query #19  Memory Usage:  43.5M -->
<!-- email-users.php::1091  Query #20  Memory Usage:  44M -->
<!-- email-users.php::1091  Query #21  Memory Usage:  44.25M -->
<!-- email-users.php::1091  Query #22  Memory Usage:  44.75M -->
<!-- email-users.php::1091  Query #23  Memory Usage:  45.25M -->
<!-- email-users.php::1091  Query #24  Memory Usage:  45.75M -->
<!-- email-users.php::1091  Query #25  Memory Usage:  46M -->
<!-- email-users.php::1091  Query #26  Memory Usage:  46.5M -->
<!-- email-users.php::1091  Query #27  Memory Usage:  47M --></pre>

I need to do some additional testing but based on these two sites now working, I am bullish on this solution to this unusual problem.

Email Users v4.6.3-beta-2 available

In the process of working with a user providing a Dutch translation, I found a number of strings which were not properly set up for language translation.  This build provides no new functionality, only changes to support translation.

This build also replaces the French translation files with new ones as the files in the release could not be opened with Poedit nor would they be loaded by WordPress.  Not sure what happened but the files were corrupt.

Note:  Updated to beta-3 late in the afternoon on 12/30.

Email Users Beta (12588 downloads )

WordPress Google Form v0.64-beta-1 available

This morning I posted the first beta of WordPress Google Form v0.64.  There is no new functionality in this version, it just addresses a bunch of strings which were not properly set up for language translation.

If you want to provide a language pack for WordPress Google Form, this is the best version to work from.  I am currently working with a user who is working on a French translation.

Google Forms Beta (17143 downloads )

WordPress Google Form v0.61 is out!

After a couple days of testing and five beta releases, I have released WordPress Google Form v0.61.  This build supports multiple instances of the same form on a single page.

Why would you do this?  It turns out, it is a fairly common request.  A number of people have uses of the same form where some of the fields are hidden and preset (both features were added fairly recently) which allows them to present the form in different ways with seeded input while allowing the user to complete the rest.  Because the form instances are all based on the same Google Form, they will have the exact same entry elements with the exact same attribute IDs.  HTML does not allow multiple elements to have the same ID attribute, the behavior is unpredictable.

To support this new feature, a new parameter, uid, has been added to the wpgform shortcode.  The uid parameter can be set to any string which is legal for an HTML element ID attribute.  When the Google Form is processed, all of the attributes are modified to include the uid parameter to ensure they each have a unique value.

[wpgform id='879' uid='B-']

GForm_SS_69

This release also addresses some issues with missing CAPTCHAs which would happen under certain circumstances.  The jQuery generation is much cleaner now as well.

You can find this update in the WordPress plugin repository or on your WordPress Dashboard.

WordPress Google Form v0.61-beta-5 available

Another beta update, #5, for WordPress Google Form v0.61.  This update fixes a couple more issues with validation which resulted in some extra jQuery content that was malformed.  It only happened IF validation was enabled but no custom validation rules were defined.

Google Forms Beta (17143 downloads )

WordPress Google Form v0.61-beta-4 now available

I have just uploaded beta-4 of WordPress Google Form v0.61. This build corrects a couple more jQuery issues when using multiple instances of a form on the same page. In order to use multiple instances of the same form on a single page, you must use the uid parameter with the wpgform shortcode. Usage of this new parameter is outlined in the v0.61-beta-1 announcement.

The uid parameter is not supported in the deprecated gform shortcode.

Please download and test this beta release, all feedback is much appreciated.

Google Forms Beta (17143 downloads )

WordPress Google Form v0.61-beta-1 available

About a week ago I was approached with an interesting problem.  A user wanted to have the same form on single page three times.  Three instances of the same form.  Each instance would have some hidden values to determine which form was submitted.

The problem was by putting the same exact form on the page multiple times, a lot of the content (id and name attributes) was duplicated and as such, caused problems upon submission or even trying to move from field to field on the form.  The current (v0.60) version of the plugin is effectively broken for multiple forms except in the simplest of cases (no CAPTCHA, validation, presets, etc.).

I’ve come up with a solution that needs some testing.  I’ve added a new short code attribute to the wpgform shortcode which takes a string value and uses it as a unique identifier to ensure the replicated fields are actually unique in the source HTML.

In the image below you can see the text “B-” has been prepended to the id attribute for the form tag and an input tag.  The “B-” was the value of the uid parameter in the shortcode for the form.

[wpgform id='879' uid='B-']

GForm_SS_69

Download this beta version and run it through its paces.  The ripple effect of this change across the code was pretty significant so I’d like to make sure it didn’t break anything.

Google Forms Beta (17143 downloads )

WordPress Google Form v0.59 released

After I have not had any reports of issues since releasing beta-3 so I have pushed out the formal release.  You can find v0.59 in the WordPress plugin repository or on your Dashboard.

Enhancements and Bug Fixes in v0.59:

  • Added ability to preset values for Google form as part of WordPress URL.
  • Added new CSS declarations to default plugin CSS to account for recent changes by Google to Forms.
  • Added ability to define fields as “hidden” and preset with a user defined or system defined value.
  • Fixed validation limitation which only allowed one validation rule per input.
  • Added basic support (CSS, jQuery) to use WordPress Google Form to view a Google Spreadsheet within WordPress.

WordPress Google Form v0.59-beta-3 now available

The beta-3 release of WordPress Google Form v0.59 is now available for testing.  This latest update includes basic support for using the plugin to embed Google Spreadsheets in WordPress (yes, it can do that – see this post).  When you use the published HTML page URL for a Google Spreadsheet as the URL source when defining a Google Form, you will end up with something which looks like this:

GForm_SS_68

You can view this spreadsheet/form page here.  You can use form specific Custom CSS to tailor the table to meet your needs too.  I used the following Custom CSS to get the columns evenly spaced:

tr.rShim ~ tr td { width: 33% !important;}

Google Forms Beta (17143 downloads )