Chasing down Email-Users out of memory issues

For the past couple of days I have been looking at two sites which were experiencing an issue with Email Users where any of the pages on the Dashboard which presents a list of recipients was incomplete.  Looking closer at the pages, PHP was crashing which resulted in an incomplete page.  One of the sites had another clue, it reported a request for memory which could not be granted.

Both users provided me access where I could upload a debug version of Email-Users.  While not an ideal debug environment, I am grateful for the trust and access both users provided as I would have not been able to chase this bug down in my own development environment.

I was able to narrow the problem down to a call to get_users() which is a standard WordPress API function.  Because Email Users has been around a while, it still contains some code which was necessary in older versions of WordPress and the arguments passed to get_users() included the ‘all_with_meta’ parameter which was the only way to retrieve the first and last name of a user prior to the magic methods which were introduced for get_users() in WordPress 3.x.  The magic methods remove the need for the ‘all_with_meta’ parameter however the plugin was never updated because it wasn’t broken.

In the process of chasing down this memory problem I added some code to partition the get_users() query into blocks of 500 users and I could watch the memory usage increase with each query.  This would continue until memory was exhausted at which time PHP would terminate ungracefully and the partial page would be rendered.

An email to the wp-hackers mailing list helped me understand how much data was being cached by calling get_users() with the ‘all_with_meta’ parameter and I realized I needed to find a different solution as what I was doing wouldn’t scale.

I had encountered the get_users() magic methods previously and I realized that I no longer needed to call get_users() with the ‘all_with_meta’ parameter so I removed it and did some testing and sure enough, I was able to successfully run on both sites without any issues.  Memory usage on the site with 13K users topped out at 47M, well under the 256M maximum defined by WordPress.

In the current beta version (4.6.3-beta-8) there is still some debug code in place to monitor memory usage.  If you look at the page source you will find something like this:

<pre id="line1"><!-- email-users.php::1091  Query #1  Memory Usage:  34.5M -->
<!-- email-users.php::1091  Query #2  Memory Usage:  34.75M -->
<!-- email-users.php::1091  Query #3  Memory Usage:  35.25M -->
<!-- email-users.php::1091  Query #4  Memory Usage:  35.75M -->
<!-- email-users.php::1091  Query #5  Memory Usage:  36.5M -->
<!-- email-users.php::1091  Query #6  Memory Usage:  37M -->
<!-- email-users.php::1091  Query #7  Memory Usage:  37.5M -->
<!-- email-users.php::1091  Query #8  Memory Usage:  37.75M -->
<!-- email-users.php::1091  Query #9  Memory Usage:  38.5M -->
<!-- email-users.php::1091  Query #10  Memory Usage:  39M -->
<!-- email-users.php::1091  Query #11  Memory Usage:  39.5M -->
<!-- email-users.php::1091  Query #12  Memory Usage:  40M -->
<!-- email-users.php::1091  Query #13  Memory Usage:  40.25M -->
<!-- email-users.php::1091  Query #14  Memory Usage:  40.75M -->
<!-- email-users.php::1091  Query #15  Memory Usage:  41.25M -->
<!-- email-users.php::1091  Query #16  Memory Usage:  41.5M -->
<!-- email-users.php::1091  Query #17  Memory Usage:  42.5M -->
<!-- email-users.php::1091  Query #18  Memory Usage:  43M -->
<!-- email-users.php::1091  Query #19  Memory Usage:  43.5M -->
<!-- email-users.php::1091  Query #20  Memory Usage:  44M -->
<!-- email-users.php::1091  Query #21  Memory Usage:  44.25M -->
<!-- email-users.php::1091  Query #22  Memory Usage:  44.75M -->
<!-- email-users.php::1091  Query #23  Memory Usage:  45.25M -->
<!-- email-users.php::1091  Query #24  Memory Usage:  45.75M -->
<!-- email-users.php::1091  Query #25  Memory Usage:  46M -->
<!-- email-users.php::1091  Query #26  Memory Usage:  46.5M -->
<!-- email-users.php::1091  Query #27  Memory Usage:  47M --></pre>

I need to do some additional testing but based on these two sites now working, I am bullish on this solution to this unusual problem.

WordPress Google Form v0.64-beta-1 available

This morning I posted the first beta of WordPress Google Form v0.64.  There is no new functionality in this version, it just addresses a bunch of strings which were not properly set up for language translation.

If you want to provide a language pack for WordPress Google Form, this is the best version to work from.  I am currently working with a user who is working on a French translation.

Google Forms Beta (40532 downloads )

WordPress Google Form v0.61 is out!

After a couple days of testing and five beta releases, I have released WordPress Google Form v0.61.  This build supports multiple instances of the same form on a single page.

Why would you do this?  It turns out, it is a fairly common request.  A number of people have uses of the same form where some of the fields are hidden and preset (both features were added fairly recently) which allows them to present the form in different ways with seeded input while allowing the user to complete the rest.  Because the form instances are all based on the same Google Form, they will have the exact same entry elements with the exact same attribute IDs.  HTML does not allow multiple elements to have the same ID attribute, the behavior is unpredictable.

To support this new feature, a new parameter, uid, has been added to the wpgform shortcode.  The uid parameter can be set to any string which is legal for an HTML element ID attribute.  When the Google Form is processed, all of the attributes are modified to include the uid parameter to ensure they each have a unique value.

[wpgform id='879' uid='B-']

GForm_SS_69

This release also addresses some issues with missing CAPTCHAs which would happen under certain circumstances.  The jQuery generation is much cleaner now as well.

You can find this update in the WordPress plugin repository or on your WordPress Dashboard.

WordPress Google Form v0.61-beta-2 available

The second beta build of WordPress Google Form v0.61 is now available.  This beta build addresses some jQuery syntax errors introduced in the process of supporting multiple form instances.  It also addresses a bug where the CAPTCHA would not be output as part of the form under certain circumstances.

Please download and test this new build if you have time and report anything which isn’t working.

Google Forms Beta (40532 downloads )

WordPress Google Form v0.61-beta-1 available

About a week ago I was approached with an interesting problem.  A user wanted to have the same form on single page three times.  Three instances of the same form.  Each instance would have some hidden values to determine which form was submitted.

The problem was by putting the same exact form on the page multiple times, a lot of the content (id and name attributes) was duplicated and as such, caused problems upon submission or even trying to move from field to field on the form.  The current (v0.60) version of the plugin is effectively broken for multiple forms except in the simplest of cases (no CAPTCHA, validation, presets, etc.).

I’ve come up with a solution that needs some testing.  I’ve added a new short code attribute to the wpgform shortcode which takes a string value and uses it as a unique identifier to ensure the replicated fields are actually unique in the source HTML.

In the image below you can see the text “B-” has been prepended to the id attribute for the form tag and an input tag.  The “B-” was the value of the uid parameter in the shortcode for the form.

[wpgform id='879' uid='B-']

GForm_SS_69

Download this beta version and run it through its paces.  The ripple effect of this change across the code was pretty significant so I’d like to make sure it didn’t break anything.

Google Forms Beta (40532 downloads )

WordPress Google Form v0.59-beta-3 now available

The beta-3 release of WordPress Google Form v0.59 is now available for testing.  This latest update includes basic support for using the plugin to embed Google Spreadsheets in WordPress (yes, it can do that – see this post).  When you use the published HTML page URL for a Google Spreadsheet as the URL source when defining a Google Form, you will end up with something which looks like this:

GForm_SS_68

You can view this spreadsheet/form page here.  You can use form specific Custom CSS to tailor the table to meet your needs too.  I used the following Custom CSS to get the columns evenly spaced:

tr.rShim ~ tr td { width: 33% !important;}

Google Forms Beta (40532 downloads )

WordPress Google Form v0.59-beta-2 now available

This morning I released beta-2 of WordPress Google Form v0.59.  This build introduces one new feature (hidden fields) and fixes one limitation (validation rules).

Much like validation rules and placeholders, an input field can now be defined as hidden.  When a field is defined as hidden, it does not appear to the user when they view the form and the value is set to a fixed value (e.g. a static string) or to something WordPress derives (e.g. the user’s IP address).

GForm_SS_64

The format of the field name is exactly the same as used for validation and placeholders.

Validation has been improved and the limitation of one validation rule per field has been lifted.  You can now define multiple validation rules for a single field.  Simply enter the field name for each separate type of validation.

Google Forms Beta (40532 downloads )

wp-SwimTeam v1.40 released

I have finally released v1.40 of wp-SwimTeam.  Thanks to everyone who helped test the new features and flush out any bugs.  This release has a slew of new features and functionality.

  • This update requires phpHtmlLib v2.6.7 or later!
  • Fixed initializion bug when Adding Events to an Event Group. Events were by default not being assigned to the correct group.
  • Fixed pagination bug when managing large number of events. When paging through events the GUI would get confused.
  • Fixed Event Group bug which was propagating the wrong event group to subsequent actions.
  • Added support for Mixed Gender and Combined age groups.
  • Added support for Mixed Gender and Combined Events.
  • Added support for exporting mixed gender event entries in SD3 and HY3 formats.
  • Added support for mixed gender events when importing events.
  • Fixed bug resulting in database error when generating RE1 roster.
  • Re-implemented Roster Export functionality – multiple formats can now be exported. The export can also be limited one gender.
  • Added support for age groups where min age and max age are identical.
  • Added smart processing of 0 as min or max age in HYV event imports, mapping 0 to minimum or maximum age setting as appropriate.
  • Fixed bug in USS Number generation.
  • Fixed bug in HY3 entries generation where E1 record did not contain gender.
  • Fixed reported entry count when exporting SDIF, count was off by 2x.
  • Fixed bug exporting roster by single gender which resulted in no swimmers.
  • Fixed bug which prevented proper HY3 meet entry generation for Opt-In swim meets.

You can find the update in the WordPress plugin repository or on your Dashboard.