Chasing down Email-Users out of memory issues

For the past couple of days I have been looking at two sites which were experiencing an issue with Email Users where any of the pages on the Dashboard which presents a list of recipients was incomplete.  Looking closer at the pages, PHP was crashing which resulted in an incomplete page.  One of the sites had another clue, it reported a request for memory which could not be granted.

Both users provided me access where I could upload a debug version of Email-Users.  While not an ideal debug environment, I am grateful for the trust and access both users provided as I would have not been able to chase this bug down in my own development environment.

I was able to narrow the problem down to a call to get_users() which is a standard WordPress API function.  Because Email Users has been around a while, it still contains some code which was necessary in older versions of WordPress and the arguments passed to get_users() included the ‘all_with_meta’ parameter which was the only way to retrieve the first and last name of a user prior to the magic methods which were introduced for get_users() in WordPress 3.x.  The magic methods remove the need for the ‘all_with_meta’ parameter however the plugin was never updated because it wasn’t broken.

In the process of chasing down this memory problem I added some code to partition the get_users() query into blocks of 500 users and I could watch the memory usage increase with each query.  This would continue until memory was exhausted at which time PHP would terminate ungracefully and the partial page would be rendered.

An email to the wp-hackers mailing list helped me understand how much data was being cached by calling get_users() with the ‘all_with_meta’ parameter and I realized I needed to find a different solution as what I was doing wouldn’t scale.

I had encountered the get_users() magic methods previously and I realized that I no longer needed to call get_users() with the ‘all_with_meta’ parameter so I removed it and did some testing and sure enough, I was able to successfully run on both sites without any issues.  Memory usage on the site with 13K users topped out at 47M, well under the 256M maximum defined by WordPress.

In the current beta version (4.6.3-beta-8) there is still some debug code in place to monitor memory usage.  If you look at the page source you will find something like this:

<pre id="line1"><!-- email-users.php::1091  Query #1  Memory Usage:  34.5M -->
<!-- email-users.php::1091  Query #2  Memory Usage:  34.75M -->
<!-- email-users.php::1091  Query #3  Memory Usage:  35.25M -->
<!-- email-users.php::1091  Query #4  Memory Usage:  35.75M -->
<!-- email-users.php::1091  Query #5  Memory Usage:  36.5M -->
<!-- email-users.php::1091  Query #6  Memory Usage:  37M -->
<!-- email-users.php::1091  Query #7  Memory Usage:  37.5M -->
<!-- email-users.php::1091  Query #8  Memory Usage:  37.75M -->
<!-- email-users.php::1091  Query #9  Memory Usage:  38.5M -->
<!-- email-users.php::1091  Query #10  Memory Usage:  39M -->
<!-- email-users.php::1091  Query #11  Memory Usage:  39.5M -->
<!-- email-users.php::1091  Query #12  Memory Usage:  40M -->
<!-- email-users.php::1091  Query #13  Memory Usage:  40.25M -->
<!-- email-users.php::1091  Query #14  Memory Usage:  40.75M -->
<!-- email-users.php::1091  Query #15  Memory Usage:  41.25M -->
<!-- email-users.php::1091  Query #16  Memory Usage:  41.5M -->
<!-- email-users.php::1091  Query #17  Memory Usage:  42.5M -->
<!-- email-users.php::1091  Query #18  Memory Usage:  43M -->
<!-- email-users.php::1091  Query #19  Memory Usage:  43.5M -->
<!-- email-users.php::1091  Query #20  Memory Usage:  44M -->
<!-- email-users.php::1091  Query #21  Memory Usage:  44.25M -->
<!-- email-users.php::1091  Query #22  Memory Usage:  44.75M -->
<!-- email-users.php::1091  Query #23  Memory Usage:  45.25M -->
<!-- email-users.php::1091  Query #24  Memory Usage:  45.75M -->
<!-- email-users.php::1091  Query #25  Memory Usage:  46M -->
<!-- email-users.php::1091  Query #26  Memory Usage:  46.5M -->
<!-- email-users.php::1091  Query #27  Memory Usage:  47M --></pre>

I need to do some additional testing but based on these two sites now working, I am bullish on this solution to this unusual problem.

WordPress Google Form v0.61 is out!

After a couple days of testing and five beta releases, I have released WordPress Google Form v0.61.  This build supports multiple instances of the same form on a single page.

Why would you do this?  It turns out, it is a fairly common request.  A number of people have uses of the same form where some of the fields are hidden and preset (both features were added fairly recently) which allows them to present the form in different ways with seeded input while allowing the user to complete the rest.  Because the form instances are all based on the same Google Form, they will have the exact same entry elements with the exact same attribute IDs.  HTML does not allow multiple elements to have the same ID attribute, the behavior is unpredictable.

To support this new feature, a new parameter, uid, has been added to the wpgform shortcode.  The uid parameter can be set to any string which is legal for an HTML element ID attribute.  When the Google Form is processed, all of the attributes are modified to include the uid parameter to ensure they each have a unique value.

[wpgform id='879' uid='B-']

GForm_SS_69

This release also addresses some issues with missing CAPTCHAs which would happen under certain circumstances.  The jQuery generation is much cleaner now as well.

You can find this update in the WordPress plugin repository or on your WordPress Dashboard.

WordPress Google Form v0.61-beta-2 available

The second beta build of WordPress Google Form v0.61 is now available.  This beta build addresses some jQuery syntax errors introduced in the process of supporting multiple form instances.  It also addresses a bug where the CAPTCHA would not be output as part of the form under certain circumstances.

Please download and test this new build if you have time and report anything which isn’t working.

Google Forms Beta (308 downloads)

WordPress Google Form v0.61-beta-1 available

About a week ago I was approached with an interesting problem.  A user wanted to have the same form on single page three times.  Three instances of the same form.  Each instance would have some hidden values to determine which form was submitted.

The problem was by putting the same exact form on the page multiple times, a lot of the content (id and name attributes) was duplicated and as such, caused problems upon submission or even trying to move from field to field on the form.  The current (v0.60) version of the plugin is effectively broken for multiple forms except in the simplest of cases (no CAPTCHA, validation, presets, etc.).

I’ve come up with a solution that needs some testing.  I’ve added a new short code attribute to the wpgform shortcode which takes a string value and uses it as a unique identifier to ensure the replicated fields are actually unique in the source HTML.

In the image below you can see the text “B-” has been prepended to the id attribute for the form tag and an input tag.  The “B-” was the value of the uid parameter in the shortcode for the form.

[wpgform id='879' uid='B-']

GForm_SS_69

Download this beta version and run it through its paces.  The ripple effect of this change across the code was pretty significant so I’d like to make sure it didn’t break anything.

Google Forms Beta (308 downloads)

WordPress Google Form v0.59-beta-3 now available

The beta-3 release of WordPress Google Form v0.59 is now available for testing.  This latest update includes basic support for using the plugin to embed Google Spreadsheets in WordPress (yes, it can do that – see this post).  When you use the published HTML page URL for a Google Spreadsheet as the URL source when defining a Google Form, you will end up with something which looks like this:

GForm_SS_68

You can view this spreadsheet/form page here.  You can use form specific Custom CSS to tailor the table to meet your needs too.  I used the following Custom CSS to get the columns evenly spaced:

tr.rShim ~ tr td { width: 33% !important;}

Google Forms Beta (308 downloads)

wpGForm Unintended Functionality

For the past couple days I have been helping a user who was running into some problems with WordPress Google Form.  There are two threads on the WordPress Support Forum (here and here) where we went back and forth with me trying to understand his problem.  Eventually we moved to email so he could send me some screen shots as he wasn’t working on a live site.

As you can see from the support threads, I was rather confused as to what the user was trying to do.  It turns out, this user was using WordPress Google Form to display a Google Spreadsheet as opposed to a Google Form.  Once I understood what the user was doing the questions made a whole lot more sense.

The part which still didn’t make sense was why he was using WordPress Google Form to display the spreadsheet instead of a plugin dedicated for the task (e.g. Inline Google Spreadsheet Viewer).  I had never tried using my plugin to view a spreadsheet as it was never designed for that purpose.

As it turns out, for the most part it works.  If you publish a spreadsheet and use the URL when defining a Google Form, you will end up with something which looks like this:

GForm_SS_65

Surprisingly, it doesn’t look all that bad.  With a little bit of Custom CSS, it could actually look pretty good.  Here is some form specific Custom CSS I added to the form definition:

td.hd {
 display: none;
}
tr.rShim td {
 width: auto !important;
}
div div span.powered {
 display: none;
}
div.listview {
 display: none;
}

The result now looks pretty good.

GForm_SS_66

The header rows and and the table content are actually output by Google in separate tables.  It would be nice if they were in a single table – I am not sure of the logic behind having them as separate tables but that is what Google generates.

I am adding some new functionality to support this unusual usage of the plugin.  Because the Google “Powered By” block contains a link to the original spreadsheet, which often times is a undesirable, if the Legal option is turned off for the defined form, the “Powered By” block will be removed with jQuery.

I am also going to add some basic CSS (like above) to the default CSS to support this as well.  Look for a new beta release shortly.

WordPress Google Form v0.59-beta-2 now available

This morning I released beta-2 of WordPress Google Form v0.59.  This build introduces one new feature (hidden fields) and fixes one limitation (validation rules).

Much like validation rules and placeholders, an input field can now be defined as hidden.  When a field is defined as hidden, it does not appear to the user when they view the form and the value is set to a fixed value (e.g. a static string) or to something WordPress derives (e.g. the user’s IP address).

GForm_SS_64

The format of the field name is exactly the same as used for validation and placeholders.

Validation has been improved and the limitation of one validation rule per field has been lifted.  You can now define multiple validation rules for a single field.  Simply enter the field name for each separate type of validation.

Google Forms Beta (308 downloads)

Email Users v4.6.2-beta-1 available

This morning I posted the first beta release (beta-1) of Email Users 4.6.2.  This release includes integration with the ItThinx Groups plugin.  I also took the opportunity to refactor the code for integration so code isn’t unnecessarily loaded when a plugin isn’t installed and clean up the plugin’s ReadMe file.

If you use ItThinx Groups I would appreciate any feedback you can provide.

Email Users Beta (149 downloads)

WordPress Google Form v0.59-beta-1 available

I have posted WordPress Google Form v0.59-beta-1 in response to a request to have the ability to preset form values by passing a parameter as part of the WordPress URL.   This is an interesting request so I did some work on it this morning and it turned out it wasn’t to difficult to implement.

Much like custom field validation and presets, this new functionality builds upon the name of the field that you want to preset.  Google has a way to preset values, the field name is passed as a parameter with a value.  This new functionality does much the same but pulls it back to WordPress for processing.

Using my Validation Demo form as an example, here are two URLs:

  1. Standard URL
  2. Standard URL with preset values

As you can see, the second URL will have the first two fields prefilled with the values which were part of the URL.  The following was appended to the URL.  The specific parameter names (e.g. entry.1666054688) had to discovered by examining the source Google Form.

?entry.1666054688=joe@example.com&entry.327623672=www.yahoo.com

GForm_SS_63

WordPress Google Form v0.58-beta-3 now available

This morning I released beta-3 of WordPress Google Form v0.58.  This update addresses one problem, properly handling newline characters in TEXTAREA entry boxes.  This issue was reported on the WordPress Support Forum.

If there are no more reported issues, I will likely release v0.58 in the next day or so.

Google Forms Beta (308 downloads)