Email-Users Debugging – why don’t my users receive email?

Periodically users post questions in the Email-Users support forum wondering why their users don’t receive email messages when they expect them to.  There are many reasons why users don’t receive email, some are within the control of Email-Users but many are not.

Email-Users User Settings

Email-Users is based on two user settings (which appear on the user’s profile page):

  1. Receive post or page notification emails
  2. Receive emails sent to multiple recipients

EU_SS_14Email-Users has a plugin level setting which allows end users to modify their own settings which is enabled by default.  When it is disabled, only users with the proper permissions can modify user settings related to Email-Users.  No matter how the user has their settings configured, a user can always receive an email when they are the only recipient.

EU_SS_15In Email-Users v4.5.2 a new Dashboard Widget has been added which reports the number of users who have their settings configured to receive each type of email.  This information also appears on the Plugin Settings Page and should either of the types of email not have any users set to receive email, a warning message is displayed at the top of the screen.

EU_SS_17Changing the Email-Users settings for a large number of users can be accomplished from the User Setting menu.  By selecting the desired Users, Notifications and Mass Email can be enabled or disabled for a large number of users at one time.  For a site that has a large number of users, the number of users which appear on each page can be changed on the Settings Page.

Test Email

Before sending out notifications or messages, it is important to validate the underlying WordPress email system is working.  On the Plugin Settings page there is the ability to send a Test Message.

EU_SS_18

Email-Users utilizes WordPress’ wp_mail() function to send email.  It does not interact with the OS or PHP mail functions directly.  By using wp_mail(), which can be overridden by other WordPress plugins, Email-Users can work with different server configurations and email infrastructure.

If you don’t receive the test message then you may have a problem with your underlying WordPress email system.  Do you receive email for user registration and comments?  If not, there is something wrong which is out of the scope of Email-Users.

On some servers plugins such as WP-Mail-SMTP (this is just an example, there are others) can be used in situations where email is being sent but isn’t being received.

There is no point in trying to send Notifications or Mass Emails if the Test Message isn’t working.

BCC Limits

Some servers limit the number of email addresses which can appear in the BCC header of an email message.  Email-Users has a setting to control the number of addresses which appear in the BCC field.  If Email-Users is working for a small number of users but isn’t working for a larger number (e.g. all of your users) , you may be dealing with a BCC limit.

If you are able to send email to 3-5 users by selecting them from the list of users, I would next try sending a message to about 10 users and see if that works.

The sending logic is only different when 1 user is selected. Anytime more than one user is selected (2, 10, 37, 67, 1000, etc.), the same logic path through the code is used.

The only thing that is different is how many emails are sent and that is based on the BCC limit setting. If there is no limit, which is the default, it sends one email with all recipients on the BCC line. For any other setting, it will send some number of emails, depending on the BCC limit. If the BCC limit was set to 30 and you had 67 recipients, it would send 3 emails: The first 30, the second 30, and the final 7.

Advanced Debugging

In some situations you may want to examine what Email-Users is doing in gory detail.  For development purposes Email-Users has a DEBUG mode.  When in debug mode Email-Users will log quite a bit of information to the PHP error console using error_log() including the actual email headers it generates.  It can be very useful to see the actual mail headers to ensure the expected recipients are actually included.

Because gaining access to the contents of the error_log() differs from web host to web host, I recommend using a WordPress debugging capability to capture it in the site’s /wp-content folder.  By setting WP_DEBUG_LOG to true in your wp-config.php file, you can easily access the resulting debug.log file.  On my development machine I use the following set up in my wp-config.php file:

define('WP_DEBUG', true); // or false
if (WP_DEBUG) {
 define('WP_DEBUG_LOG', true);
 ini_set('display_errors', 'on');
 error_reporting(E_STRICT | E_ALL);
 define('SAVEQUERIES', true);
 define('SCRIPT_DEBUG', true);
 error_log('Debug On!') ;
}

You would not want this on a production server but to temporarily chase down a problem, it works well.  Unfortunately some plugins issue a lot of notices or warnings so you can run into the “Headers already sent” problem so you may want to tweak the above settings to meet your needs.

Once your WordPress debugging settings are tuned to your liking, you need to enable Debug for Email-Users.  Doing this requires manually editing one of the plugin files.

On or about line 46 of the email-users.php file you will find a line of code which reads:

define( ‘MAILUSERS_DEBUG’, false);

You need to change the code such that it reads:

define( ‘MAILUSERS_DEBUG’, true);

After you have made this change Email-Users will output information to the error_log.

In the image below you can see a series of email headers each with 10 email addresses in the BCC field (BCC limit is set to 10).

EU_SS_19

If the information in the error_log is what is expected then there is little, if anything which can be done from Email-Users.  At this point you will need to figure out where the email is routing to and where it isn’t which is beyond the scope of the Email-Users plugin.

Anecdote

I was using Email-Users on our local Swim Team’s web site and heard from a number of parents that they weren’t receiving emails.  I noticed that all of the reports were coming from families with RoadRunner email addresses.  It turned out that Time Warner had blacklisted our Swim Team’s domain.  By filling out a form with TWC and verifying some information, we were able to get white listed but for several weeks I wasn’t sure how to resolve it.

WordPress Google Form v0.55 released

Yesterday WordPress 3.6 was released.  The bundled version of jQuery was updated to 1.10 which broke the jQuery Columnizer plugin I use to split a form into columns.  The result was jQuery would go into an infinite loop and eventually you would have to kill the page.

Fortunately someone had already encountered this problem and provided a patch to the jQuery plugin.  I have incorporated the patch and released v0.55.  If you’re running WordPress 3.6, this is a critical release.  Older versions should continue to run correctly.

You can find v0.55 on your WordPress Dashboard or in the plugin repository.

WordPress Google Form v0.54 released

This morning I released v0.54 of WordPress Google Form.  It has been a while since I’ve had any bug reports for the current version or the beta version.  Things appear pretty stable.

  • Added internationalization support for jQuery Validation messages.
  • New language support files.
  • New jQuery Validation based custom validation option.
  • Fixed problem with escaped characters ending up in Google spreadsheet.
  • Moved transport control out of debug module and into core code so it can be a permanent setting for some server environments.
  • Fixed PHP warning messages which happen with Logging Enabled when some of the server variables don’t exist.
  • Fixed bug with Form Submission Log setting stickiness.
  • Added an optional CAPTCHA message which will appear below the CAPTCHA input when set.

WordPress Google Form v0.54-beta-6 now available

I have a posted a new beta version of WordPress Google Form.  This version (v.054-beta-6) fixes a problem reported today which happens when Logging is enabled on certain servers.  This update will now test for the variables it wants to track before trying to use them.

This update also includes an enhancement with respect to HTTP API Transports.  Previously the plugin allowed control of the various transport options in debug mode.  Recently I found a case where it made sense to disable one of the transports on a permanent basis so I have pulled that functionality out of the debug module and moved it into the plugin core.

Google Forms Beta (8583 downloads )

Debugging a weird one

Yesterday I was contacted by a WordPress Google Forms user who was getting a permission denied problem when submitting his form.  After trying a number of obvious things (like disabling all other plugins), I asked him to try switching to one of the default WordPress themes.  He switched the site over to use TwentyTwelve and the problem went away.  Yeah!  I was worried that we might be dealing with another ModSecurity problem.

So we narrowed the problem down to something in his theme but what would generate a 403 error?  The user was gracious enough to let me have access to his site.  By enabling Debug, I was able to see a dump of the header information.  The header information showed a redirect was being requested (makes sense due to 403).

Using Firebug I could see that the POST variables looked correct.  Oops, maybe not.  Everything looked ok except a hidden variable that Google uses in forms to pass information between the pages of multi-page forms.  Even single page forms have this hidden variable.  On the initial load of the form, this variable should be empty (a null string) but it wasn’t.  How could that happen?

Looking at the HTML page source for the form, I could see there was quite a bit of odd stuff in the form code.  It doesn’t come from Google, where does it come from?  Because form submission worked with TwentyTwelve but didn’t work with the user’s theme, I suspected there was a filter mucking with the HTML.

Sure enough, digging through the theme’s functions.php file I found a custom filter that did several things, one of which was make a call to wptexturize().  Well that would certainly explain the manipulated HTML.

The next step was to comment out the calls to the filter to verify that was indeed the problem.  As expected, the form submitted without any issues.  At that point I wasn’t sure what to tell the user.  We could have left the filters commented out but they are part of the theme and there for a reason.  I decided to look at the filter a little closer.  It turns out, the theme defines a short code, actually, it doesn’t define one, the filter looks for one, called [raw][/raw] and if it finds it, leaves the content between the short code tags untouched.

Well this will work!  So I created a test page and placed the wpgform short code between [raw][/raw] and sure enough, the form submitted properly.

While I don’t like what the theme is doing, at least there was a work-around for it.  Unfortunately there is no way (yet) to accomplish the same thing with the Form URL.

This whole process took a couple of hours – far more than I am typically able to spend helping people but I was genuinely curious what was going on.  I am glad we got the user up and running.

I am trying to think of a good way for people to define forms which can handle such a situation.  I can define a hook but that means people need to write some code to use it or I could define more options for the plugin that allow the generated HTML to be wrapped in some prefix and suffix text (plain or HTML).  I am not sure what to do so will continue to think about it for a day or two.

WordPress Google Form v0.46 is finally out!

After 19 beta releases, the first one being back on November 14, 2012, this afternoon I released v0.46 of WordPress Google Form.  This update represents a significant change in how the plugin works.  The plugin now defines a Custom Post Type for defining forms which is much easier to use than the old shortcode.  The shortcode was getting very complicated with all of the options.

The Custom Post Type functionality is supported by a new shortcode, wpgform, which is much easier to work with as it only has one argument, the Post Id of the form.  The original gform shortcode will continue to work however it will not received new features (e.g. columns) and at sometime in the future, I will likely deprecate it.

During the development process of v0.46, Google made a significant change to Google Forms.  The change required re-work of a number of features, support for multi-page forms being the most important and hardest to figure out.  Things changed on Google’s side several times as they rolled out the update to Google Forms.  The most recent change restored the ability to control the language of certain parts of the form immediately after I had spent several days working out a jQuery solution to allow text replacement on certain form elements.

There has been quite a bit of testing of this update over the past few months and I want to thank the people who tested the early releases and reported problems.  In particular I want to thank several people who trusted me with access to their server in order to chase down and debug some very odd problems.  In the end , I think I was able to resolve all of them.

The only known issue at this point is with respect to IE7 support.  I cannot get the validation support to work with IE7 which means the CAPTCHA doesn’t work properly either.  If someone can figure it out I am happy to incorporate a fix but I have exhausted by ability to chase down an odd problem with an ancient browser.

The number of new features and fixes is pretty long, I recommend reading the v0.46 release notes here.

You can find the update in the WordPress plugin repository or as an update on your WordPress Dashboard.  If you have any problems, I have retooled my Help and Support form to mimic the fields on the Custom Post Type screen.  Don’t hesitate to use it if you run into something which isn’t working.

WordPress Google Form v0.46-beta-18 now available

I am getting close to a formal release.  This beta update adds support for using the public facing Custom Post Type posts directly.  This means when you define a Google Form using the Custom Post Type interface and publish it, the permalink can be used to view the form.  It can also be used as a menu item or link.

It is no longer necessary to add a short code to a separate page or post although it will still work.  I am strongly encouraging users to move away from the original gform shortcode with a bazillion options to using the Custom Post Type.  Going forward, new features will only be added to the Custom Post Type (e.g. columns only work with the Custom Post Type).

GForm_SS_47GForm_SS_46

Google Forms Beta (8583 downloads )

Why are my buttons in Chinese?

Periodically I get a question from someone using WordPress Google Form asking why their buttons (and some other text) is in Chinese (or some other language but Chinese is the most common)?

If you haven’t run into this problem, consider yourself lucky!  The problem manifests itself similar to the images below.

GForm_SS_35 GForm_SS_36

As near as I can tell, this problem happens when the Web server that is running WordPress Google Form is geographically located in a part of the world where Google thinks it should serve up a particular language.

So how do you fix it?  In the previous version of Google Forms, the solution was simple:  Add &hl=en to your form URL and Google would deliver the content in the specificed language (in the case English) – refer to to this post for more details.  Unfortunately the new version of Google Forms doesn’t support this language parameter so I have been at a loss to help people who find themselves in this situation.

For several days I have been trying to find a solution.  Adding cookies to the wp_remote_get() request, .htaccess entries, PHP locale settings, etc.  Nothing made a difference.

While at the gym this morning it dawned on me that there isn’t any reason why I couldn’t “fix” the problem by processing the HTML that Google generates with jQuery.  So that is what I have done.

GForm_SS_37

 

I have renamed the Debug tab to Advanced Options and added the ability to define the text that will appear on the Submit, Back, and Continue buttons as well as the text which indicates fields are Required.

WordPress Google Form v0.46-beta-13 available

This afternoon I uploaded WordPress Google Form v0.46-beta-13.  Yes, this is the 13th beta release of WordPress Google Form v0.46.  Why so many?  Mostly because made a significant change to Google Forms at about the same time I was introducing a major change to WordPress Google Forms.

This release adds a new feature – the ability to log form submissions.  This is something I’ve wanted to do for a while.  The next update will include some settings to control the log (on, off, entries per page, etc.)  but in this beta release, it is on and shows 10 entries per page.  There is a known bug in the pagination of the log file.  The URL to move between pages is wrong and I haven’t been able to determine how to add the CPT and page information which the URL requires to be valid.

Most importantly, this beta release addresses a major bug with multi-page Google Forms AND Google Forms with checkboxes that are created with the new version of Google Forms.  This functionality needs testing so please put it through its paces.

Google Forms Beta (8583 downloads )

New Google Forms break multi-page support

Ugh. Google has made the process of using multi-page forms much harder than it used to be. In the older version of Google Forms the values were passed from one page to another in an array.

In the new version of Google Forms the data is still passed from page to page in a variable (called draftResponses) however PHP sees this data as a string instead of an array so the code which reformats the array from a PHP style (which WordPress uses) to a Python style (which Google uses) doesn’t run.

It looks like I will need to parse the string into a PHP array and then encode the array to be compatible with Python. As I said, ugh.