Swim Team v1.42 is finally out

Yesterday I finally released Swim Team v1.42.  With almost 50 downloads of the beta and no problems reported, I figured it was as tested as it was going to get!  Some of the fixes and enhancements include:

  • Added checking and error messages for creation of temporary files used during export (CSV, HY3, SD3, RE1).
  • Added support for Event Number suffixes.
  • Added support for Transients as temporary storage when exporting data or generating reports.
  • Major change to how files are included to better support newer versions of PHP. The PHP include_path is no longer modified nor assumed.
  • Fixed bug in MyJobs which appears when no active season is designated.

Functionally this release is identical to v1.42-beta-8.  You can find the update in the WordPress plugin repository or as an update on your Dashboard.

Swim Team v1.42-beta-6 available

This morning I have posted a new build, v1.42 beta-6 of the Swim Team plugin.  This beta release makes a major change to how the plugin includes the various files the plugin uses.  Historically, Swim Team modified the PHP include_path as part of the initialization to include the paths where included files are expected.  In newer versions of PHP this seems to cause some problems for some users.

I have changed the plugin to eliminate the need to modify the PHP include_path however the process of doing so required touching almost 100 files within the plugin.  I have been through all of the menus and tabs and believe I found accounted for every file which included but there is always a chance I have missed something.

Please report any problems and I will do my best to fix it quickly.

wp-SwimTeam Beta (2258 downloads )

Google Forms v0.66 available

This evening I released v0.66 of Google Forms.  This version fixes two major issues:

  1. The conflict with WordPress SEO has been resolved.  The implemented solution will likely be replaced with something which runs less frequently.  However, the solution implemented in v0.66 should make sites running both Google Forms and WordPress SEO a little happier.
  2. Due to a bug, all form submissions have been logged regardless of the plugin setting.

You can find the update on the WordPress plugin repository or on your Dashboard.

Google Forms v0.64-beta-3 available

This evening I uploaded v0.64-beta-3 of Google Forms.  This version adds a check for the WordPress HTTP API cURL transport and issue a notification if it isn’t present.

There was a substantial change to the WordPress HTTP API between 3.6.1 and 3.7.  I don’t know the full details of the change but what I found in pretty extensive testing is the streams and fsockopen transports work in 3.6.1 with the Google Forms plugin but from 3.7 and later, they do not.

However, the cURL transport does work and as long as it is available, the plugin will work as it always has.  The problem is I cannot find any way to work with sites that don’t have the cURL transport available so I have added this check and issue a notice when it isn’t present.

GForm_SS_70

 

Please download and test out this beta version and report any issues found.

Google Forms Beta (7978 downloads )

Google Forms, Checkboxes and PHP 5.4 or 5.5

WordPress Google Form v0.63 introduced a change  which allowed the plugin to work on newer versions of PHP, notably 5.4 and 5.5.  Unfortunately that change has broken support for checkboxes.

For those who are new to the plugin or never needed to know how it worked, what the plugin does is retrieve the HTML for the form from Google and render it within the context of WordPress.  When the form is submitted, it is actually submitted within the context of WordPress.   The data is collected by the plugin and then submitted to Google.  The retrieval from and submission to Google is done with the WordPress HTTP API.  In particular, the wp_remote_get() and wp_remote_post() functions are used to retrieve and submit the form.

To complicate the problem further, Google uses Python as the backend for their form processor where as WordPress uses PHP.  For the most part, the fact that they are based on different scripting languages isn’t a big deal.  Until you get to checkboxes.  Checkboxes in Python are handled differently than they are in PHP.

I had solved the compatibility problem a couple years ago (see this thread on the wp-hackers mailing list) using a small jQuery script which fixed the form variables on the WordPress side and manual construction of the body parameter for wp_remote_post() when submitting the data to Google.  This solution worked fine until I received a bug report that nothing was being submitted on a site which was running PHP 5.4.x.

Fortunately the user who encountered the problem provided me with a patch that I was able to fold in which changed the way the body parameter was constructed (array instead of a string) which worked with PHP 5.4.x and also worked with older versions.  However, I didn’t test it thoroughly as I have had several reports that checkbox content was not being submitted correctly.  Uh-oh.  I was able to verify the problem fairly quickly and was able to push out a version which essentially reverts how the body parameter is constructed (string instead of an array).  The problem is, this build doesn’t work with newer versions of PHP.

I have PHP 5.5 running on an Ubuntu virtual machines for testing and so far, I have not found a solution which (a) works with PHP 5.5. and (b) submits checkboxes correctly.

Stay tuned.

Chasing down Email-Users out of memory issues

For the past couple of days I have been looking at two sites which were experiencing an issue with Email Users where any of the pages on the Dashboard which presents a list of recipients was incomplete.  Looking closer at the pages, PHP was crashing which resulted in an incomplete page.  One of the sites had another clue, it reported a request for memory which could not be granted.

Both users provided me access where I could upload a debug version of Email-Users.  While not an ideal debug environment, I am grateful for the trust and access both users provided as I would have not been able to chase this bug down in my own development environment.

I was able to narrow the problem down to a call to get_users() which is a standard WordPress API function.  Because Email Users has been around a while, it still contains some code which was necessary in older versions of WordPress and the arguments passed to get_users() included the ‘all_with_meta’ parameter which was the only way to retrieve the first and last name of a user prior to the magic methods which were introduced for get_users() in WordPress 3.x.  The magic methods remove the need for the ‘all_with_meta’ parameter however the plugin was never updated because it wasn’t broken.

In the process of chasing down this memory problem I added some code to partition the get_users() query into blocks of 500 users and I could watch the memory usage increase with each query.  This would continue until memory was exhausted at which time PHP would terminate ungracefully and the partial page would be rendered.

An email to the wp-hackers mailing list helped me understand how much data was being cached by calling get_users() with the ‘all_with_meta’ parameter and I realized I needed to find a different solution as what I was doing wouldn’t scale.

I had encountered the get_users() magic methods previously and I realized that I no longer needed to call get_users() with the ‘all_with_meta’ parameter so I removed it and did some testing and sure enough, I was able to successfully run on both sites without any issues.  Memory usage on the site with 13K users topped out at 47M, well under the 256M maximum defined by WordPress.

In the current beta version (4.6.3-beta-8) there is still some debug code in place to monitor memory usage.  If you look at the page source you will find something like this:

<pre id="line1"><!-- email-users.php::1091  Query #1  Memory Usage:  34.5M -->
<!-- email-users.php::1091  Query #2  Memory Usage:  34.75M -->
<!-- email-users.php::1091  Query #3  Memory Usage:  35.25M -->
<!-- email-users.php::1091  Query #4  Memory Usage:  35.75M -->
<!-- email-users.php::1091  Query #5  Memory Usage:  36.5M -->
<!-- email-users.php::1091  Query #6  Memory Usage:  37M -->
<!-- email-users.php::1091  Query #7  Memory Usage:  37.5M -->
<!-- email-users.php::1091  Query #8  Memory Usage:  37.75M -->
<!-- email-users.php::1091  Query #9  Memory Usage:  38.5M -->
<!-- email-users.php::1091  Query #10  Memory Usage:  39M -->
<!-- email-users.php::1091  Query #11  Memory Usage:  39.5M -->
<!-- email-users.php::1091  Query #12  Memory Usage:  40M -->
<!-- email-users.php::1091  Query #13  Memory Usage:  40.25M -->
<!-- email-users.php::1091  Query #14  Memory Usage:  40.75M -->
<!-- email-users.php::1091  Query #15  Memory Usage:  41.25M -->
<!-- email-users.php::1091  Query #16  Memory Usage:  41.5M -->
<!-- email-users.php::1091  Query #17  Memory Usage:  42.5M -->
<!-- email-users.php::1091  Query #18  Memory Usage:  43M -->
<!-- email-users.php::1091  Query #19  Memory Usage:  43.5M -->
<!-- email-users.php::1091  Query #20  Memory Usage:  44M -->
<!-- email-users.php::1091  Query #21  Memory Usage:  44.25M -->
<!-- email-users.php::1091  Query #22  Memory Usage:  44.75M -->
<!-- email-users.php::1091  Query #23  Memory Usage:  45.25M -->
<!-- email-users.php::1091  Query #24  Memory Usage:  45.75M -->
<!-- email-users.php::1091  Query #25  Memory Usage:  46M -->
<!-- email-users.php::1091  Query #26  Memory Usage:  46.5M -->
<!-- email-users.php::1091  Query #27  Memory Usage:  47M --></pre>

I need to do some additional testing but based on these two sites now working, I am bullish on this solution to this unusual problem.

Where did all of this wp-SwimTeam stuff come from?

You may have noticed an influx of content (posts, pages, etc.) related to wp-SwimTeam and wonder where it came from.  For the past four years I have been developing, enhancing, supporting, etc. a WordPress plugin called wp-SwimTeam.  This plugin can be used to manage the registration, volunteers, participation, and other aspects of running a youth swim team.  Several groups have used it for Masters Swimming as well but it is targeted at youth swim teams.

I first started wp-SwimTeam to support our local neighborhood team when my wife was the swim team chair.  Over the years it grew in features to the point where it was fairly comprehensive.  My children no longer swim and my involvement in swim team has largely ended but I still maintain my plugin because (a) people use it and (b) I still feel it serves a need and it doesn’t take much of my time to continue supporting it.

I have always had a separate web site for wp-SwimTeam but over time that came to make less and less sense. In fact, I had some people contact me on this site and some on the other site so I had questions and solutions on both sites.  I decided that I would migrate everything here So I exported all of the content and then imported it.  There were a few hiccups, mostly around downloads but I think everything else came over ok.

The old URL redirects to the wp-SwimTeam page on  this site.  From time to time you may see updates for wp-SwimTeam or phpHtmlLib in addition to the normal flow of information on Email Users and WordPress Google Form.

Email Users v4.6.0-beta-3 now available

If you’ve been following the Email Users Support Forum, you are aware that for the past couple of days I have been chasing a bug which affects some users and places all of their intended recipient email addresses in the CC header instead of the BCC header.

This has been a challenging bug to find as it doesn’t happen on any of the systems I have access to.  Fortunately I had a few users affected by this problem willing to help me and I appreciate their assistance.  I am not sure how I would resolved this problem without their assistance.  Again, thanks.

So what was the problem?  The WordPress wp_mail() API allows you to pass extra header information as part of the mail content.  Email Users makes use of this by constructing a BCC header which contains all of the email addresses for the intended recipients.  The wp_mail() function supports headers as a single string or an array of headers, each header as an array element.

Up until this evening Email Users had constructed all email headers as a string.  With a large number of email addresses this string could end up being very long.  However, I don’t believe the length of the string had anything to do with the problem.

When wp_mail() receives headers as a string, it tries to intelligently decompose them.  In some cases it appears that the string wasn’t splitting correctly which means the BCC field wasn’t ever seen.  The “Bcc:” text became part of an email address which was subsequently discarded because it resulted in a bad address.

Email Users v4.6.0 has a rewrite of the mailusers_send_mail() function plus some new debug functionality which helps resolve these sorts of issues.  Additionally, the new debug is under user control so can be turned on and off as needed.

Email Users Beta (5003 downloads )

EU_SS_29

For testing this new version, I recommend turning Debug on until you’re comfortable with the the structure of the email addresses.

Questions or comments -don’t hesitate to ask.

Debugging the Email Users CC issue

For the past few days I have been chasing a reported issue where Email Users is putting recipient addresses into the CC header instead of the BCC header.  This has been a challenge to chase down.  To assist in this I have add a bunch of  new debug and checking to Email Users.

Email  Users will now check and report if wp_mail() has been overloaded by a plugin or theme.  Since wp_mail() is a pluggable function, this is the one area which is really out of the control of Email Users.  Simply knowing if wp_mail() has been overloaded is very helpful.

Additionally, I have finally found a way to initiate a complete email which is aborted just prior to the email actually being sent.  The content of several internal structures is displayed on the Dashboard so the email headers can easily be examined.

This build is in DEBUG mode which means it will never actually send an email unless wp_mail() has been overloaded in which case all bets are off.  Please visit the Settings page first to determine if wp_mail() has been overloaded.  If it has, do not send any email but let me know which plugin or theme has overloaded wp_mail().

Email Users Beta (5003 downloads )

A Really Odd Email Users Problem

I was recently contacted about an Email Users issue on a site that is using bbPress and S2Member.  The person had come to the conclusion that there was some sort of conflict between the three plugins which resulted in the Email Users -> Users Settings screen being empty and asked me to look into it.

EU_SS_20

Clearly something is amiss as in order to access the WordPress Dashboard you need to login and in order to login, there needs to be at least one user!  This isn’t the sort of situation I can easily replicate.  Fortunately the person with this problem provided me access to their server so we could do some debugging.

The functionality in Email Users is based on the WP_List_Table class and an example plugin which you can find in the WordPress Plugin Repository.  When I first implemented it I was retrieving the list of users with a call to get_users() however I never got the sorting working correctly.  After playing around with it for a while, I gave up and wrote my own query.  The query I wrote yielded the results and sorting I wanted so I didn’t think much about it.  Until today.

This the query that WordPress generates and sends to MySQL:


SELECT ID, display_name, user_email, user_login, m1.meta_value first_name, m2.meta_value last_name, m3.meta_value massemail, m4.meta_value notifications
FROM wp_users u
LEFT JOIN wp_usermeta m1 ON (m1.user_id = u.ID AND m1.meta_key = 'first_name')
LEFT JOIN wp_usermeta m2 ON (m2.user_id = u.ID AND m2.meta_key = 'last_name')
LEFT JOIN wp_usermeta m3 ON (m3.user_id = u.ID AND m3.meta_key = 'email_users_accept_mass_emails')
LEFT JOIN wp_usermeta m4 ON (m4.user_id = u.ID AND m4.meta_key = 'email_users_accept_notifications')
ORDER BY last_name ASC LIMIT 0,100

What this query does is select all of the WordPress User Ids, first names, and last names, which have the proper Email Users user meta keys. The data is returned and stored in a format which can be rendered in the List Table.

Because the table was empty, I wanted to see what the results of the query were as that would tell me where the issue might be.  Because I had access to the MySQL database via phpMyAdmin, I was able to run the query directly against MySQL.  When I did so, this is what I found:

EU_SS_21

This certainly was unexpected but it explained why the List Table was empty.  I have never run into this sort of problem, in fact, I had never heard of either the SQL_MAX_JOIN_SIZE or the SQL_BIG_SELECTS MySQL system variables.

I was able to alter SQL_BIG_SELECTS using the following SQL:


SET SQL_BIG_SELECTS=1

When I did this and re-ran the query, phpMyAdmin returned the results I would have expected to see.  This was good news – it appears this users problem is due to a MySQL setting.  The bad news is sometimes it is hard to change these settings and it isn’t something I thought I could add to the query itself.

As a test, I modified the source for one of Email Users files so it was calling the original method I had coded  based on get_users() to see if it would work and it did.  This is very good news as I suspect fixing the sorting issue (or living with it) will be easier than changing the MySQL system variable settings.

My next step is to see if I can fix the sorting problem and roll out an update.