Google Changing Google Forms again?!?

Earlier today I was alerted to the possibility of Google changing Google Forms again.  Based on my reading, I think it is more than a possibility, it looks like a done deal to me.  This is a major change.

That doesn’t look very good.  Not only does it not look very good, it doesn’t work either.  I have no idea when Google will roll this out.  Almost certainly before I can update the plugin.  Based on my quick look at the HTML of the new form, it is very different than what Google has been generating for a number of years.

The biggest change is Google appears to be using their own custom DIV elements instead of real HTML INPUT elements for things like radio buttons and checkboxes.  I can’t think of why this could/would be a good thing but I am sure there is some rationale behind it. These news forms require Javascript from Google to work and it looks like the actual form submission process is also different.

So … it looks like a lot to figure out at a time when I  am really busy at work.  I am hoping this isn’t wide spread for a while as I don’t expect to have much free time until the Christmas holidays arrive and work will slow down a bit.

Chasing down Email-Users out of memory issues

For the past couple of days I have been looking at two sites which were experiencing an issue with Email Users where any of the pages on the Dashboard which presents a list of recipients was incomplete.  Looking closer at the pages, PHP was crashing which resulted in an incomplete page.  One of the sites had another clue, it reported a request for memory which could not be granted.

Both users provided me access where I could upload a debug version of Email-Users.  While not an ideal debug environment, I am grateful for the trust and access both users provided as I would have not been able to chase this bug down in my own development environment.

I was able to narrow the problem down to a call to get_users() which is a standard WordPress API function.  Because Email Users has been around a while, it still contains some code which was necessary in older versions of WordPress and the arguments passed to get_users() included the ‘all_with_meta’ parameter which was the only way to retrieve the first and last name of a user prior to the magic methods which were introduced for get_users() in WordPress 3.x.  The magic methods remove the need for the ‘all_with_meta’ parameter however the plugin was never updated because it wasn’t broken.

In the process of chasing down this memory problem I added some code to partition the get_users() query into blocks of 500 users and I could watch the memory usage increase with each query.  This would continue until memory was exhausted at which time PHP would terminate ungracefully and the partial page would be rendered.

An email to the wp-hackers mailing list helped me understand how much data was being cached by calling get_users() with the ‘all_with_meta’ parameter and I realized I needed to find a different solution as what I was doing wouldn’t scale.

I had encountered the get_users() magic methods previously and I realized that I no longer needed to call get_users() with the ‘all_with_meta’ parameter so I removed it and did some testing and sure enough, I was able to successfully run on both sites without any issues.  Memory usage on the site with 13K users topped out at 47M, well under the 256M maximum defined by WordPress.

In the current beta version (4.6.3-beta-8) there is still some debug code in place to monitor memory usage.  If you look at the page source you will find something like this:

<pre id="line1"><!-- email-users.php::1091  Query #1  Memory Usage:  34.5M -->
<!-- email-users.php::1091  Query #2  Memory Usage:  34.75M -->
<!-- email-users.php::1091  Query #3  Memory Usage:  35.25M -->
<!-- email-users.php::1091  Query #4  Memory Usage:  35.75M -->
<!-- email-users.php::1091  Query #5  Memory Usage:  36.5M -->
<!-- email-users.php::1091  Query #6  Memory Usage:  37M -->
<!-- email-users.php::1091  Query #7  Memory Usage:  37.5M -->
<!-- email-users.php::1091  Query #8  Memory Usage:  37.75M -->
<!-- email-users.php::1091  Query #9  Memory Usage:  38.5M -->
<!-- email-users.php::1091  Query #10  Memory Usage:  39M -->
<!-- email-users.php::1091  Query #11  Memory Usage:  39.5M -->
<!-- email-users.php::1091  Query #12  Memory Usage:  40M -->
<!-- email-users.php::1091  Query #13  Memory Usage:  40.25M -->
<!-- email-users.php::1091  Query #14  Memory Usage:  40.75M -->
<!-- email-users.php::1091  Query #15  Memory Usage:  41.25M -->
<!-- email-users.php::1091  Query #16  Memory Usage:  41.5M -->
<!-- email-users.php::1091  Query #17  Memory Usage:  42.5M -->
<!-- email-users.php::1091  Query #18  Memory Usage:  43M -->
<!-- email-users.php::1091  Query #19  Memory Usage:  43.5M -->
<!-- email-users.php::1091  Query #20  Memory Usage:  44M -->
<!-- email-users.php::1091  Query #21  Memory Usage:  44.25M -->
<!-- email-users.php::1091  Query #22  Memory Usage:  44.75M -->
<!-- email-users.php::1091  Query #23  Memory Usage:  45.25M -->
<!-- email-users.php::1091  Query #24  Memory Usage:  45.75M -->
<!-- email-users.php::1091  Query #25  Memory Usage:  46M -->
<!-- email-users.php::1091  Query #26  Memory Usage:  46.5M -->
<!-- email-users.php::1091  Query #27  Memory Usage:  47M --></pre>

I need to do some additional testing but based on these two sites now working, I am bullish on this solution to this unusual problem.

Time to learn Ruby on Rails?

Last Friday my manager approached me with a problem.  It is interesting problem, the sort of out of left field problems he likes to throw at me from time to time.  It is one of those problems that there isn’t a good solution for without developing something completely proprietary as it would have zero value to anyone else but the members of our team.

So what is the problem?  We have a bunch of text files that have diagnostic information about one of our products.  There is no way to do any analysis on this data because they are just ASCII text files.  Like lots of companies, we do lots of Pivot Tables and Charts in Excel to analyze data but in this case, I don’t think it is the right answer.  At least not initially.  We want to collect these text reports over time and load them all into a single database so the collective data can be analyzed.  We’re looking for trends in reliability and things like that.  Once all of the data is compiled into a database, we’ll likely extract slices of it for analysis using Excel but I don’t want to build up an “Excel database” – wrong solution to this problem.

I’ve been using WordPress, PHP, and MySQL for years so it is my “go to” solution for problems like this one.  I don’t see any reason to involve WordPress as it is really just a data parsing and database application that needs a minimal UI to upload the data and browse it.  I’ve done things like this in PHP previously but I am thinking this might be a good opportunity to learn something new.

learning_railsI had picked up Learning Rails a couple years ago when I considered Rails, for similar reasons, for another project which never went anywhere.  I dusted off mycopy of Learning Rails and for the last few days I’ve been reading through it while I was at the gym.

So far from what I’ve read it looks like a viable solution to quickly put together a solution which (a) needs a database and (b) will be very low usage.  I think the biggest challenge I’ll face putting something together will be parsing the ASCII file in Ruby (which I’ve never used) but I am sure I will be able to figure it out.  The format of the file is fairly regular with sections of data but each section is slightly different.  Unfortunately it isn’t something simple to parse like a CSV file.

This morning I created a new Ubuntu (also a first for me) Virtual Machine and used this Guide from GoRails to install and configure RoR.  In the course of two hours I have built a new VM, installed Ubuntu, and installed RoR.  I am ready to start playing around and see what I can do.

Depending on how this goes, I’ve got a couple other projects in mind that RoR might be a better solution for than building a WordPress plugin.  We’ll see …

Supporting Mixed-Gender Events

It turns out adding support for mixed-gender events isn’t as simple as I thought it might be.  Because so much of wp-SwimTeam revolves around age groups, I haven’t made it very easy on myself to add this enhancement.

Events are tied to age groups so in order to add a mixed gender event, I need a mixed gender age group to associate it with.  A mixed-gender age group doesn’t really make any sense although it is similar to another feature which isn’t easy to support – age groups which are supersets of other age groups.

I have learned that some leagues have “special” events which are open to swimmers from multiple age groups.  For example:  The league my niece and nephew swim in has open freestyle events at the end of the meet.  There are two age groups (10 and under, 11-18) for each gender.  The wp-SwimTeam definition of an age group doesn’t work well for these groups either although it isn’t as problematic as a mixed gender event it.

After noodling around with a few ideas I think I have come to the conclusion that I need some sort of Age Group classification to allow defining a group which spans ages and/or genders.  I am struggling with nomenclature for these groups but I believe the fundamental idea is sound.

For now I am referring to these special age groups as a “Combined” age group.  They will not be counted in swimmer population numbers but will be used for Events and Entries.

phpHtmlLib updated to v2.6.4.3566

I received another report from a user experiencing blank pages on some of the wp-SwimTeam tabs.  I had run into this previously and determined it to be a PHP issue in the early 5.3.x releases (.1 and .2 for sure).  PHP was terminating with a fatal error which resulted in an incomplete HTML page.  If you looked at the page source for these “blank” pages you would see nice HTML simply terminate with no closing tags which obviously resulted in missing content.

It turns out that phpHtmlLib was the culprit (which I suspected) and it was behavior that is inconsistent between releases of PHP including PHP4.

Many of the widgets in phpHtmlLib build upon one another by extending classes.  It is not uncommon to find some classes that are extended 3-4 levels.  No problem, that is one of the beauties of classes.  The problem came from some classes having constructors where some of their descendants did not AND the grand child (or great grand child) class referenced the parent constructor.

Because phpHtmlLib originated in PHP4, all of the syntax remains in PHP4 format including the use of class constructors.  The following example illustrates the problem I had run into:

<?php
/* vim: set expandtab tabstop=4 shiftwidth=4: */

class Sample_A {
    /**
     * Constructor
     */
    function Sample_A() {
        printf('<h3>%s::%s - Sample A Constructor</h3>', basename(__FILE__), __LINE__);
    }
}

class Sample_B extends Sample_A {
    /**
     * No Constructor!
     */
}

class Sample_C extends Sample_B {
    /**
     * Constructor
     */
    function Sample_C() {
        printf('<h4>%s::%s - Sample C Constructor</h4>', basename(__FILE__), __LINE__);
        parent::Sample_B() ;
    }
}


ini_set('display_errors','stdout');
error_reporting(E_STRICT | E_ALL) ;

printf('<h2>%s::%s - Instantiating Sample A</h2>', basename(__FILE__), __LINE__);
$A = new Sample_A() ;
printf('<h2>%s::%s - Instantiating Sample B</h2>', basename(__FILE__), __LINE__);
$B = new Sample_B() ;
printf('<h2>%s::%s - Instantiating Sample C</h2>', basename(__FILE__), __LINE__);
$C = new Sample_C() ;

?>

Running the code above results in the following with certain versions of PHP (e.g. 5.3.1).

<h2>sample.php::33 - Instantiating Sample A</h2>
<h3>sample.php::9 - Sample A Constructor</h3>
<h2>sample.php::35 - Instantiating Sample B</h2>
<h3>sample.php::9 - Sample A Constructor</h3>
<h2>sample.php::37 - Instantiating Sample C</h2>
<h4>sample.php::24 - Sample C Constructor</h4>

Fatal error: Call to undefined method Sample_B::Sample_B() in /var/www/clients/client3/web44/web/sample.php on line 25

Why does this happen? It turns out that earlier (and now later) versions of PHP would handle the missing constructor in Sample_B implicitly. Because some constructors are fairly complex and pass multiple arguments, I don’t really want to add all of the intermediate constructors just because some versions of PHP don’t handle the constructor chain properly (or at least consistently). So I came up with the following solution:

class Sample_B extends Sample_A {
    /**
     * Constructor - needed for early PHP 5.3.x compatibility
     *
     * @param Parent Constructor
     * @param array of Constructor Arguments
     */
    function Sample_B() {
	    call_user_func_array('parent::Sample_A', func_get_args()) ;
    }
}

Adding this constructor to the Sample_B class results in the following (which is what I want):

<h2>sample.php::39 - Instantiating Sample A</h2>
<h3>sample.php::9 - Sample A Constructor</h3>
<h2>sample.php::41 - Instantiating Sample B</h2>
<h3>sample.php::9 - Sample A Constructor</h3>
<h2>sample.php::43 - Instantiating Sample C</h2>
<h4>sample.php::30 - Sample C Constructor</h4>
<h3>sample.php::9 - Sample A Constructor</h3>

There is a chance there are more places within phpHtmlLib that could have this problem with some versions of PHP5.3.x. Now that I know how to solve them, fixing them is pretty simple and very quick. If you run into any blank pages within wp-SwimTeam, let me know ASAP and more than likely a solution like the above within phpHtmlLib will address the issue.

The phpHtmlLib plugin has been updated to v2.6.4.3566 and committed to the WordPress plugin repository.

More on Event Groups

As I continue to work away on Event Groups and Event Management overall, I have realized that what I had implemented earlier was really poorly thought out.  The way I had defined events and assigned them to a swim meet really isn’t very useful.  The Event model will change quite a bit as I flush this out.  It is possible that it will be necessary to delete any existing events and redefine them using the new model.  I haven’t completely worked it out yet but that is likely where I am headed.  It makes the code too ugly to try and deal with old data that really isn’t used for anything in the new model.

I hope to have a release soon which has the base functionality done since I’ve also fixed a few other odds and ends while I’ve been in the code.

Unintended Consequences …

Last night while I was at my daughter’s basketball practice I finished up modifying the Events functionality to account for the new events groups which will allow events to be bundled into logical groups.  As I was testing everything and things seemed good to go, I realized that the Drag-and-Drop event re-ordering won’t work correctly.

The DnD re-ordering assumes there is only one set of events so as the events are rearranged they are renumbered from 1-N.  This isn’t a show stopper but I will have to come up with a new solution.  In the process of playing around with some ideas I have determined that the jQuery plugin I was using has been updated to support touch so I have folded that in and everything which was working still works.

I had hoped to get a new release out this week but that is unlikely to happen at this point.  I am not going to release anything until I have the event reordering fixed unless I get a serious bug report in which case I’ll simply disable Drag-and-Drop reordering until I get it working again.

Event Groups

I’ve started on the Event Groups idea I posted about last week. The more I think about this, the more I like it.  In my case, I have three distinct sets of events I have to worry about:  Short Course Yards, Short Course Meters, and our local Invitational which is also Short Course Yards but is a different set of events from what our league normally swims.

By “grouping” a set of events with a label (e.g. Short Course Yards), it will be trivial to assign a set of events to a swim meet.  I have decided to hold off on events import until Events Groups are done so a set of events can be assigned to a group upon import.

Re-Thinking Events

Nothing like a long flight to provide an opportunity to really look into wp-SwimTeam!  After fixing a couple more bugs, I started looking at Events as they are the foundation of meet entries.

A couple years ago I added Events to wp-SwimTeam but they don’t really do much.  It was the start of some new functionality that I never had the time to finish as fixing or enhancing other aspects of the plugin were always more important.

This season I really want to add the ability to generate a Meet Entries file directly from wp-SwimTeam.  In order to do that, the first order of business is to make sure the events associated with a swim meet can be managed.

Which brings me back to Events.  In looking at Events, the functionality I originally built is ok but has what I see as one pretty serious gap.  I allow the creation of a set of “Standard Events” –events which you can consider the base set of events that your team will swim most often, probably your home pool events.  This is ok because they can easily be added to an individual swim meet.

But what happens if you have a pool, like our home pool, which is measured in Meters and all of your opponents swim in Yards pools?  Even if the events are the same, you’d probably like to have a standard set of Meters Events and a standard set of Yards Events.  Now if I am going to manage two groups of Events, I might as well manage N groups of Events right?

I am toying with the idea of adding another field to the events table (Event Group) which would allow events to be assigned to a group.  Groups could be optionally defined (default to None).  Once Events are in a Group a Group of Events could easily be assigned to a swim meet.  Additional refinement at the swim meet level would still be supported.

I also plan to support the ability to import a Hy-tek Event File (.hyv) to either a specific meet or as a set of standard events.

I’ve got a couple more hours of flight time to noodle on this, we’ll see where it goes.

Gearing up for Swim Team 2012

It will be summer swim team season before I know it so it is time to start gearing up for the upcoming season.  I haven’t really looked at wp-SwimTeam since last fall and WordPress has had several updates since then. I found out today that one of the updates causes the “real name” for the users not to be returned so that is first on my list of things to fix.  There are also some GUI inconsistencies that I’d like to fix.

This is my short list of items which I want to implement this season:

  1. New/update Swim Team theme.  This really isn’t related to wp-SwimTeam per se but our site is looking a little dated and it is time to freshen it up.
  2. Results Import:  I said I was going to do this last season and I never finished it.  I really want to get this done this year.
  3. Export of Meet Entries:  The wp-SwimTeam plugin has all of the informtation (roster, scratch list, event list, etc.) to generate a Meet Entries file in SDIF format.  Providing this file will greatly simplify getting a team’s entries into either Hy-tek or WinSwim (or any other tool which imports meet entries).  This will likely be first on my list after fixing the name bug and the GUI inconsistencies.
  4. Document all of the short codes on the wp-SwimTeam demo site.  I really need to do this.  It would make it much easier for new people to pick up the plugin and do something useful with it quickly.