The Campus Map application has been updated with all of the new wireless access points added over the last several months. This is a long overdue update on my part, but I’m happy to say that I’m now receiving a list of spots to add as they’re added from Petar Mitrevski, a Computing Specialist with the Helpdesk staff. The excellent work done by Facilities Services and Systems and Network Infrastructure to get these new wireless locations up and running should also not escape notice. Here’s a list of the additions:

• Allen Hall
• Atwater Residence Halls
• Battell Hall
• 26 Blinn Lane
• Cousteau
• Chellis House
• 242 College Street
• DKE Alumni House
• Earhart
• Forest Hall
• Hadley House/Barn
• Hepburn Hall
• Johnson Memorial Building
• McKinley House
• Mead Memorial Chapel
• Nelson Recreation Center
• Norgay
• Pearsons Hall
• Peary
• Ride
• Service Building
• 107 Shannon Street
• 56 South Street
• Stewart Hall
• Wright Memorial Theater

I haven’t had an update in a while because I’ve been working on two very large projects that aren’t quite ready for launch. I’ll have more information about the Athletics Recruiting form application, a form that will look like the Middlebury College web site but collect information about potential Athletes directly into Banner, as it continues to move toward launch stage. This post will touch on an aspect of that project. The second project is a secret for now, but there should be plenty of interesting content about it beginning in March. To give a tiny hint: it will be the first site using MCMS other than the Middlebury College web site. As you might imagine, this created some interesting conflicts and challenges.

But this post is about filtering content and information contained in one data source and preparing it for insertion into another. Specifically, I’ve been trying to move historical Athletics recruit records from a FileMaker Pro database into Banner. The FMP database is fairly unstructured, not so much caring whether you enter “Lit”, “English Lit”, or “English Literature” in the field marked “SAT II Subject Test Name”. Banner is significantly less care free about this type of data integrity.

I’ve created a program that contains several custom data structures to hold the values for the data import. Using “structs” is much easier than the way I used to do these types of scripted data transfers: by using untyped arrays to hold the information. To give you a concrete example, consider the following two code snippets:

string[] record = new string[4];
record[0] =  “101010″;
record[1] = “Ian”;
record[2] = “McBride”;
record[3] = “S”;

Or….

public struct Record {
public string ID;
public string SRTIDEN_FIRST_NAME;
public string SRTIDEN_LAST_NAME;
public string SRTIDEN_MI;
}

Record record = new Record();
record.ID = “101010″;
record.SRTIDEN_FIRST_NAME = “Ian”;
record.SRTIDEN_LAST_NAME = “McBride”;
record.SRTIDEN_MI = “S”;

Now the second example definitely contains more lines of actual code, but it’s also much less ambiguous. Consider too what happens when, rather than 4 data fields, we really have 50-100. “Now was birth month the fifteenth array index or the seventeeth?” versus, “OK, just type ‘record.SRTPERS_BIRTH_MON’”. Being able to name each data element the same as the database column in which you’ll insert the value is also tremendously helpful when you’re paging through a couple thousand lines of code.

Additionally, rather than paging through a switch statement to handle all of the exception cases for transferring one data encoding to another, I decided to just use a Hashtable and access it dynamically to translate between the two data sets. Here’s some more examples, starting with the old way of using a switch statement:

switch(fmp_line["coach"]) {
case “DAN”:
record.COACH = “11111111″;
break;
case “JME”:
record.COACH = “22222222″;
break;
case “RDR”:
record.COACH = “33333333″;
break;
case “SCM”:
record.COACH = “44444444″;
break;
default:
record.COACH = “55555555″;
break;
}

And now using a Hashtable:

htCoaches = new Hashtable();
htCoaches["DAN"] = “11111111″;
htCoaches["JME"] = “22222222″;
htCoaches["RDR"] = “33333333″;
htCoaches["SCM"] = “44444444″;
htCoaches[""] = “55555555″;

record.SYRRLST_RECRUITER_PIDM = (string)htCoaches[cols[(int)htCols["coach_recruiting"]]];

To further explain, the Hashtable htCols is dynamically populated by parsing the first line of the comma separated data file that I get as an extract from FileMaker Pro. The value it returns is the index position of that column in the file, meaning that if the column position changes I don’t have to update my script. This is useful if it’s decided that the CSV needs to contain more or fewer columns. This index is passed to the string array “cols”, which contains one of the column values from each line of the CSV. The value from the CSV is matched against the value in the htCoaches Hashtable, which returns a PIDM to insert into the Banner database.

Another interesting problem that I ran into is that a comma separated file can and will also contain commas that are not to be used as separators. When you export a CSV from Excel, it will enclose any fields that contain commas in their values in quotes so that Middlebury, VT becomes “Middlebury, VT”. Unfortunately, the way that I was parsing each line of the CSV was just to split it into chunks every time there was a comma. This meant that “Middlebury, VT” became “Middlebury [break for next column] VT”. This would typically be the place where you’d write a complicated regular expression to further break down the text and make sure that you weren’t grabbing “Middlebury, VT” as two separate columns, but I found a better solution.

// feed the current line of the file into a string variable
string line = stream_reader.ReadLine();
string[] pieces = line.Split(”\”", StringSplitOptions.None);
string result = “”;

// go to EVERY OTHER piece in the string and look for commas
for (int i = 1; i < pieces.Length; i = i + 2) {
pieces[i] = pieces[i].Replace(’,', ‘ ‘);
}

// put it back together
for (int i = 0; i < pieces.Length; i++) {
result += pieces[i];
}

line = result;

Note that we need to go to every other instance because the first index will be the line up to the first quote mark which will likely contain commas that are supposed to break the line into columns. Then the space between the first quote mark and the second quote mark will always contain only commas that were in the original data and so on. I chose to replace these commas with just a blank space character, but it could easily be modified to contain a string you’d never find in the source content like “<here_is_a_comma>”, which gets translated back into a comma during post processing.

One more thing before I sign off for the day. I noticed that if you had a spreadsheet with a lot of blank cells and tried to save it as a CSV you wouldn’t always get a comma for each column, meaning that my script would think that there were several fewer columns than there really should have been in a line. I found this helpful tip for making sure that you got a properly formatted CSV:

1.  Select one of the cells in your header row.
2. Press Shift + Ctrl + 8.
3. Press Ctrl + H to get the Replace dialog box
4. Leave the “Find What” section empy.
5. In the Replace With box enter a single space character
6. Click Replace All.

Then save your Excel file as a CSV and you’re all set for the most fun thing in the world: parsing a CSV with regular expressions and data structures!

One of the things you’ll doubtless notice about the current design of Middlebury College’s website is that it’s always the same width, about 780 pixels. This design decision was made in 2002-2003 when many people were browsing the web with monitors configured to 800 x 600 pixel resolutions. While it gives our site a very professional and contained look-and-feel, it does make several tasks difficult.

The biggest of these issue is what to do about tabular data. In the screenshot above you can see a search interface for a small database of undergraduate research opportunities. The results of this form will be displayed in a table with sortable columns. Around 15 sortable columns. That just won’t fit in the roughly 450 pixels we have to work with in the official site design.

So we have a secondary design (pictured above), which has the approved logo, the title of the page and a blue bar, which is the approved shade of blue, or as close we can get to it and still have it look the same across all browsers. Below the dotted line we’ve been given semi-free reign over what can be done in the design. Chief among our restrictions is that it shouldn’t look much like the official site design.

This CMS “rendering script” uses the same header, left navigation, and right navigation controls as the rest of the CMS, meaning that we can modularly update the way those scripts work and have it work across both site designs (as well as another new site design that’s coming up). It took a little bit of mucking with our current scripts for me to pull this off because they were designed with only having a single site to support. For instance, the header pulled in two CSS files that I didn’t want showing up on the new design. We now have a special ASP.NET User Control called “Head”, which has a placeholder for adding new CSS files and some code for handling RSS feed, meta tags, and search robots. There are then two other user controls called “HeadTags” which both inherit “Head” and override a method that lets them add in custom CSS files, depending on what part of the CMS we’re in. Athletics, Language Schools, Breadloaf, etc. all have the same basic CSS files, but each add a couple extra to complement the site design.

Similarly, I created instances of the left and right navigation columns that only include the links to content, leaving out the extra images that frame the columns on the official design and some of the extra functionality included in those columns, like the links to Gateways. These will again be useful when we implement a new site design for another area of the College that isn’t going to have these elements. Or, at least, it won’t have them in quite the same way.

I’ll have more on this search page: what it is for, what it does, and a little on how it works in my next post which will be after I get the search functionality working.

I’ve decided to set this up as my new blog, removing content from the old blog which was hosted on our local development copy of Microsoft Office SharePoint Server 2007. Nothing against that software, but it’s easiest for me to maintain this content offsite, since I don’t have to log in through the VPN to do my editing. Also, with upgrades and other software moves, I didn’t want to keep re-creating my blog when we moved the development server around.

I’ve imported some of the more public-friendly posts from the old blog, which were mostly updates on our development progress with various systems. There was some other old content having to do with visits to conferences, but the content skirted the line on what I’d like to have on a public site, which is where I’d like to focus the efforts of this blog from now on.

I want to get back in the swing of actually posting on this thing and fortunately, I have a reason to do so. There are two major changes to the CMS that will go in place, hopefully early next week. These are to accomodate plans for an Emergency Response web site, but are also things that we’ve wanted to do on the CMS for some time. Now that we actually have a reason in the form of a project, it’s easier to sit down and write them out.

Last Modified By

The first change will affect almost every page on the CMS. Anywhere that’s publicly viewable and has editable content, except for the front page, will have a tagline at the bottom that says “Last Modified by Person on Date“. The person and date information are drawn automatically from the metadata that the CMS keeps about page revisions.

This feature will let users know who to contact if there’s content on the page that’s inaccurate, needs to be updated, or that they have more questions about. Prior to this update, you had to notice that there was a link named “Contact Us” on the bottom left of most pages, click on it and try to decipher who to direct your request to. Now you’ll know exactly who is responsible for maintaining the information on the page, making it easier to track them down.

There are a few exceptions, of course. Many pages are edited by LIS staff during support calls about the CMS. Others are automatically re-approved by a nightly batch script (HR Job Descriptions fall into this category). These will always appear to be edited by the “Middlebury College Web Master”, rather than by me or the backup script user account. Also, much of the content on the CMS is actually edited by students and then approved for publication by a staff member. When you’re viewing this content in published mode (always unless you’re a CMS editor) you’ll see the staff member’s name. If you’re trying to edit the content, you’ll see the person who last edited it, which might be that student worker.

It’s my hope that this change will raise awareness on campus and off as to how many people manage content on our website. It’s well over 200 active accounts spread out throughout the CMS. Many of those, like the Academic Coordinators, manage the entire site for their department, including the profiles of the Faculty. It takes considerable work to keep all of the content (somewhere on the order of 30,000 postings with even more documents and pictures) up to date.

Contacts Lists

Even though we’ve updated how “Last Modified” information is posted on the CMS, there’s still a need for up-to-date lists of contact personnel. In the past, we’ve used these exclusively for Contact Us sections on the CMS to identify persons responsible for content. Well, the LIS staff directory also. For that purpose, all that was required was the person’s name, email address, office telephone, and office location. However, with the prospect of an Emergency Response site, more information is required. Everything from home phone numbers (which we have), to cell phones (which we might have), to radio numbers for Facilities staff (which we don’t have).

Information like this begins its life in Banner, updated by HR for Faculty and Staff accounts or by the Dean of Student Affairs office for students. There are also all kinds of other accounts: Emeriti professors, High School students, contractors, some Alumni, Spouse/Partner accounts, and now the account information for MIIS, which is stored separate of Middlebury records. Some of this information is then moved into the Active Directory by scripts that run at least nightly. The Active Directory is like a phone book for computers and contains all of the Windows and Exchange user accounts. If you have an email account at Middlebury, you’re in the AD.

These contacts lists have been modified so that they can display any number of attributes from the Active Directory. By default, they still show the name, email, office location, and office phone number. There are several attributes, like ID Number that we don’t want all people with access to edit Contact postings to be able to add to the list. These are reserved for a select set of CMS administrators. Additionally, the Contacts posting will maintain the display permissions you set up in the Directory’s Change Information screen. You can go there to set it up so that your email address doesn’t appear to people off campus. There is an override option available to administrators to force the Contacts posting to show the information, regardless of what the user specified. This is a necessary addition for the Emergency Response site. We need to be able to see your Cell Phone number to call in the event of an emergency, even if it’s not shown to folks outside of that site.

Coming Changes for the CMS

That nearly wraps up changes we will be making to the current code base for the CMS, which is running on Microsoft Content Management Server 2002. That product has been sunsetted and we’re examining replacements with the idea of being fully off MCMS by Summer 2009.

The final major revision to the MCMS code prior to then will be the migration of the current Events at Middlebury calendar from custom PHP. This will allow us to have a more stable Events calendar and also have site-specific events listings in the navigation controls. Imagine going to the Math Department’s site and seeing their Seminars listed automatically from the Events Calendar. That’s the general idea here as well.

I think I’m going to move this to a once-a-week posting thing in theory since that’s what it’s already become in reality. Sunday seems like a good time for this too, because I should have all of the code updates for Monday’s push done by now and with nobody else here, I can usually finish up all of my tasks for the day and still have a bit of breathing room at the end, as happened today!

Athletics Recruiting

I’ve finished the Data Analysis, which should really have been completed before any work on the project was started. There are about 700 columns touched by the Web Front End, excluding when the form repeats like when the user enters more than one sport. A single user could bind to more than 1000 Banner columns in the course of filling this thing out. It will be interesting to see if any actually do.

The big trick with this will still be making sure that the default values we use for type codes, such as the home phone number of the recruit, don’t conflict with anything else Admissions is doing. We’ll also have to do some work with the initial population of tables like the questionnaire questions, and person and relative codes that don’t exist yet.

We’ve also made the decision to do this thing in C# rather than PHP, so that we can use the built-in data handling functions to bind to the database, rather than trying to muck with a custom PHP data abstraction layer. I think the savings we’ll see in being able to use built-in form field object, data binding, and native validation exceptions will be well worth abandoning the work that was initially done on the PHP form, much of which would need to be redone at this point anyway.

Athletics Rosters for the CMS

Progress on this has been going faster than I initially thought it would. I’ve completed the code to add in both normal and captain’s profile text to the roster XML. This profile text will be formatted HTML in a CDATA block being generated using the RadEditor control. I had to work with Brett to get the binaries for this in the right folder on the server and we haven’t used the regular RadEditor yet, just the one that binds to CMS placeholders. Because the roster is one big placeholder, we need to use the basic RadEditor so that we can grab the HTML on postback and insert that into an XML element in the big roster placeholder.

I’ve also completed a couple custom data placeholders, including a checkbox to denote whether the student is a captain or not. The checkbox was really tough to do because ASP.NET will return a non-boolean null value from an empty XML field of type “boolean”, which can’t be converted to boolean in the front-end page. This issue was difficult to resolve because all the online examples are of binding to a database backend, rather than XML, and use SQL statements to fetch the data, rather than XPATH or other XML data lookup functions. I finally resolved it by declaring a default value for the field and manually setting the returned data type in the code behind.

I also changed the user interface in a way that I think will make it easier to use. Initially we wanted to have a textbox in which Brad would type the name of an image, then click a button and the CMS would see if the image was in the Resource Gallery. I was having trouble getting an event handler on a button inside a data grid working, so I changed this to just be a drop down box of the elements of the current resource gallery, which is easier for Brad to use anyway. I just need to add some code to handle the event where he returns to edit a record where the image has moved to a new Resource Gallery.

The bulk of the remaining work on this project is creating a rendering script for the captains channels that honors two potential data sources: captains postings from legacy sites and rosters from new sports year channels. With another solid week of programming I could probably have the whole thing done. Of course, there’s no such thing as a solid week of programming

Spam Filtering for Form Email

I mentioned off hand to Brett that we’d had a couple users who were getting spammed by email from a web form submission. He told me that the emails from web forms don’t go through the spam filter because they are treated as coming from “on campus”. He suggested that we send emails to shark.middlebury.edu rather than the default smtp address, to force them through the spam filter.

I made this change on plasma and tested it out. It works better than we thought it would because now email addresses are checked for validity when the form is submitted to the server. A non-valid email address will throw a PHP exception, which I can catch and handle. Now we can assure our customers that any emails addresses entered are real, plus give them spam protection. This will also answer a lot of “why didn’t I get a confirmation email?” questions from people who enter their phone number in the email box. Yes, that has happened.

PHP 6 and the Future of PHP

I’ve spent a bit of time this week reading through the PHP Internals Mailing List, which is the one where the core developers for PHP decide on the direction and new features that are going to be available in future releases. There have been two rather large discussions about PHP recently.

First, one of the lead developers randomly decided to create an extension for PHP 6 which will enable “namespaces”. You’ll be able to declare a namespace at the top of any PHP file and then any classes within the file will be considered as part of that namespace. This means that you can have two Exception classes loaded globally if they are logically referenced as part of two different namespaces. The advantages there are obvious to anyone who’s used a language with real namespaces or even something like Python’s packages. It’s looking like this extension will be part of PHP 6’s core, which will be a large functionality improvement for PHP, a language who’s core function library consists of hundreds of functions, all declared globally. Maybe over time these will be moved into namespaces.

And that’s sort of where the second discussion begins. The PHP team would like to retire support of the ereg_* function library into the PECL extension repository, supporting only preg_* regular expression functions in the PHP core from here on. This, of course, triggered a big discussion about what the development team for PHP would continue to support post-PHP 4.

One of the big issues is whether PHP 6 will have an on/off switch for unicode support in the php.ini file. With it, developers will need to write at least two versions of their scripts: one for unicode and one for non-unicode since the string functions in each will work differently. They may need to develop a PHP 5 version as well if they want to continue support for PHP 5. It was mentioned that this support structure is totally unreasonable for large scripts that are expected to work in multiple environments, many times without the users being able to control what is set in php.ini. It remains to be seen if the unicode on/off switch will remain in PHP 6 and what the resolution of these issues means for PHP moving forward.

Abroad View

Looking forward, it is likely that the focus of most of my work this week will be on the Abroad View site design implementation. We’re going to implement this in straight-up XHTML first so that we have templates that can be used on multiple platforms if we’re not able to get the SharePoint site fully operational by the time needed.

Chris has complted work on a unified Photoshop file for the Abroad View design that cleans up a lot of what the vendor left for us. I’ll be using this file and the images that he sliced out of it to create XHTML versions of the three core site templates: front page, aggregation site, and content site. I expect to spend most of Monday and Tuesday working exclusively on this, with deliverables to present at a meeting with the Abroad View folks Wednesday morning.

PaGES Framework

I’ve spent today tuning up the PaGES framework in anticipation of its new use as a frontend to Banner. In order for that to work, the PHP framework really needs to be a well-tuned, well-documented, well-tested machine. We don’t want to get into another Financial Aid web forms support nightmare. The improvements on the PaGES framwork focused on three areas:

1. Eliminate use of cacophony, PEAR, and MiddSettings
2. Comment and test everything
3. Use the new PHP 5.2 features in the SPL and PDO

The first point is an ongoing process. In order to be truly self reliant, we need to get rid of anything that references PEAR and the cacophony project does this all over the place. Fortunately, PaGES was started with some of this in mind and only reverts to relying on PEAR in areas where significant redevelopment is necessary, most notably the reliance on MAuthRA, which uses PEAR::DB_DataObject and PEAR::DB_ldap.

PaGES will continue to rely on MAuthRA for the time being, but anything else is being ported. This means that utility classes, like the Exceptions in cacophony and HTTPQuery need to be redeveloped in PaGES, though this is fairly minor work. I’m also killing off the centralized Singleton class, MiddSettings, and moving any settings into class constants and variables. Includes are done through use of the __autoload magic function in PHP rather than static function calls to MiddSettings::involve.

It’s also important that PaGES have a really solid foundation of code comments and unit tests. The idea for PaGES is that it will be supportable by a team of 2-5 developers within LIS and extensible to meet ongoing needs. For that to work, the library needs to be documented far better than the cacophony project ever was. Unit tests help here too by letting us run a barrage of tests against the whole framework after any change is made to the code base.

The last point focuses on what will be the replacement for some of the utility functionality I was using in PEAR: the standard PHP library functions for container objects and PHP data objects as an engine on top of which the PaGES database abstraction layer will sit. The PHP development team has really done some good work into having a more sensible (though still silly) core library of functions available for PHP 5, so we might as well use them. Hopefully by PHP 6 they’ll have killed off all non-PDO database functions so we won’t have 4 different ways to make a MySQL connection in the core code base. But probably not.

I focused today on porting some of the core classes of the PaGES framework into the not-relying-on-MiddSettings version of the framework with classes like Breadcrumb, Stylesheet, and Header included. The rest of the core classes should more along quickly with more work required for the forms related classes.

Athletics Recruiting
I spent most of last week putting together a spreadsheet of the non-sports-related questions for Athletics Recruiting and their estimated positions in the database schema. There are around 300 affected columns in Banner for these questions and many of the questions, like parent/guardian questions, allow you to add multiple records (i.e. you can continue to dump in parent/guardian information for as many parents/guardians as you like). Of these, there are approximately 10 for which we don’t yet have a column in the database.

There is also the case of many columns for which we don’t yet have a description code. For example, when you add a telephone number you have to specify its type code. There is a type code for “home phone”, but there is a concern that we may want to specify a specific type code for the Athletics Recruiting web front end so that it doesn’t interfere with recruit reporting elsewhere. Hopefully we can find a solution that doesn’t require us going through DIG for approval.

I’d say I’m about 65% done with this spreadsheet with work halted for the moment while I await a revised database schema from Heidi with the changes applied from our discussion last week. We met and made some changes to the original schema to reflect the fact that we’d be gathering data for multiple sports per application on the web form. This was not possible with the original schema which was designed for the tape load process where we only had to worry about a single sport per applicant. There are a number of other revisions, like a way to associate a coach with a sport and adding a relation type field to the relations table for parent, guardians and siblings.

General
I recommend paging through the Software Engineering for Internet Applications book. It’s courseware from an MIT course about designing a web application framework for an online community. There is really good and specific information in there about how to use up user databases and also good philosophical information about how to design for communities. The section on usability is particularly good.

What a day! This one involved a busy morning followed by a productive afternoon. There was also a scheduled update of production today and, as you will see, an unscheduled update of production.

Fall Out from the PHP Upgrade

Yesterday I updated PHP in production after a couple weeks of testing. This included an update to almost all of our PEAR modules. PEAR is an extension repository for PHP that includes a lot of third party scripts that make working with PHP easier. We use PEAR_ldap to make the Directory work and PEAR_DB_DataObject as a database abstraction layer for forms and TouchNet-connected forms.

It was that last PEAR module that really caused problems. Because PHP had been updated so infrequently, we jumped a number of versions of this module and I hadn’t reviewed all of the change notes for all of the different versions. A significant update was made to this package that didn’t show up on the development server, but manifested itself in production.

Each database connection through DB_DataObject is generated by parsing a set of schema files that let the script know what the tables and columns are named and what data type each column is. One of the config files, known as the “link” file can be set up to make joins easier. We never use this because it’s a real pain to write out a custom config file any time you want to add a join. Much easier to just write out the SQL in the query when passing it to the database. Most of the MySQL scripts don’t even use joins. The XML Forms and TouchNet Forms all have only a single table, so joins aren’t necessary for their functionality.

So we never updated the “links” files in production.

And now they’re required.

This caused the production scripts to bomb out with a non-descript error message. The error could not be reproduced in development and the only way to print out the error message in production would have been to start printing the error message, which includes the connection string (i.e. the database username and password). We were told by at least one office that one of their forms was getting hammered, so this wasn’t an option.

After Googling around a bit, I finally landed on a forum post that had to be translated from German that told me what was happening in broken English. I uploaded the file and all was well. Then other forms started breaking, so I uploaded the “links” files for all of the MySQL databases. Now I have to remember to do this whenever the table schema changes for any of the MySQL databases, which happens very frequently for the Forms and TouchNet forms databases.

All in all, this has strengthened my opinion that we just need to move away from using PEAR packages. The PEAR_ldap package in particular is cumbersome and horrendously buggy. The PaGES project is moving away from including anything of the cacophony project, including files that rely on PEAR packages. Hopefully by the time I’m ready to deploy the next version of the Directory, these problems will be a thing of the past.

Athletics Recruiting

I’m nearly done figuring out where all of the information goes, but I need to do a better job of putting it in the correct order. For example, I figured out how to enter information for coaches before I figured out how to enter information for a general high school sport, but the coach questions have to come after the general sports questions.

There are also a lot of these questions which can have multiple answers, such as those regarding siblings, coaches, sports, summer clubs, etc. For these I’ll need to create a mechanism where the student can select to add a new record or go on to the next question. For example, let’s say they have two siblings: a brother and a sister.

1. Select “New Sibling”
2. Select “Brother”
3. Fill in information for brother (name, contact, school)
4. Select “New Sibling”
5. Select “Sister”
6. Fill in information for sister (name, contact, school)
7. Select “No more Siblings”

The process needs to allow the student to repeat it as many times as necessary for the number of siblings they have (or want) to enter into the form, but must also allow the student to move onto the next question at any time. This is very important when we get into multiple stage question groups. For example, for each sport the student is going to have to enter general information, coach information, and statistical information.

This is yet another kind of custom form functionality that I’ll need to develop for the Athletics questionnaire. I don’t think even KeySurvey, which we recently purchased, is able to do repeating branching logic like this. The challenge is to make it transparent to the end user while also coding it in a way that doesn’t break under the strain of added complexity.

Scheduled Production Upgrade
Noting very exciting on this point today. The Fall Family Weekend form mentioned in yesterday’s post was uploaded along with a push of the MCMS project to correct a spelling mistake on one of the error pages.

I’ve been slacking off a lot on writing entires to this thing, so I think I’m going to start picking it up a bit and spending about a half hour at the end of each day writing down what I did during the day. I hope this provides a little insight into what’s going on in Web Services on a day-to-day basis. I fully encourage comments from anyone on how we could be doing these things better or what else you’d like to hear about.

PHP Upgrade to Production

Today marked the first upgrade of PHP in production that we’ve done in a really long time. This one was upgrading PHP 5.1.2 to PHP 5.2.3. There’s really nothing spectacular about this upgrade, except for a pile of security updates. We’re going to be doing many more regular upgrades of PHP than in the past. This updates is a little out of synch of this new philosophy, but we’ll be upgrading to the next-more-recent point version of PHP in the future. Currently PHP 5.2.3 is the most recent version.

This means that if PHP 5.2.4 is released tomorrow, we’ll be waiting until the release of PHP 5.2.5 to upgrade to PHP 5.2.4. If PHP 5.3.0 is released tomorrow, we’ll be waiting until the release of PHP 5.3.1 to upgrade to 5.3.0. The official releases of PHP are typically fully stable by the time the next point release is announced. We want to maintain a balance between keeping the version of PHP current for security purposes with maintaining application stability. We also test the code in development for a week prior to the upgrade to production.

Other than security fixes, PHP 5.2.3 (and the 5.2 branch in general) contains an improved version of the date and time objects, which are now built into the core of the language where they used to require a separate PEAR module. The JSON functionality, used to serialize objects for use in JavaScript, is also built into the language core now. The final big update is really only exciting for hardcore PHP geeks, but the default behavior for an object being printed or echoed is now to call it’s built-in __toString() function.

This last update means that we can do this:

<?php

$addrlist = new AddressList();

$addrlist->add(”Ian McBride”, “imcbride@middlebury.edu”);

print $addrlist;

?>

Instead of this:

<?php

$addrlist = new AddressList();

$addrlist->add(”Ian McBride”, “imcbride@middlebury.edu”);

print $addrlist->__toString();

?>

I’m sure you can see where saving those 13 keystrokes is well worth this massive change to application behavior. Quick note: if the first code block were executed in PHP versions prior to 5.2 the interpreter would print out the Object ID of the AddressList object rather than the return value of the __toString() function.

PHP 4 End of Life

It was recently announced on http://www.php.net that PHP 4 will end its life at the end of this year, meaning that security updates will no longer be issued for the product. For those who don’t know, PHP 4 is the version of PHP currently used on CAT for stuff like the Online Box Office, College Store, and Events Calendar as well as several pro bono hosted sites. While we certainly don’t have the most recent version of PHP 4 installed on CAT, so security updates are something of a moot point anyhow, this should still concern us as it highlights once again our need to remove ourselves from that environment.

Because of the instability of CAT on a weekly basis, the Events Calendar is already targeted for migration to FUSION and PHP 5 as soon as I can get to it. The other applications on this server that we’re responsible for mostly include Business Services applications that may change in nature when the new Point of Sale system is acquired for that functional area. There are also a whole host of web forms on the server that Travis is in the process of migrating to the XML Forms framework on FUSION. If all goes well, we should have major services off CAT by the end of the year, at which point the abandonment of PHP 4 will not concern us.

The impending release of PHP 6 on the other hand….

SharePoint versus MCMS

I answered two CMS related support tickets today as well. The first was a Faculty member who is also an Editor (can make and approve their own changes) to a department site on the CMS. They were asking why a student worker couldn’t add images to the site’s Resource Gallery.

Due to auditing requirements, students working on the CMS are all Authors (can make changes, but not approve them), except for a very small group directly supervised by Tim Etchells. This is to keep students from being able to directly affect the production web site. Unfortunately, while MCMS lets us have Authors able to edit but not approve content, if we give that same person the ability to add multimedia items to the Resource Gallery we have to also give them access to delete multimedia items. This opens the exact issue we’re trying to avoid by making the student an Author: they can now adverseley affect the production site.

Once we’re able to move the CMS content into SharePoint, we will no longer have this issue because the permissions structure of SharePoint is much more granular. Not only can we set permissions at the document level, rather than the container level, but we can set exactly what actions are enabled for a group of users, so we can let students drop their content into the Resource Gallery, but prevent them from deleting any content. SharePoint also provides a multi-stage deletion control system that gives Editor-level users access to a Recycle Bin from which they can restore items without contacting Site Administrators. The Site Admins can set how long items remain in this Recycle Bin after which they’re dumped into another Recycle Bin at the Admin level and then eventually purged.

Fun Note: MCMS never purges data. At all. Once the item is deleted by the admin it’s added to a special place in the database referred to as the “Archive Folder”. There’s no way to get at this data through the API, meaning that you have to violate the MCMS license to delete the data permanently. Consequently, our MCMS database has continued to grow exponentially since implementation.

The ability to set document-level permissions in SharePoint will also help us out with the next support request I closed today. Faculty profiles can be spread accross multiple departments if they teach courses in many disciplines. In order for a Faculty member to add multimedia content to a profile, they have to place this content is a bin where all of the Academic Coordinators have full permissions. With SharePoint, we can just set document level permissions on the posting and related documents and let the Academic Coordinators responsible for the content deal with it.

SharePoint Team Sites

On a less positive SharePoint front, I discovered two problems with the SharePoint Team Site today. First, the custom search box that I had implemented as part of the new design last week doesn’t work correctly. It shows no results for any search, even though search is working correctly. I’ve switched back to the default search box for now until this problem is corrected.

The second issue I discovered is that there is only one RSS feed for each list. This means that you can’t have different feeds for different views in the list. This was brought up because Mike wanted to have a feed for just Banner updates delivered through the Change Log. To do this with the SharePoint list, however, you need to apply filtering on the client end. Coincidently, there was a blog post about this very issue from one of the SharePoint developers today. They recommend using a CQWP to display the custom feed, but that requires writing custom XSLT for your feed unless you want the default view, which is just the title of the item. Interestingly, if you want to get email alerts from a list, you can do that on a per-view basis, so that might be a solution for people not wedded to RSS.

Fall Family Weekend Registration Form

Now that this form has been ported to FUSION, all of the alumni forms have been moved off of CAT. This should help us retire those forms. Heather Cahill had asked us to do so as many of the forms are still linked to through Google and they’ve had a series of registration’s come in for events that were offered years ago. The new XML Forms Framework is much better at handling the on/off nature of these forms.

I’d still like to investigate a better way of handling these items. This was originally part of the Alumni Online Community but was pushed back to the old web forms model because our payment gateway is difficult to integrate with. Each of these takes me 1-2 hours to fully implement correctly and their presence sometimes interferes with features of critical forms like the Giving form. We really need to find a self-service model for this area.

Online Box Office Yearly Maintenance

I discovered today that there’s yet another flaw in the Online Box Office user interface. Debbie Anderson asked for the events from last year to be removed from the database. After asking Chris how we did this in the past, I dumped the database to a backup file and then tried to use the “Delete this Event” option in the admin interface. This dumps you on the image upload page with a bad request and doesn’t remove the event. Now I see why Debbie was asking me to do this.

A quick “delete from events” solved that problem, but with yet another wart on the face of the Online Box Office application. This is one of Tyler’s old development projects. To be fair to him, it has run fairly well considering the business logic it has to bend to. “We’re setting aside this many tickets for online use and this many for in-store use, but adding them together will get you more tickets than we actually have, so we have to limit the number provided to…”, etc. I’ll reiterate my desire that this application reside within the College Book Store, or whatever we brand the combined online Business presence for the College, and tickets be sold just like any other item through that system.

Athletics Recruiting

I’m continuing work on the schema for this questionnaire. Hopefully, when we’ve fully tied each of the questions we want to ask to a field in Banner we’ll all be on common ground when thinking about the implementation of this project. I don’t have a whole lot to say about this right now, but check back for more updates this week as I’ll be working closely with Heidi to square away some of the weirder questions.

One interesting point came up in that we want to tie a recruit’s coach to the High School for which they coach. Coaches are stored in the relations table (SYRRFRL) with siblings and parents. Siblings and parents have college’s associated with them in this table in some columns I proposed named SYRRFRL_SBGI_CODE and SYRRFRL_SBGI_DESC which store the six digit identification number and 30 character school name of the college.

High schools also have SBGI codes! Now, should we shove the coach’s high school code in there and force the application to understand the context of the data in the column? That seems like really bad practice, but the alternative is having separate columns for high schools and colleges and then we get into the case where the coach’s high school isn’t the high school they attended, it’s the high school they work for. Meanwhile the parents and siblings SBGI codes refer to intsitutions which they attended. And what do we do about transfer students?

This project is particularly tricky because of the sheer volume of questions we’re asking of the user. I’m up to thirteen pages of questions and haven’t even got to the sports-specific, athletics-related, extracurriculars, or test score questions yet!