Railo 4 and IIS

So it has been a long, strange year of intensive Ruby on Rails development and I am glad to say I got both projects live and on to maintenance plans. I've been steadily releasing updates monthly and managing the feature list. Next I will be making a fork of one of the projects to make a similar app for a different department. That will require much more server and deploy issues, which I may talk about when that comes around.

However, now that I have had time to clear those up, a new project is to get our ColdFusion code from our old and hastily build Adobe ColdFusion 9 server to a new Railo 4 box. I've spent about a week working with a test IIS 7 box provided to me and have been allowed to build and fail and make a mess of things and I think I just about have it right.

There are documentation and discussions all over the place so I thought I should record my notes here. In general our server serves 4 sites and we had used multiple instances in Adobe's CF. I had wanted to replicate that process as well as lock down the Administrative interface from being accessed from the web (local access only). I don't think Railo does Multiple Instances as we have known them, but I wanted to run multiple web apps from the one server, which is nearly the same thing, almost. Not really, but for our purposes the technical differences are not meaningful. We want multiple sites/apps and no Admin access from the internet.

Prepping IIS

Ok, so the systems team built me a virtual server and he made some C-Names so that I could test multiple sites on the one box: app-testrailo.unm.edu and mobile-testrailo.unm.edu. I used my remote console and created the Web Server (IIS) role and included basic Web Server services and the Application Development services (ASP, .Net, CGI, ISAPI Extentions and Filters).

We keep our web files on a different drive from the C: so I created my sites: app and mobile and bound them to the urls and the path was set to z:\app and z:\mobile. Then I dumped in some sample apps so I would have something decent to test. Then I tested out the web server and viewed the successful HTML pages.

Prepping Railo

I read and followed the install instructions for Railo. The installer does it's thing very quickly. Up comes the basic Railo welcome page and links to my administrator. However, this is the Default root that lives in the default C:tomcat... Railo... Root folder. My regular web applications just gave me 500 errors. To link up my apps I needed to edit the Tomcat Host Config and restart the Railo service. Near the bottom of the Host Config file I needed to add these entries near the bottom:

         <Host name="hsc-testrailo.health.unm.edu" appBase="webapps">
             <Context path="" docBase="E:\hscapp" />
        </Host>

        <Host name="mobile-testrailo.health.unm.edu" appBase="webapps">
             <Context path="" docBase="E:\hscmobile" />
        </Host>

I also commented out the "localhost" entry since I was not using C:\inetpub\wwwroot\. I'm not sure if that was useful, but it seemed like a good idea. After a restart I am getting my apps to process the ColdFusion. Yay! Even cooler is seeing my sites in the list of Web contexts in the administration page. Double Yay!

Secure the Administrator

Now, I run some tests and the server administrator is available on the open internet. Boo! So we must secure the context. There are a number of Apache fixes discussed, but the key is to address the Bon Code Tomcat Connector for IIS. A Google Groups post gave me a hint. There is a "BonCodeAJP13.settings" file in the C:\railo\ajp13 directory where I can set the

<EnableRemoteAdmin>True</EnableRemoteAdmin>

to False. However this file is just the sample one, in my case the real one lives it the C:\Windows directory. It also administrative protected, so updating it was kind of a pain: file saved, Railo restarted, and my sites still work. And the url "http://app-testrailo.unm.edu/railo-context/admin/server.cfm" produces this error:

Access from remote not allowed (2).

Success!

Making Report Views in Ruby on Rails Pt. 2

Following up, once I have access to my views, I simply need to group and loop over my results and put them in a nice table. So, in my app/views/admin/reports/gme_by_opening_html.erb lets get started:

<h1>GME Department Coverage by Site Openings</h1>
<p><%= @results_count %> records returned.</p>

<% @results.group_by(&:dept_code).each do |dept_code,providers| %>
<table class="table table-striped table-bordered">
<thead>
  <tr>
    <th>Name</th>
    <th>Date</th>
    <th>Shift Hours</th>
    <th>Site</th>
  </tr>
  <tr>
    <td colspan="4"><h4><%= dept_code %></h4></td>
  </tr>
</thead>
<tbody>
  <% providers.group_by(&:full_name).each do |full_name, info| %>
    <tr>
      <td colspan="4"><b><%= full_name %></b></td>
    </tr>
    <% info.each do |r| %>
    <tr>
      <td>&nbsp;</td>
      <td><%= r.date_open.try(:strftime,'%-m/%-d/%Y') %></td>
      <td><%= r.hours %></td>
      <td><%= r.name %></td>
    </tr>
    <% end %>
  <% end %>
</tbody>
</table>
<p>&nbsp;</p>
<% end %>

So, what we have here (building on the previous post) is a list of doctors, from different departments who work at sites on different days for specific shifts. The grouping in this report is by department, then by doctor's name and a list of the sites by date. The key here is to group_by(&:_field_).each and then break out a sub grouping. Not too bad. But, wouldn't it be nice to have a filter to cut the data by date or other result? Like if I wanted a quarterly report?

In app/controllers/admin/reports_controller.rb I'm going to have a default time frame and accept filters. Also I'm going to create a nice label based on the dates (I think I'm supposed to put the label construction in the view, but I like it here where I can see all the filters I'm making- like if I wanted to filter on department instead). This code comes from Stack Overflow.

  def gme_by_opening
    if params[:start_date] && params[:end_date]
      filter_start = params[:start_date]
      filter_end = params[:end_date]
      @results = ReportGmeByOpening.where(
        "date_open <= ? AND date_open >= ?", Date.parse(filter_end), Date.parse(filter_start)
      ).order(
          "dept_code", "full_name", "date_open"
      )#.group_by(&:dept_code,:full_name)
      @results_label = "Records from " + filter_start + " to " + filter_end
    else
      @results = ReportGmeByOpening.where(
        "date_open <= ? AND date_open >= ?", Date.today, 1.month.ago
      ).order(
          "dept_code", "full_name", "date_open"
      )
      @results_label = "Records from " + 1.month.ago.try(:strftime,'%-m/%-d/%Y') + " to " + Date.today.try(:strftime,'%-m/%-d/%Y')
    end
    @results_count = @results.count

    respond_to do |format|
      format.html # gme_by_opening.html.erb
    end
  end

Then in my gme_by_opening.html.erb I change the header to:

<h1>GME Department Coverage by Site Openings</h1>
<div class="row">
<p class="span6"><%= @results_label %>, with <%= @results_count %> records returned.</p>
<div class="span6"><%= form_tag(admin_reports_gme_by_opening_path, {:class => 'form-inline'}) do %>
  <%= text_field_tag :start_date, nil, :class => 'input-small' %> to
  <%= text_field_tag :end_date, nil, :class => 'input-small' %>
  <%= submit_tag "Update Date Range Filter", :class => "btn" %>
<% end %></div>
</div>

Making Report Views in Ruby on Rails

I am in the middle of launching another Ruby on Rails web application that was left to me to finish. I've spent several months adding features, fixing the data models, cleaning up styles and UI. Pretty much maintenance tasks and support but without the main architect or developer. The remaining major issues left for me was to import 8 years of data and develop some reporting functions. I will probably write about the data export/import experience later. However, creating reports was a fairly complicated effort due to not a lot of explicit help or examples.

When I talk about reporting, I am referring to data models that bring together lots of tables worth of data and then allow the viewer to select, sort and filter on their own. In our shrinking and harried unit it is becoming clear that web apps are not the best place to manage data sets of any significance. Security issues, application responsiveness and development speed are all significant obstacles for us. Only ease of deployment and flexibility of access work in a web app's favor. And for in-house, departmental apps that may not be enough of an advantage.

Regardless, this app needs reports!

So, first I create the Views I need in SQL Server. No problem, I do that all the time, but how do I call them in Rails? Stack Overflow has a start for me. And I create a model to call the View I built.

rails generate model report_gme_by_opening

I throw away the migration that is generated and paste in the example.

class ReportGmeByOpening < ActiveRecord::Base
  # attr_accessible :title, :body
  set_table_name "report_gme_by_opening"
  #set_primary_key "if_not_id"
end

I don't need the primary key (there is none) and I don't think I need any accessible attrs since it is read only.

Wait, I need a reports controller in my admin part of the app if this is going to work.

rails generate controller admin/reports gme_by_opening

This gives me a blank controller and a definition of one report controller.

class Admin::ReportsController < Admin::AdminController
  respond_to :html
 
  def gme_by_opening
    @results = ReportGmeByOpening
    @results_count = @results.count
    respond_to do |format|
      format.html # gme_by_opening.html.erb
    end
  end
end

And then I make a gme_by_opening.html.erb file with some basic info.

<h1>GME Department Coverage by Site Openings</h1>
<p><%= @results_count %> records returned.</p>

And then I need to make a route available.

namespace :admin do
  match '/reports/gme_by_opening', :to => 'reports#gme_by_opening'
end

Finally I can make a link like this.

<%= link_to "GME by Openings", admin_reports_gme_by_opening_path%>

All the pieces work and I can get to my page and have some basic output. But next I need some filtering and grouping to make this useful to my customers.

Server Migration

So, networking found a massive compromise on our ColdFusion server and we've been spending a couple of weeks building a new one and getting it set up. Unfortunately, through the loss of employees to better opportunities, nearly all our expertise in building and deploying is gone. So, I've had to drop nearly everything and relearn how to set up a CF server. Not my favorite thing to do.

Rails vs MS SQL Server

So, I am finishing off a couple of Ruby on Rails apps that fell to me (from where I had just a supporting role) to finish and deploy. Several bugs that appeared in the production app where it had worked fine in development gave me fits.

Most had to do with the way that the development SQlite seemed to be more permissive than the production's MS SQL Server. Sometimes, updating rails to 3.2.x would fix it. But that would also introduce new problems.

The worst was when I finally got my bug list squashed only to have new bug for settled issues appear. Suddenly my .empty? statements stopped working. So I switched to .blank? on advice from Ruby Quicktips. I suppose I need to do some deeper reading on the language rather than just Googling when I need a function. Part of the trouble is that I will have to pay attention to Ruby versions and what might or might not work with a database.

I think it is also time to stop doing tutorials and actually write an app up from scratch.

Client and Server Side Form Validation Pt. 3

So, with a nice looking form, and some decent JavaScript processing my form works very nicely. However, I need to make sure that the server receives good data even if the JavaScript was turned off or not in play. All that form niceness is to help the user, now it is time to help the server.

Currently, my thinking on this isn't very solid. I'm asking the server to do a lot of work to double check the form and then determining if it should pass on the information to be processed or if the form needs to be displayed again.

This is not too bad if the form is small and has a predetermined set of fields. However, many of these forms are dynamically generated, so the processing has to be dynamically processed as well. This is where I try to take advantage of frameworks and partials to keep my display code in modular chunks- breaking out the form from the rest of the page.

Basically, I try to cfparam all the form fields and then validate the results and report them with the form. So if JavaScript does its job on the original form, great. If not, it still is taken care of before it hits the server. Regardless of the amount of work the server or I have to do, it is probably for the best.

Client and Server Side Form Validation Pt. 2

So continuing from oh, so long ago. I have moved from the previous basic form and posting and blended in Bootstrap, jQuery.Validation and a lot of Googling/research.

Styling up a nice, easy to read and flow form with Bootstrap's form code is straight forward and pleasant. Even if radio button groups can be tricky.

Next, I needed to bring in some jQuery validation. Pretty much the go to is the jQuery Validation plug-in from bassistance.de. It seems to have the most work and there were many tutorials and references in the Bootstrap issues discussion about getting it to work.

My starting point to getting from Ben Nadel's basic form upgraded is this little code sample called jQuery Validate Demo from aLittleCode.com. This will punch up your forms nicely. But the forms I work with extensively use radio groups (evaluation ratings of 1-5, sometimes with N/A and some need to be required).

From here it takes a little fixing and JavaScript to include radio buttons to put the error message in a decent place. I got my notes from these two posts: Mihir Chitnis and this discussion (starting with the last half of kwilliams's post) on the Bootstrap issues.

I'd like to see Bootstrap fold in jQuery Validation to their JavaScript library or for a solid plug in to be developed. But with some tweaking I think I got a code base to work for my code set up.

Lets see if I can find some place to put code...


Form HTML
Form JavaScript

Client and Server Side Form Validation

Something that has bothered me for a while is how to combine Client and Server Side form validation as painlessly as possible (as a developer) while still maintaining a pleasant user experience. Its not just about being able to fall back when Javascript is not available, but having secure input to the database. I like to consider the Ajax-y/jQuery to be mostly in service of making data entry more efficient.

To that end here is my form design thoughts.

1. Writing clear and concise labels and instructions (too many instructions means that my form has business logic problems)
2. Scripting input events to alert the operator to problems as they enter data and as they submit
3. Checking and sanitizing the data on the server before making changes to the database
• If the form is deemed valid, proceed to processing
• If there are problems, re-post the form and repopulate the data with error messages

From the reading I've done, there will always be some amount of duplication of code. This is the nature of web applications. This can be mitigated by having reusable code, both as JavaScript libraries and having modular server side code. How to do this will be the subject of this experiment.

My starting point is the article by Ben Nadel from December 2011 called How Client-Side Validation Is Changing The Shape Of Server-Side Validation. Part of the issues will be integrating the code into our style of FuseBox, finding a decent jQuery plug-in, I may have to buckle down and figure out how CFCs really work, figure out how to use Ajax and standard posts with the same code (probably related to CFCs) and lastly how to deal with dynamically generated forms.

Tall order. I should have done this years ago.

Temporary Break

So, I've hit sort of a slow down here. A few things will keep me away from posting on a schedule.

1. Seasonal family activities
2. Seasonal work load
3. Temporary hold on the Expired Passwords script

This time of year is very busy with family, (uninteresting) work back-log, and deployment of my script is on hold until various stakeholders can get together to talk about deployment.

I'm not ready for this to become a link-blog, so I'll return with new code when I get a chance to explore new scripts and ideas- hopefully mid August.

Put it All Together

After beating myself up over the last post, I rolled up my sleeves and got working logic code and it is sending me notifications of who's account password is expiring. I'm still having trouble with getting useable information out of the LDAP hash. I'm probably doing it wrong, so I've asked for a code review and maybe we'll get to the bottom of how Ruby is doing this. I keep getting values in the format of:

["gcolasurdo"] in stead of just gcolasurdo

when I try to get the cn, mail or other LDAP key field.

First, an outline of my code's logic (based on the previous ColdFusion script):

1. Loop over the relevant OUs (organization units in the LDAP)
2. Query the LDAP
3. If there is an expiration date and mail then...
4. Find the expiration date and calculate the difference from today (script will run daily)
5. If the daydiff is on a warning interval...
6. If the OU is for the College of Nursing or everyone else, prepare personalized email text
7. Then send an email with the details of the impending password expiration

Two questions that came up from the CF code. Should I:

1. Set a timeout at some point so as to not choke the mail relay
2. Log the results

One independent question I had:

1. Should we do anything about already expired passwords (I'm sure the business reason was already set, but as the programmer, I kinda want to make sure)

This is all working locally, so we will need to start in on the deployment process.

Ok, here is the code I'm using:

S is for Stalled

I was hoping to not have to do this so early on. It's the biggest problem in writing a blog; maintaining consistency and finding a voice. But here I am.

I've run into a block and I don't have any code to talk about this time.

Fortunately, I am learning something from this.

My difficulty with Ruby, and by extension Rails, is that it seems easy to get an application up and do fun things like utilize Ajax, query an LDAP or send email. The part I'm having trouble with is manipulating results.

It seems funny to stumble on the simplest of things, but when it comes down to it, that is the whole reason for an app: to manipulate and present data.

I'm just not getting the the pointers and hashes. I'm not getting the syntax and conventions. I'm not getting the methods that can be used on an object. It is going to take some reading and re-reading to get it. And beyond re-reading, I need to make some code samples and play with the results. Maybe I can show that next time.

Sending Email

Next up for my little scripting project is to send some email from our IT's mail relay.

There are a several nice Ruby gems that can make sending mail easy to do. Two are Pony and Mail. Pony is super simple, so I went with that for this script. Mail has a lot of features for importing body text, attachments and other great stuff. I will probably use it in a web application.

I would post some demo code, but it's basically right there in the GitHub Read Me.

I did run into some trouble. I got an error connecting to my IT department's relay.

/Users/garthcolasurdo/.rvm/rubies/ruby-1.9.2-p318/lib/ruby/1.9.1/openssl/ssl-internal.rb:121:in `post_connection_check': hostname was not match with the server certificate (OpenSSL::SSL::SSLError)

Stack Overflow asked and answered. By adding:

:openssl_verify_mode => 'none'

to my

:via_options => {...}

I'm receiving email from my department's relay.

Connecting to LDAP with Ruby

Looking at my tasks, the first order of business is to get connected to our LDAP server through a Ruby script. I'll start with a few Google searches and see what kind of LDAP gems I can use and if there are any examples or tutorials.
Here are a few results:

• Ruby/LDAP Tutorial
• ruby-ladp/test (github)
• HowtoAuthenticateWithRubyNetLDAP

For whatever reason, I chose ruby-ldap as my gem to install and ran with its tutorials and examples. Net-LDAP also seemed like a good choice.

I ran a little script just to get started.

But I'm not able to connect. Our LDAP is on SSL with a self signed certificate and I need to import it to my environment. I'm jumping ahead here a bit as I got that script from Christian Hofstadtler's post that helped me troubleshoot the issue. Since I'm developing on OS X, these instructions from Apple got my TLS_CACERT in /etc/openldap/ldap.conf configured correctly.

Now I'm getting data from our LDAP and I need to start filtering and refining. My basic script looks like this:

Move password expiration script from ColdFusion to Ruby

Today I started an interesting project migrating a ColdFusion scheduled task to Ruby. Why would we do such a thing? We have a number of problems with our CF server, primarily the CF spooler has been sending two copies of our organization's notice of password expiration. We've investigated and researched and have never come up with a satisfactory (or logical) solution for it. In addition our unit is moving away from CF and to Ruby on Rails for our application development. It would be in our interest to migrate existing business processes to the new environment.

Me, I'm struggling to get up to speed on Rails. I still maintain a lot of CF code and I have at least two current projects being actively developed in CF. I can support our other developer with html, css, testing and mock ups, but I just can't quite get on the Rails train yet.

My boss, probably at the suggestion of our other dev, assigned me to rewrite our failing expiration notification script in Ruby. This I can probably do. And I'll probably learn a lot about Ruby along the way. I'll break this down in our usual SBAR format.

Situation: Our organization expires passwords every six months. We should probably warn our users when their expiration is coming due; at 30, 14, 7, 3, 2 and 1 day. A script that runs every morning should search the LDAP looking for upcoming expirations, and at those intervals send an email.

Background: We have a working ColdFusion script and a history of utilizing our organization's LDAP. ColdFusion was quick and easy to set up, but there are problems with our server. Additionally, our unit is moving away from CF development. Developing a new script should not be difficult since the logic and assets are readily available. Hopefully, we can return responsibility of this business process to the Identity Management unit and not take so much flack for what the warnings do or do not do.

Analysis: To accomplish this project I have created the following tasks:

1. Connect to LDAP with Ruby
2. Report contents of LDAP in Ruby
3. Refine LDAP results
4. Send email to myself
5. Build logic structure based on existing code
6. Test locally
7. Deploy on server
8. Test on server
9. Schedule as chron job

Results: Successfully find and send expiration notices daily.