Fils

Varnish Reverse Proxy Accelerator (Ca...

2009-05-01T15:44:00.001-05:00

Intro:
The need to offer a mechanism to accelerate the performance of web applications could arguably be even more relevant as we move into a a more linked data approach. With the need to possible offer multiple representations of a single resource URI to address needs for XML, RDF or XHTML+RDFa (or microformats) etc. the need to improve performance and address request from cache when possible increases.

Related:
Past efforts in this area have involved Apache Connectors to Tomcat (Ref: http://tomcat.apache.org/connectors-doc/index.html ) and also reviewing Glassfish. Glassfish Portfolio (Ref: http://www.sun.com/software/products/glassfish_portfolio/ ) has many elements focused on performance and there is also Galssfish Grizzly (Ref: https://grizzly.dev.java.net/ ) which offers some some impressive sounding performance aspects using NIO.

Approach:
However, to address needs of maintaining URI persistence and caching what was really needed was a "reverse-proxy accelerator". Given that set of criteria I arrived soon at using Varnish (Ref: http://varnish.projects.linpro.no/ ). The target application is hosted on Glassfish but obviously Varnish can reverse proxy for a number of machines and any backend talking over http. Indeed, it's this ability that is important to me in order to address my desire to be able to alter systems and networking aspects behind Varnish and have it maintain my URI's in a "Cool URIs" style/approach (Ref: http://www.w3.org/Provider/Style/URI ).

Following the nice tutorial by Jay Kuri, that is located on that site it is rather easy to get Varnish up and running on Linux and doing reverse proxy and caching. I would like highlight a few things Jay, mentioned in that write up and detail a few aspects I ran into.

One of the biggest that Jay, mentions is the issue around Varnish not wanting to cache anything that has a cookie reference. Since almost all web based applications are going to do this as a simple session management approach (even if you are not doing accounts), Varnish by default will not cache your URIs. Jay, goes on to recommend code like:

    if (obj.http.Cache-Control ~ "max-age") {
        unset obj.http.Set-Cookie;
        deliver;
    }

to override this behavior and respond from cache for content. As noted this does mean that is starts to become depending on the content provider/web app developer to issue the commands to inform cache system like Varnish about the behavior we expect from them.

Setting behavior in Grails:
Using Grails (Ref: http://www.grails.org ) it's easy to set the format of our return in via the withFormat{} syntax. Note we would want to make the approapriate entries in our Config.groovy grails.mime.types section (Ref: The "content negotiation" section of http://www.grails.org/1.0+Release+Notes )

So something like:

    withFormat {
      html {
        response.setHeader("Vary", "Accept")
        def nowPlusHour = new Date().time + 3600000
        response.addHeader("Last-Modified",
            String.format('%ta, %<te %<tb %<tY %<tH:%<tM:%<tS %<tZ', new Date()))
        response.addHeader("Expires",
            String.format('%ta, %<te %<tb %<tY %<tH:%<tM:%<tS %<tZ',
             new Date(nowPlusHour)))
        [allSites: allSites, allAutoSites: onlyAutoSites]
      }
      rdf {
          def data = modelAsRDFService.asRDF(AgeModel.findAllByLeg(params.id),
   "/loc/sites/${params.id}")
          response.setHeader("Vary", "Accept")
          response.contentType = "application/rdf+xml"
          def nowPlusHour = new Date().time + 3600000
          response.addHeader("Last-Modified",
            String.format('%ta, %<te %<tb %<tY %<tH:%<tM:%<tS %<tZ', new Date()))
          response.addHeader("Expires",
            String.format('%ta, %<te %<tb %<tY %<tH:%<tM:%<tS %<tZ',
             new Date(nowPlusHour)))
          response.outputStream << data
      }
    }
}

In this code we have set the LAST-MODIFIED and EXPIRES header entries. Note in this simple example I have simply pushed the expires time ahead by one hour. You can set this however you wish depending on your view of the relative age your resources can be and still be valid.

The VARY header is set to address some linked data best practices. This informs the client that the representation of this resource URI can change based on how we call it. Here it can be requested as HTML (what is in fact XHTML+RDFa in our case.. a whole topic in itself) and RDF.

There is no builder for RDF, so I simply pass the model to a service that generates and returns this for me. The Export Plugin (Ref: http://www.grails.org/Export+Plugin ) might be worth looking at if you are looking for various other formats to serialize your data to. I don't set the content type for the HTML in the code above, on review I likely should though by default it is coming back as text/html so that may be fine.

Be sure to have your RDF contain a reference to itself and its resource URI if you are doing a linked data approach.

There is also likely other response header elements one could consider here based on the reverse proxy and caching needs. A review of http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.4 and your accelerator package may reveal other setting of value for your particular environment.

Using curl to validate behavior:
While I really love Firebug, I found it to not be the easiest to use to verify proper cache behavior by Varnish. Rather, I used (being a bit of CLI lover), curl. A curl request on a resource URI is made using the -D out option to capture the headers. Two consecutive headers are shown below:

HTTP/1.1 200 OK
X-Powered-By: Servlet/2.5
Server: Sun Java System Application Server 9.1
Vary: Accept
Last-Modified: Fri, 1 May 2009 16:10:00 CDT
Expires: Fri, 1 May 2009 17:10:00 CDT
Content-Type: image/png
Content-Length: 20951
Date: Fri, 01 May 2009 19:59:32 GMT
X-Varnish: 868897910
Age: 0
Via: 1.1 varnish
Connection: keep-alive

HTTP/1.1 200 OK
X-Powered-By: Servlet/2.5
Server: Sun Java System Application Server 9.1
Vary: Accept
Last-Modified: Fri, 1 May 2009 16:10:00 CDT
Expires: Fri, 1 May 2009 17:10:00 CDT
Content-Type: image/png
Content-Length: 20951
Date: Fri, 01 May 2009 19:59:34 GMT
X-Varnish: 868897911 868897910
Age: 2
Via: 1.1 varnish
Connection: keep-alive

Note the two values in the X-Varnish response header entry indicating to us that the content came from the previously cached content. If we continually saw only one number here would need to investigate why our request is being passed through to the backend server.

Conclusion:
Varnish, with a few considerations related to explicate cache control instructions provides a nice acceleration to web application performance. As a from the start reverse proxy (vs something like Squid which started life as a forward proxy) it plays a valuable roll in linked data approaches by allowing changes to systems and networks behind the scene while allowing URI persistence to be maintained. I've also had no issues with it with respect to 303 redirects on generic documents, also important in linked data approaches.

Neptune Database Services and Scripting:

2009-02-16T11:48:00.003-06:00

The use of services on the net has on interesting set of issues to address with respect to acting as generic data resources. Doing mash ups with RSS or ATOM feeds or weather or other somewhat periodic and serial data has become rather common. Yahoo Pipes (Ref: http://pipes.yahoo.com ) and other such web applications do a good job dealing with these types of data end points.

However, when working with scientific data sets and services like those at CHRONOS (Ref: http://portal.chronos.org/gridsphere/gridsphere?cid=res_dev ) a different set of use issues comes up.

Let's look at a basic XML-RPC styled service endpoint for the Neptune database at CHRONOS (Ref: http://www.chronos.org ):

http://services.chronos.org/xqe/public/chronos.neptune.samples.advanced?callback=displayNexus&time_range=33-34&fossil_group=Diatoms&include_dated=true&serializeAs=wrs&

Here I have used a service from the XQE engine that has set the time range to 33-34 Ma, fossil group to Diatoms, indicated that only dated items should be used and set the output serialization to web row sets (generic XML).

However, let's say by use of a recent example, we wanted to locate all the unique taxon names and drill holes from various time ranges and fossil types from the CHRONSO Neptune database. We now need to loop on a set time times and names and then locate the unique, not full, set of names and locations. We can do this with the service above, but not directly. We need to collect and process a range of calls on that service to achieve this.

One could develop a small program in Java, Ruby Groovy or some other language. Dynamic languages like Groovy are especially well suited for such efforts and a a simple script like:

10   //Upper Oligocene   Sub-Series/Sub-Epoch    23.8    28.5    Oligocene 
11   //Lower Oligocene   Sub-Series/Sub-Epoch    28.5    33.7    Oligocene 
12   //Upper Eocene      Sub-Series/Sub-Epoch    33.7    36.9    Eocene 
13   //Middle Eocene     Sub-Series/Sub-Epoch    36.9    49      Eocene 
14   //Lower Eocene      Sub-Series/Sub-Epoch    49      55.05   Eocene 
15   //Paleocene         Series/Epoch            55.05   65      Paleogene 
16    
17   import sun.net.www.http.HttpClient 
18   import org.apache.commons.httpclient.methods.GetMethod 
19    
20   def ageRanges = ['23.8-28.5', '28.5-35.7', '33.7-36.9', '36.9-49', '49-55.05', '55.05-65'] 
21   def fossilTypes = ["Diatoms", "Planktonic+Foraminifera", "Radiolarians", "Calcareous+Nannoplankton"] 
22    
23   fossilTypes.each() {type -> 
24       ageRanges.each() {range -> 
25           def urlxml = "http://services.chronos.org/xqe/public/chronos.neptune.samples.advanced?callback=
displayNexus&time_range=${range}&fossil_group=${type}&include_synonyms=true&include_dated=true&serializeAs=wrs&" 
26    
27           def client = new HttpClient() 
28           def get = new GetMethod(urlxml) 
29           client.executeMethod(get) 
30    
31           def xmlSlurped = null 
32           try { 
33               xmlSlurped = new XmlSlurper().parseText(get.getResponseBodyAsString()) 
34    
35               def entries = xmlSlurped.data.currentRow 
36               def legSiteHole = [] 
37               def bug = [] 
38               entries.each {entry -> 
39                   legSiteHole += "${entry.columnValue[3]}_${entry.columnValue[4]}_${entry.columnValue[5]}" 
40                   bug += "${entry.columnValue[11]}" 
41               } 
42    
43               println "Total tax count is ${bug.size()} and total LSH count is ${legSiteHole.size()}" 
44               println "In range " + range + " of type " + type + " found " + bug.unique().size() + " unique 
taxa in " + legSiteHole.unique().size() + " unique leg/site/holes" 
45               println range + " " + bug.unique().size() 
46           } catch (org.xml.sax.SAXParseException e) { 
47               println "Exception parsing XML" 
48           } 
49       } 
50   }

This script uses the Apache Commons HttpClient, but there are many way to do this. The point I am after is not to show a best practice in using Groovy to access services (I'm sure there are many aspects of that code that people could comment on). Rather it is to highlight an issue related to balancing service generality and usefulness.

Note I did attempt to use both the XML and the character separated values
version via Yahoo Pipes and also the defunct Google Mashups Editor.
However, the need to have a more programmatic interface to conduct
loops, basic array manipulation and such is beyond what these can do
easily. While it might be possible, I did not find it easy or intuitive. These resource seem more focused on RSS or ATOM sources and simple filtering and counting.

Additionally the
Kepler work flow application is a bit of over kill and doesn't yet seem to
work well with REST or XML-RPC style services as well as I feel it should. A heavy focus on WS* and other more heavy duty scientific data flow operations mean it's lightweight rapid service mashup capacity is limited (IMHO).

Making a resource or service of general "popular" use may make it of generic interest but can limit its utility on its own to address specific scientific efforts.

Expecting generic services to be of use via the implementation of packages like Yahoo Pipes or Kepler runs into implementation aspects of those tools. Aspects which may make use of them sufficiently complex as too discourage users.

However, expecting people to write code like the example above, even in a language like Groovy that makes calling, using and parsing the data rather easy is also potentially unrealistic. Not everyone likes to write code.

So, there is a dilemma for service providers of science data like CHRONOS. Make the services too generic and you risk them being of little use to focused research communities. Address the vertical needs of a specific community or tool interface and risk making them of limited utility to others. Additionally addressing the needs of several vertical communities could be taxing from a human resource point of view.

One does wonder if the creation of a domain specific language (DSL) might have benefit to address this. If there was enough community interest to justify it's creation a service that consumes and process a DSL would allow a sort of "custom command line interface.

The Sloan Digital Sky Survey SkyServer (Ref: http://cas.sdss.org/astrodr7/en/ ) addresses this by simply using SQL as that "DSL" language. The page http://cas.sdss.org/astrodr7/en/tools/search/sql.asp allows one to structure and submit SQL directly to the database.

However, a more focused DSL might be able to address the needs of special research groups while consuming more generic service end points that could be exposed for other to use in similar approaches to the code above, via the DSL or their own approach.

Also the development effort in making such a DSL might be able to be spread across multiple efforts making it more appealing from a human resource point of view.

Just some ramblings on how to balance generic services (resources) vs the need of special communities for more focused and unique services. Regardless if they are REST, XML-RPC or WS* in nature.

Other references:
Kepler Workflow Application: http://kepler-project.org/
Google Mashup Editor http://code.google.com/gme/ (defunct)
XPROC http://xproc.org/ and http://www.w3.org/TR/xproc/
GRDDL: http://www.w3.org/2004/01/rdxh/spec
DSL in Groovy: http://docs.codehaus.org/display/GROOVY/Writing+Domain-Specific+Languages

REST SOA and multi-parameter databases

2008-12-02T10:52:00.002-06:00

Thoughts on REST and SOA's for database access with multiple parameters

Following that a URI represents a "resource" then it can become an involved process to create a REST compliant approach to RDBM's based resources that allow users to define multiple parameters to request results in a Resource Oriented Architecture (ROA) approach.

One begins to view "Plain old XML" (POX) defining parameters and constraints passed over HTTP as a REST approach. However, this tends to border on a basic RPC approach. Especially when the endpoint that receives the POX represents a method to run. Thus we are more Services Oriented Architecture (SOA) now than ROA.

Alternately a URI template approach quickly becomes complex and also very RPC like when we simply define a syntax for the parsing of a URI into what is honestly still a method call. There is also the issue that even something as simple as .../resource/facetA/facetB is different to our caching and general web architecture stack than ../resource/facetB/facetA even if the result (body) is both associative and commutative in operation with respect to these facets. Arguably a URI doesn't need to be this way, but that seems to imply a URI template again and then it seems there is prior knowledge involved server side.

One could argue that an aspect of the uniform interface approach in REST is that resources are manipulated via model transforms. So a URI represents a resource and we operate on these resources through application of transformations. Obviously this raises visions of resources as XML and transformations being applied via XSL. Whether this is a valid REST/ROA approach remains as an argument I suspect (or does it?).

However, if the goal of the architecture is to allow a user to generate a result based on arbitrary parameters and constraints and then allow the capacity to pass a representation of these results along to another user as a resource then the results of the initial request must result in the generation of a new resource. This new resource being a representation of the various parameters and constraints requested from the database.

Defining that as a constraint of the architecture we are looking for then there are perhaps two questions to answer:

If the endpoint we are passing our parameters to is a method then we are RPC and likely should just call ourselves a SOA and move on.

If "yes" to 1 then could the result of this RPC/SOA call generate a resource that could then be passed along on a ROA side of this architecture?

If we believe a REST approach is useful here can we define a "resource" that provided a transform or set of operations results in a set of resources that represent our "results"

One has to be careful not to "create" something we call a resource that simply is a method anyway and thus violates a ROA approach anyway.

Does a REST approach imply requesting resources with individual parameters/facets and then any subset/intersection or other transform is duty of the client to perform. The results of any multi-parameter request is a set of resource links,
not the data. The data the client will use (plot, animate, etc.) comes
from those resource links.

This puts undue (or not) burden on the client for large data sets
This makes passing simple representations of resulting data set to other users/machines problematic.

However, REST is optimized for large-grain hypermedia data transfer (Ref: http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm ) and thus one arrives at this point wondering if this posting is discussing a fine grained approach to data that is not the forte of REST.

One has to consider WS* if you are allowing for RPC anyway. If you are going to open the door for RPC, then why not just take on
the WS* stack and benefit (and suffer) from the formality of that? One can argue about the conspiracy
of tool developers in the evolution of WS* all you want. However, in
the end you are attempting to get a goal achieved and if your goal is fine grained parameter access to database over HTTP then perhaps the SOA/WS*/SOAP approach is what you need (even if it's not what you want).

Clearly though WS* brings a level of complexity that is not welcome in many places. So then, as mentioned at the start of this post, one looks to POX over HTTP via POST as an interesting approach.

There is nothing that says the result (or an addition by product) of this service call can't include the generation of a resource or that a resource creation could be explicitly requested when passing a representation/resource is desired. The implication being that such a resource is not implicitly generated for each service call. The "Layered System Constraint" allows us to build up more
representations (views) into our resources to accommodate architectural
approaches needed in our effort.

A process file representing a workflow to be conducted can be used to pass to others to allow for "gettable" results. Or could be POST'ed back to the server (another server) to generate a "gettable" result. How then is this "workflow document" constructed? JSON ala CouchDB views (Ref: http://wiki.apache.org/couchdb/Introduction_to_CouchDB_views?action=show&redirect=Views ), XProc (Ref: http://www.w3.org/TR/xproc/ ), GRDDL (Ref: http://www.w3.org/2004/01/rdxh/spec ) transform links, other? The exact nature of the construction isn't important at this level in the discussion.

If application state is the purview of the client, then the client is
free to use any method to maintain that state. Itself, the involved
application server or some other application server.

Implications for HATEOAS (hypertext as the engine of application state)?

links can be implicate or explicate
links in mediums other than HTML (examples)

So then if the issue of multiple parameters is one that is fine grained and best approached by SOA while a resource driven approach is naturally ROA in nature something like the following can be seen as a zeroth order start on database exposure in a hybrid SOA/ROA approach.

ROA

Represented by URI's like:

.../res/paramter
(a large number of resources returned using HATEOAS to navigate the result tree to data)

.../res/ID
(a single resource data represented based on request type)

Can use the standard CRUD REST mapping (and all its good and bad points) to manipulate these resources and mimetype/request type approaches to alter the serialization of the data.

ROA with rendered views*

Represented by views on the resources like

.../res/age/10
(implying all elements 10Ma or older)

.../res/depth/345
(implying 345 mbsf or deeper)

../res/fossilegroup/nanno
(resources with data from fossil group Nanno)

../res/taxon/X
(resources with data about taxon X)

.../res/ics2004/earlyjurasic
(resource with data from early Jurassic as defined by ICS 2004 timescale)

SOA (WS*, POX via POST, etc.)

Standard SOA method calls via SOAP envelopes.
(lots of baggage)

POX over HTTP

JSON plus REST

or even just pushing .JS in general over to a server ala Perserver or FeatherDB.

* All these URI examples are based on paleontology/geology approaches associated with the kind of data CHRONOS works with and may be somewhat opaque at first glance.

Resource Oriented Architecture

"Classic" REST in a ROA approach. URI point to individual resource which return their data. The "view" of the data may change based on the request type and other elements of the web architecture also are there to address caching (e-tags) and other elements of scaling. Implementation wise this could be Jersey or CouchDB or many things.

Resource Oriented Architecture with dynamic creation of views

Adding the ability to create dynamic views is a new level. While CouchDB supports views they have to be registered and indexed. Other packages like Perserver (Ref: http://www.persvr.org/ ) and FeatherDB allow for views to be dynamically created and run via REST calls. The performance and scalability issues of allowing such dynamic view creation would need to be evaluated. Also the views themselves tend to be Javascript based or based on JSON query/path expression syntax. Though this is not always the case and it should be noted that CouchDB, though requiring prior view creation, supports a wide range of languages for views.

ROA / SOA hybrid

If the goal as stated is to allow fine grained multi-parameter access to a database while leveraging off the scaling and ease of use aspects of REST/ROA then a review of various approaches to a ROA/SOA hybrid is beneficial. Obviously there is a high degree of coupling between the service (method) and client in all these cases.

Seq A (One call, all the data returned)
Classic service call as either XML-RPC, WS*, POX over HTTP (RPC), JSON over HTTP (RPC) or however you need to communicate a request to a method and get your results back. Whether the method is directly exposed or a more document centric approach where an XML or JSON package is processed is not relevant.

Seq B (One call for ID collection then N calls for matching resources)
Method call made but rather than data a collection of ID's for matching resource are returned. The client then makes further calls for each matching resource.

Seq C (One call for ID collection, One call for collection resource, N calls for matching resources)
Method call made which results in the creation of a new resource. The ID of that resource if returned to the client which then calls to it to gather the associated resource ID that are called to return the requested data.

Comments on these approaches
Obviously each version is getting more and more complex and involving more and more network calls and the inherit latency that creates. Seq C does result in a new resource that can retried via REST compliant GET and can then return a HATEOAS compliant collection of resources to be retrieved

Accessing (opening and closing) N number of resources over the network is also more involved than returning one large document of all the resources combined. The first approach does lend itself well to a scaled out architecture though. One might consider the utility of AtomPub (Ref: http://www.atompub.org/) in this architecture.

Not mentioned yet but important especially with creating new resource dynamically is the use of http codes (Ref: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html ). In particular 201 Created or 303 See Other may be relevant. Obviously using the existing web architecture as much as possible has value. The general shunning of the web architecture by WS* services has not been to its benefit in the authors opinion.

Conclusion
The large grained focus of REST makes it hard to accommodate access to database with a large number of parameters and criteria exposed to clients for them to construct queries against.

SOA based approaches that are more RPC in nature match this need better but introduce strong coupling between client and server and are generally more complex to create, invoke and evolve.

An architecture than combines ROA and SOA approaches is not impractical

A ROA (REST) approach for easy and highly scalable access to data based on basic parameters and a number of pre-defined "views" to the most common queries
Service/method calls to support multi-parameter calls to a data source to allow arbitrarily complex requests
Service/method calls that allow new "views" to be registered when it is felt than they represent a new and potentially popular view to the data or when a passable, REST compliant resource creation is desirable

How an implementation of this might be constructed is a future topic.

take care
Doug

Refs used for this document:
http://www.slideshare.net/alan.dean/separating-rest-facts-from-fallacies-presentation/
http://josh-in-antarctica.blogspot.com/2008/11/restful-query-urls-cont.html
http://www.eherenow.com/soapfight.htm
http://www.informationweek.com/blog/main/archives/2008/08/rest_vs_soap_ro.html
http://www.25hoursaday.com/weblog/2008/08/24/RESTfulJSONBringingRESTAndRPCCloserTogether.aspx

REST via URI's and Body Representations

2008-11-18T10:44:00.001-06:00

I've been thinking about REST URI's and their structure for a while now and been talking with my friend Josh about these. As an exercise to try to codify my own views on this I have written up some of my thoughts on this topic.

URI templates:
First order approach to REST is the URI used each day via GET

Good:

Can become book marks (thus IM'd, twitterd, and emailed to ones content)
Simple and easy to code to in a variety of languages and API's
Classic resource representation

Bad:

Limited ability to pass multiple parameters
Limited ability to deal with more than one facet (at least with the example template below, any template could be established, but then everyone calling needs to know it)
Require a template even to pass one parameter
Can easily start to try and carry to much info with all sorts of verbs and values

So by example a URI template might look like:

.../resource/facet   (where the facet is some value like ../lithology/sand)

Then a template could define something like

.../resource/facet/[match]   Given a single value after facet, try to match it (equals or substring?)
.../resource/facet/[/min/max] Given two values, assume they are a numerical range to search for

Of course, the issues already start to build up. In the single value case we could just try to match but if we want a sub-string match then we are assuming we are not in a numerical environment. Or we are requiring the implementation of the service to check for primitive data types which may or may not work. In the second case our min max has to be sure to address numeric primitives (easy to check ) but also we may want to address non-numeric ranges (January/March)

There is also an interesting use of language that can be done here. Where resource (singular) requires an id to define a single resource. Conversely, a plural resources then would indicate an additional facet (either plural or singular) to carry the template forward.

So:
.../resources/sand   (all resources with sand)
.../resource/34/sand (sand attribute (facet) of resource 34)

The use of singular and plural attributes in URI templates seems logical. As a complete aside I have often wondered if a more Latin style grammar would allow more descriptive URI's to be formed since word order is less important and meaning is carried in word itself. Totally impractical though of course.

URI templates with representation in the body
So if our template is not enough, exploit the fact that the web architecture doesn't just pass URI's but additional elements in a request/response including headers and a body(representation).

So now our template (../resource/facet) might get a POST call with a payload of XML or JSON (pick your flavor I guess, how about Microformats in REST ). That payload is free to define a wide variety of parameters and actions.

Good:

Far more flexible in terms of defining what we want to do with a resource facet
A response payload can define downstream (next) actions to take. Thus giving us a kind of client moderated work flow (ie, 201 created or 303 see other next steps to take)

Bad

These workflows really can't be very complex since we are "waiting around" for the completion with our session. Really only good for events which are "quick". *
The XM or JSON schema/language for the payload has to be agreed on before hand (though this is really true of the URI template too)

As an aspect of this the service might establish (via the 201 or 303 codes) a new URI that is a GET'able representation of the response of this (complete with etag and everything). The use of status codes (Ref:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html ) needs to be
integral part of any REST approach and REST clients need to realize
that if they are going to use the web architecture to get a resource
they must look for and deal with the status codes of that architecture.

However, there are issues with the service provider caching requests and responses:

How long does the service provider keep this?
A service may be called thousands of times by a single person in a sinle session, it doesn't scale (does it?) to keep and etag them all
Just let the web arch do the caching

Perhaps, one might allow this stored representation of the response to be the URI plus the POST'ed request and then if some does a GET on that URI representing this earlier POST request allow whatever web caching there is to address that scaling Ie:

POST request to ../resource/facet
response could be the actual data with a 201 created URI for future referencing. That new URI, say (../resource/request/[id]) could then in the future be called via GET. All the service provider is storing in that case is the URI plus the request payload, since the results would be rebuilt by the service engine (and then available for near term caching via existing web architecture). Etags (hash sums) could be used to ensure requests are to a specific resource.

One element I have been working to deal with in REST approaches is the whole "hypermedia as engine of application state". It's a guiding principle and yet sometime hard to understand what exactly it is or if an approach is in compliance with it at times.

One approach would seem to be to say that all elements of a response must define down stream references as URI's or other (are there other?) hypermedia compliant references. I am not sure if if an img tag or microformat or RDFa embedded content is compliant with this "application state" approach. Also, how exactly does this impact the use use of XML or JSON. While XML could be embedded into XHTML or similar effort done with microformats, does the use of JSON in the response to a REST URI mean that "hypermedia as engine of application state" is not even an option? JSON embedded in XHTML or microformats perhaps might address this but then one argument the JSON proponents make is that they are tossing out the cruft. One could also place the URI into the JSON as a value. (Josh, I think you are looking into this.. I will be curious to read about)

Indeed it's also interesting to look at GRDDL and note how it approaches the walking and extraction of information (RDF) from documents (like XHTML) when looking at issues of REST.

take care
Doug

Ref:
http://josh-in-antarctica.blogspot.com/2008/10/restful-query-urls.html
http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
http://oreilly.com/catalog/9780596529260/
http://www.infoq.com/articles/tilkov-rest-doubts
http://www.infoq.com/articles/mark-baker-hypermedia
http://microformats.org/wiki/rest/json#Proposals
http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
http://www.w3.org/2004/01/rdxh/spec

* With regard to only good for quick work flows.. one must agree that it is easy to send in an event that may take a long time to run and simply reqister a message to a queue (ATOM feed perhaps) that has the current status and the next event on status update to undertake. This turns the burden of monitoring and moving messages through the queue to the client in this case, but it does allow for a disconnected work flow in compliance with web architectural approaches.

JSR-311 and JPA with Intellij 8

2008-11-06T15:32:00.004-06:00

IntelliJIDEA 8 recently came out and I wanted to take the time to try out a couple of interesting technologies as a means to test out the new version. Specifically JSR-311 (Via Jersey ) and Java Persistence API (JPA) via Hibernate. I created a simple application using these two technologies. The data store I used in this was a PostgreSQL database with lithological data. Note I actually did this with version 8.0 M1 of IntelliJ (build 8664) and I will go on the assumption nothing changed in this regard to the 8.0 final annouced just this last day or so.

The first step was to register up the database as a datasource. Under the TOOLS menu is the DATA SOURCE... menu item. Selecting this allow us to register our database as a source through the dialog below:

Once this is done we can go ahead and create out new project. You wouldn't have to register a data source first, but for sake of a canonical "10 minute demo" the net seems to love it makes life a bit easier as we are about to see.

Go ahead and start a new project with FILE -> NEW PROJECT. Use the default "Create project from scratch" and name your project. We will be building a "Java Module", the default option when creating a project. Pick the default option for the src directory and when you get the "New Project" window selections options like in figure 2 below:

I did NOT select "Hibernate" at this stage, though I did select it under the pull down for the "JavaEE Persistence" option. Also, under "Web Application" -> "WebServices" select the "Jersey" option.

The project will start to be fleshed out and you will be prompted to import the database schema (assuming you selected the "Import database schema" option). Now select the source you created via the "Data Source Properties" dialog. For me this was "Lith". This database is just a test database for now so it consists of only a few tables with no defined relations at this time. For purposes of this simple demo that should be fine.

Go ahead and select a datasource and define a package name for the entities to be generated into.

You will be prompted that the OR mapping is about to be generated, go ahead and let it start. Depending on your database and your machine this will take a minute or so.

Once this was generated I was confronted with an Error that the org.hibernate.ejb.HibernatePersistence class/package could not be resolve for its reference in the persistence.xml file that was generated for us by this process. Long story short I went to http://www.hibernate.org/6.html and downloaded the annotations, entitymanager and core (distribution) packages from here.

Setting up the libraries (as is often the case) was the only real tedious part and in the end the library collection looked like the following figure:
NOTE:   I have to wonder if I would have selected "Hibernate" in the "New Project" dialog above (leaving the import and class generation option unchecked) if Intellij would have imported the Needed Hibernate libraries for me.
NOTE 2: Don't forget to add in your database driver too (not that I did that or anything)

Parallel to all this JPA/Hibernate stuff Intellij has created a simple JSR-311 (Jersey) class for us with the following default structure:

package example; 
 
import com.sun.net.httpserver.HttpServer; 
import com.sun.jersey.api.container.httpserver.HttpServerFactory; 
import java.io.IOException; 
 
import javax.ws.rs.GET; 
import javax.ws.rs.ProduceMime; 
import javax.ws.rs.Path; 
 
// The Java class will be hosted at the URI path "/helloworld" 
@Path("/helloworld") 
public class HelloWorld { 
    // The Java method will process HTTP GET requests 
    @GET 
    // The Java method will produce content identified by the MIME Media type "text/plain" 
    @ProduceMime("text/plain") 
    public String getClichedMessage() { 
        // Return some cliched textual content 
        return "Hello World"; 
    } 
 
    public static void main(String[] args) throws IOException { 
        HttpServer server = HttpServerFactory.create("http://localhost:9998/"); 
        server.start(); 
 
        System.out.println("Server running"); 
        System.out.println("Visit: http://localhost:9998/helloworld"); 
        System.out.println("Hit return to stop..."); 
        System.in.read(); 
        System.out.println("Stopping server"); 
        server.stop(0); 
        System.out.println("Server stopped"); 
    } 
}

I wont bother to break down the structure or go into the various annotations used in JSR-311. You can Google up quite a bit of material on all that of far higher quality than I could produce. Starting at the Jersey site (https://jersey.dev.java.net/ ) is as good a place as any as I think it's likely the most evolved JSR-311 implementation at this time.

For simplicity we will leave the main() method alone and modify a few other elements in this class. First I changed the class level @Path annotation to:

@Path("/lith);

and also added in a create and close method for the JPA EntityManagerFactory. For fun I modded the getClicedMessage to parse out the URI path sent to it via a:

@Path("location/{latlong}") annotaion along with a @PathParam("latlong") String latlong annotation. The later requires the javax.ws.rs.PathParam import.

So our final code looks like this (interesting parts in bold):

package example;

import com.sun.jersey.api.container.httpserver.HttpServerFactory;
import com.sun.net.httpserver.HttpServer;

import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;
import javax.persistence.Query;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.ProduceMime;
import java.util.List;

@Path("/lith")
public class HelloWorld {

    private EntityManagerFactory emf = null;

    protected void createEMF() {
        emf = Persistence.createEntityManagerFactory("NewPeristenceUnit");
    }

    protected void closeEMF() throws Exception {
        emf.close();
    }

    public String doQuery(String latlong) {
        createEMF();
        StringBuffer sb = new StringBuffer();
        EntityManager em = emf.createEntityManager();
        Query query = em.createQuery("select c."+latlong+" from CmpEntity c");
        List<Double> list = query.getResultList();
        for (Double c : list) {
             sb.append(latlong + ": " + c.toString() + "\n");
        }
        try {
            closeEMF();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return sb.toString();
    }

    @GET
    @Path("location/{latlong}")
    @ProduceMime("text/plain")
    public String getClichedMessage(@PathParam("latlong") String latlong) {
        return doQuery(latlong);
    }

    public static void main(String[] args) throws Exception {

        HttpServer server = HttpServerFactory.create("http://localhost:9998/");
        server.start();

        System.out.println("Server running");
        System.out.println("Visit: http://localhost:9998/helloworld");
        System.out.println("Hit return to stop...");
        System.in.read();
        System.out.println("Stopping server");
        server.stop(0);
        System.out.println("Server stopped");
    }
}

A few things to note:

The List<Double> list = query.getResultList(); is not good(tm) as we have an unchecked assignment of java.util.List to java.util.List<java.lang.Double> Better would be to use something like List<CmpEntity> in this case and alter our query to something like Query query = em.createQuery("select c from CmpEntity c"); However, I ran into some issues with return in my entity class being null. Perhaps I needed to allow certain columns to be null, I am not certain of that, but I was more interested in the path than resolving the query. Likely I will resolve this issue as the next step in this experiment.
Parallel to the aboe the point my List<Double> work since I know that the return from the query is a Double
I altered things a bit in the above example so that I could use both /lith/location/longitude and /lith/location/latitude as calling URI's. This works of course since I know ahead of time what my column names were that I wanted to use for this test and that they were of type Double to address the above two points. This is whay I use the @Path and @PathParam annotations in the getClichedMessage method and strip out and pass along elements of the URI.
Like you would NOT want (ie DO NOT WANT) to use the column names in your mapped data source in your REST URI. You would establish your URI template to whatever degree you wished and then use a more logical approach to generating the structure and content of your services and their replies. This post is talking about the pipeline and is not worried about the very important architectural issue of mapping resources to URI's. Take a look at my friends Josh's post on that topic and the nice InfoQ How to GET a cup of Coffee posting about all that.

One last important point in all this. I got an error: No identifier specified for entity. When I looked at my generated entity classes all the columns recieved a @Basic and @Column(...) annotation. At least one however needs to have a @Id annotation. Reference the Hibernate FAQ at http://www.hibernate.org/329.html#A6 for more info. I had a unique integer column I could do this to and so I alterd my effected entity classes with @Id @Column(...) on one of the methods. You, however, should not see this as I suspect it was due to the fact I never botherd to set a unique key column in that table on my database (don't tell my DBA).

That's about it. At this point you should be able to run the project and make calls and get data out. The built in REST client for IntelliJIDEA 8 is quite nice though perhaps not as advanced as the nice rest-client project. In my case I parsed out the URI just for the sake of some fun. With all the elements connected one can move on to more interesting "real world" applications of the JSR-311 and JPA API's used in this simple demo.

Sproutcore on Ubuntu 8.04

2008-06-19T09:32:00.004-05:00

I was interested in the Sproutcore (ref: http://www.sproutcore.com) package that got some attention from the Apple developers conference. (AppleInsider article / MacBreak weekly podcast where Sproutcore is talked about)

Getting it to work on Ubuntu (verison 8.04 for me) took a few steps I thought I would place here. Of course first you need to make sure you have Ruby gems and Rails installed.

apt-get install rubygems rails

one items you may miss to install though is the dev package. Be sure to do

apt-get install ruby1.8-dev

This will show itself when Ruby trys to build native extensions with an error like:

extconf.rb:1:in `require': no such file to load -- mkmf (LoadError)

once you have this done you can do:

gem install sproutcore

and accept all its questions (it will take a while as several packages need to be installed)

At this point you will need to:

cd /var/lib/gems/1.8/gems/sproutcore-0.9.10/bin
chmod 755 *

to make these scripts executable and also add this location to your path with:

export PATH=/var/lib/gems/1.8/gems/sproutcore-0.9.10/bin:$PATH

or just add them to your shell init scripts like .bashrc

At this point you can start down the tutorial examples at: http://www.sproutcore.com/documentation/hello-world-tutorial/

take care
Doug

[update]
After talking with my friend Josh I thought I should add in a little note pointing to information related to Objective-J by 280North of 280Slides development as another data point in some of this Javascript talk.

5 Groovy / Grails recommendations

2008-06-06T11:27:00.001-05:00

So I was asked about what my top five recommendations would be for Groovy/Grails resources so I thought I would post them here to get feed back and other ideas from the community. I decided to break it up into five categories.

1) Web sites
Of course the top two would have to be the main Groovy and Grails sites themselves. Not just because they are the home of the respective projects but because both truly are good resources with plenty of examples, documentations and links to mailing lists and other resources. Others not associated with elements already to be placed in following categories might be the Spring and Hibernate sites.

2) Podcasts
Sven Haige started the Grails podcast and was recently joined by Glen Smith. They do a wonderful job with the podcast and it's a great resource for the community.
The Java Posse is high water mark in technical podcasts and I also enjoy Software Engineering Radio.

3) RSS feeds
There are a lot of blogs, rss feeds, etc. that a person can track down. The following three aggregate such feeds and provide a good starting point for locating feeds you resonate with:
http://groovyblogs.org http://groovy.dzone.com/ http://www.groovyongrails.com/

4) Software
Focusing on IDE's the first I would have to recommend is IntelliJ IDEA as its Grails/Groovy support is top notch. It's commercial, but well worth the cost and I believe the best of the those out there for Groovy Grails work period.

Following this I would (personally) would go withNetbeans 6.1 as it also now has Grails support. However, you need to use 6.1 development builds. The plugin home page is at: http://plugins.netbeans.org/PluginPortal/faces/PluginDetailPage.jsp?pluginid=6265

Eclipse also has support and in fact was the first to do so if I recall correctly. However, I have not heard much about it and suspect its support has not been a high priority. Many love Textmate (for good reason) and its various bundles and while on the non-IDE side of software one could likely do a lot with Jedit as well (I have).

Both Teamcity and Hudson provide build environments and code coverage (by way of plugins for Hudson) for Grails projects. Hudson requires a little work for setting up Grails builds but there are some nice blog posts on such efforts. Again the IntelliJ produced Teamcity shines in its Grails support right out of the box.

5) Books
Groovy in Action is a wonderful resource to have as is The Definitive Guide to Grails. There are several new Groovy books coming out and more likely to follow as the Groovy/Grails community continues to grow.

Would love to hear other ideas/comments.
Doug

Fkey template for IntelliJ

2008-05-20T12:35:00.001-05:00

I've come to really appreciate IntelliJ IDEA. It's a wonderful IDE and so glad I discovered it as a by product of working with Grails and Groovy. As I use it more and more I wanted to start to exploit more of its features and also to begin to get the key patterns to memory so I can work a bit faster. Failing to find an fkey overlay for IntelliJ I made one. I include a link to the SVG so anyone can modify/improve it as they wish. If you do improve it, please email me updates so I can include them here.

SVG Fkey template for IntelliJ IDEA (I would highly recommend the InkScape SVG editor, if you want to edit this document)

I am also linking to the document page with the official IntelliJ PDF key maps documents which list keyboard shortcuts. This is a nice document I think should be more prominently noted at their web site. You can add in your own keyboard short cuts and there is a blog posting by Ted Young that shows an example of how to do that.

I recently noticed the Shortcut Key List plugin ( http://plugins.intellij.net/plugin/?id=2391 ) which displays a simple box with short cuts in it.

Also there is Key Promoter (http://plugins.intellij.net/plugin/?id=1003 ) which can look for repetitive actions and ask if you want to assign a key sequence. I have started to fill in opening in the template above with some of the common things I do.

I'd also like to mention a new keyboard that I am totally taken with (as is my 3 month old son). It is based on the old IBM Model M keyboards. You can learn more about these keyboards at http://www.clickykeyboards.com/ which re-sells old Model M keyboards and also the Wikipedia entry (Ref: http://en.wikipedia.org/wiki/Model_M_Keyboard ). These are wonderful, if not a little noisy, keyboards that many of you may recall from your first experiences with computers.

A company called Unicomp bought the rights to the buckling spring technology and makes brand new USB based versions of the Model M style keyboard called the Customizer which is what I picked up and is pictured below. If you don't know what buckling spring keyboards are give them a look. In a time when keyboards are getting far too complex for their own good, I'll take a keyboard that just does one thing and does it very well.

take care
Doug

Using IUI in Grails

2008-04-28T12:16:00.001-05:00

So the first package I used in reviewing phone based UI's from Grails was CIUI. The other popular one that I found was IUI. This one seems a bit more advanced or at least feature rich than CIUI. I should point out it will be interesting to see how move advanced iPhone and HTML 5.0 features are going to play out in all this. That aside, implementing IUI was only marginally harder than CIUI. Again we download and place the CSS and JS files into the css and js directories located in [project]/web-app. In the case of IUI I did have to edit the CSS file to have it correctly locate the images folder (where I placed all the images that come with IUI). You could make a separate directory for these images as a sub-directory of images and alter the CSS file as you see fit in that case. Note, this package didn't render well for me outside of the webkit engine. I did a post on Webkit javascript performance on Ubuntu that also links to resources for compiling this package on Linux (Mac and Windows builds are available at the webkit site )

For the case of IUI I used the samples that came with the package as a guide to for all this. My GSP files looks like:

<%@ page contentType="text/html;charset=UTF-8" %> 
<html> 
<meta name="layout" content="iuiLayout"/> 
 
<head><title>IUI test</title> 
    <meta name="viewport" content="width=320; initial-scale=1.0; maximum-scale=1.0; user-scalable=0;"> 
</head> 
 
<body onclick="console.log('Hello', event.target);"> 
<div class="toolbar"> 
    <h1 id="pageTitle"></h1> 
    <a id="backButton" class="button" href="#"></a> 
    <a class="button" href="${createLink(controller: 'iui', action: 'search')}">Search</a> 
</div> 
 
<ul id="home" title="Home" selected="true"> 
    <li><a href="${createLink(controller: 'iui', action: 'category1')}">Category 1</a></li> 
    <li><a href="${createLink(controller: 'iui', action: 'category2')}">Category 2</a></li> 
    <li><a href="${createLink(controller: 'iui', action: 'stats')}">Stats</a></li> 
    <li><a href="${createLink(controller: 'iui', action: 'settings')}">Settings</a></li> 
    <li><a href="${createLink(controller: 'iui', action: 'about')}">About</a></li> 
 
</ul> 
</body> 
</html>

Here I just linked into my IuiController.groovy file to various methods to demonstrate some of the different features IUI has. The only interesting points in this are that it does demonstrate how to use the render method for cases like this where you need to include attributes and such. One interesting note was that the render method refused to provide a node named "form" in the XML return. I assume this is perhaps related to a keyword issue or some such. However, MarkupBuilder has no issues so where this does NOT work:

    // shows a form but doesn't work due to keyword issue with "form'? 
    def searchDoesNotWork = { 
        render(contentType: "text/xhtml") { 
            "form"(id: "searchForm", class: "dialog", action: "foo") { 
                fieldset { 
                    h1("Search the records") 
                    a(class: "button leftButton", type: "cancel", "Cancel") 
                    a(class: "button blueButton", type: "submit", "Search") 
                    label("First Name") 
                    input(id: "fname", type: "text", name: "fname") 
                    label("Last Name") 
                    input(id: "lname", type: "text", name: "lname") 
                } 
            } 
        } 
    }

this method will:

    def search = { 
        def mywriter = new StringWriter() 
        def xml = new MarkupBuilder(mywriter) 
 
        // had to do this since the render method seemed to not want to return form (key word?) 
        //  this is really how they should dbe done anyway... but I am too lazy to change all the 
        // examples 
 
        xml."form"(id: "searchForm", class: "dialog", action: "${createLink(controller: 'iui', action: 'doSearch')}") { 
            fieldset { 
                h1("Search the records") 
                a(class: "button leftButton", type: "cancel", "Cancel") 
                a(class: "button blueButton", type: "submit", "Search") 
                label("First Name") 
                input(id: "fname", type: "text", name: "fname") 
                label("Last Name") 
                input(id: "lname", type: "text", name: "lname") 
            } 
        } 
        render mywriter.toString() 
    }

In point of fact it's likely better form in most cases to use the MarkupBuilder as you can incorporate it into code logic easier. Though for simple cases there seems to be nothing wrong with render. As with the CIUI example I'll just post up the whole controller file for people to look at (IUI Controller ).

Note, I did modify the iui.js file line:

req.open(method || "GET", href, true);

to

req.open(method || "POST", href, true);

in order to get the search call to work to my controller. For some reason the URL encoding version would not send back the variables. I may contact the IUI people about this and see if I can resolve why this occured.

I was not able to wire up the toggle button to a controller method.. I got the error:

console message: http://localhost:8080/phoneUIGarden/iui#_Settings @35: Can't find variable: settingCallBack

Likely this is something simple (a guess) but I didn't bother to track it down since this was just a "for fun exploration" project.

It's easy to see how this could all be extended into a secured area of a site via Acegi or jsecurity and perhaps even attached to webflow . All fun stuff if I was paid to do that. This is by no means and in depth review, it's just my 4am feeding hacking to see what is involved in using IUI and CIUI. I thought I would share since I haven't seen much on using these in the Grails framework.

It's obvious that
A) it's simple
B) Groovy and Grails are a wonderful language/framework environment to do this in.
C) Someone with more time for this could flesh it all out rather quickly

enjoy...
Doug

Using CIUI in Grails

2008-04-28T12:14:00.001-05:00

So this past weekend I needed a fun yet mindless task one morning during at 4 am. I have been looking at both CIUI and IUI and knew they would lend themselves well to being implemented in Grails what with it's oh so easy REST/AJAX style call to the controller layer. In fact this was ever so easy and so I thought I would mention it here so people would know of these nice packages and add these as tools to their Grails toolbox. If you are doing this from Linux (Ubuntu in my case) like I am, you may want to use the webkit engine (Ref: Webkit on Ubuntu performance ). Especially for the IUI package since it seems to render best in Webkit even over the latest FF3 engine.

Starting with CIUI the approach is easy. Place the CiUI.css in your [project]/web-app/css and the CiUI.js into the [project]/web-app/js directories. You may need to scan the CSS file and change image paths and such if you are using images from an other than default location.

I created a simple controller CiuiController.groovy and an associated GSP page index.gsp. Into the GSP page be sure to place:

<link rel="stylesheet" href="${createLinkTo(dir:'css',file:'CiUI.css')}" />
<g:javascript library="CiUI"/>

to load the CiUI css and js files. Obviously you can alter the directory paths and such to keep your application file structure in whatever order you wish.

My whole GSP pages looks like:

<html><head> 
    <title>Grails and CiUI Demo</title> 
    <meta name="viewport" content="width=320"/> 
    <meta name="viewport" content="initial-scale=1.0, user-scalable=no"/> 
    <link rel="stylesheet" href="${createLinkTo(dir: 'css', file: 'CiUI.css')}"/> 
    <g:javascript library="CiUI"/> 
</head> 
 
<body> 
<div id="iphone_header"> 
    <div id="iphone_backbutton"> 
        <img src="images/back-button-tip.png" border="0" align="left"/> 
        <a id="iphone_backbutton_text" href="#" class="go_back">Back</a> 
    </div> 
    <div id="iphone_title"></div> 
</div> 
 
<div id="iphone_body" style="clear:both;"> 
    <ul class="menu"> 
        <li><a href="${createLink(controller: 'ciui', action: 'page', id: '1')}" class="go_forward" title="Feeds"> 
            Groovy and Grails Feeds 
        </a> 
        </li> 
    </ul> 
</div> 
 
<div id="iphone_footer">My footer for CiUI demo</div> 
 
<div id="iphone_loading_page"> 
    <div id='loading' class="info_msg"> 
        <img src="images/loading.gif"/><br/> 
        loading... 
</div> 
</div> 
</body> 
</html>

Next it's just a few simple method calls in the controller. I decided to have a little fun and used the Groovy XML features to pull down RSS feeds from Groovy Blogs, Groovy on Grails and Groovy Zone. There are really only a couple of comments:

1) This whole effort is made very easy by use of the Grails "render" call. It means that we can do things like:

   def page = { 
 
        //  you would make this more dynamic in a real app 
        def feedMap = ['Groovy Blogs': 'http://feeds.feedburner.com/groovyblogs', 'Grroovy on Grails': 'http://feeds.feedburner.com/groovyongrails', 'Groovy Zone': 'http://feeds.dzone.com/zones/groovy'] 
 
        render(contentType: "text/xhtml") { 
 
            //  If there are some menu items to show..  show them 
            ul(class: "menu") { 
                for (i in feedMap.keySet()) { 
                    li { 
                        a(class: "go_forward", title: "${i}", href: "${createLink(controller: 'ciui', action: 'getFeed')}/?url=" + feedMap[i]) { 
                            div("${i}") 
                        } 
                    } 
                } 
            } 
        } 
 
    }

Check out more on the render method at http://grails.org/Controller+Dynamic+Methods. For sake of a short blog entry I have just placed the whole controller file on-line (Ref: CIUI Controller ) It's all just a simple zero'th order pass at using CiUI. There are many extensions that one could do with this (more with templates, webflow, etc.). I didn't implement the search interface or anything fancy here as this is just a review to see what is involved (practically nothing) in using CIUI in Grails. I will likely go back and add something like that in and see if I can call back with parameters to my controller.

Note however that when generating the mark up in the controller it's likely a bit more "proper" to do something like the following MarkupBuilder example. This is actually an example from the IUI test I did and not used in this CIUI example, however the principle is the same.

    def search = { 
        def mywriter = new StringWriter() 
        def xml = new MarkupBuilder(mywriter) 
 
        xml."form"(id: "searchForm", class: "dialog", action: "${createLink(controller: 'iui', action: 'doSearch')}") { 
            fieldset { 
                h1("Search the records") 
                a(class: "button leftButton", type: "cancel", "Cancel") 
                a(class: "button blueButton", type: "submit", "Search") 
                label("First Name") 
                input(id: "fname", type: "text", name: "fname") 
                label("Last Name") 
                input(id: "lname", type: "text", name: "lname") 
            } 
        } 
        render mywriter.toString() 
    }

Here we used the Groovy MarkupBuilder to generate our XML outside the render method and then just pass it along. This means we can push this logic down into a service or at least have a bit better logic to our controller code designs.

This youTube video shows what it looks like in action

Implementing these JS/CSS packages inside Grails is trivial. As the iPhone and Google Android phones become more and more a mechanism for people to obtain information from the web the ability for developers easy represent data in appropriate formats will be more important. Grails is a wonderfull platform for addressing that.

enjoy
Doug

xrandr with Ubuntu 8.04 and external monitors on laptops

2008-04-25T10:25:00.005-05:00

So I installed Ubuntu 8.04 onto my Dell Inspiron 700m. I was impressed with everything it is doing well right out of the box (display, power management, wireless, etc.).

However, one item I was really looking forward to was xrandr and the ability to dynamically plug in external monitors or projection units. In trying this I failed to get my external monitor to dynamically configure via xrandr. However, the solution to this was simple.

The default xorg.conf was:

Section "Screen"
Identifier "Default Screen"
Monitor "Configured Monitor"
Device "Configured Video Device"
EndSection

I modified this by adding in the subsection display:

Section "Screen"
Identifier "Default Screen"
Monitor "Configured Monitor"
Device "Configured Video Device"
SubSection "Display"
Virtual 2560 1024
EndSubSection
EndSection

Next restart X (logout and in, or cntrl-alt backsapce). Note that for me the entry was 2560 1024 since my internal display is 1280x800 and the external is 1280x1024 so 1280 + 1280 = 2560 and 1024 > 800 so use 1024. You will need to do the math for your case.

At this point the "Screen Resolution" application works correctly to set up the displays. Readers may also be interested in the article at http://www.thinkwiki.org/wiki/Sample_Fn-F7_script which shows a nice shell script that can be connected to key combinations for all this.

take care
Doug

UPDATE April 28th:

Just a follow up to this posting. It turns out that in several cases (and low end laptop graphics like mine among them) that doing this will

disable DVI and thus not allow you to run compiz

For me nice graphics vs dual monitor is a no contest. I'll take the dynamic xrandr over compiz any day.

Sorting multidimensional arrays in Groovy

2008-04-22T08:41:00.001-05:00

I didn't find this solution as quickly as I thought I would so I thought I would post about it here to provide some search engine fodder.

If you are dealing with multidimensional arrays in Groovy (ref: http://groovy.codehaus.org/JN1025-Arrays ) and want to sort them you can expand on the sorting concepts for collections in Groovy (ref: http://groovy.codehaus.org/JN1015-Collections ).

If you look in the last reference you will find the example:

def mc= [
  compare: {a,b-> a.equals(b)? 0: Math.abs(a)<Math.abs(b)? -1: 1 }
] as Comparator

We can modify this for a multidimensional array. So some excerpts from my code:

println "Pre sort: ${uniqueAges}"
def mc = [
   compare: {a, b -> a[0].equals(b[0]) ? 0 : a[0] < b[0] ? -1 : 1 }
] as Comparator
println "Post sort: ${uniqueAges.sort(mc)}"

results in

Pre sort: [[11.2, 5.32], [1.77, 0.01], [3.58, 1.77], [5.32, 1.77], [5.32, 3.58]]
Post sort: [[1.77, 0.01], [3.58, 1.77], [5.32, 1.77], [5.32, 3.58], [11.2, 5.32]]

 println "Pre sort: ${ageZoneList}"
 def mc2 = [
    compare: {a, b -> a[0][0].equals(b[0][0]) ? 0 : a[0][0] < b[0][0] ? -1 : 1 }
 ] as Comparator
 println "Post sort: ${ageZoneList.sort(mc2)}"

results in

Pre sort: [[[5.32, 11.2], [163.5, 189.62, "Late Miocene"]], [[0.01, 1.77], [0.05, 105.58, "Pleistocene"]], [[1.77, 3.58], [114.82, 123.68, "Late Pliocene"]], [[1.77, 5.32], [103.13, 163.53, "Pliocene"]], [[3.58, 5.32], [137.16, 156.19, "Early Pliocene"]]]
Post sort: [[[0.01, 1.77], [0.05, 105.58, "Pleistocene"]], [[1.77, 3.58], [114.82, 123.68, "Late Pliocene"]], [[1.77, 5.32], [103.13, 163.53, "Pliocene"]], [[3.58, 5.32], [137.16, 156.19, "Early Pliocene"]], [[5.32, 11.2], [163.5, 189.62, "Late Miocene"]]]

Note that in Groovy multidimensional arrays need not have the same exact length at each level as noted in the Array reference above. This does mean that you need to take care with respect to iterating your indexes if your array is triangular. This is not likely the most efficient search for a large multidimensional array, but for simple sorts on smaller examples its working well for me. I would appreciate references for quicker sorts if people have them.

take care
Doug

Spaces in REST URLs

2008-04-21T11:06:00.001-05:00

So I need to deal with spaces in REST URL's. Something like:

.../timescale/Late Jurassic

I'd rather not allow these types of URL's (since I don't think the spec even allows them) and was thinking about the programming and also community practices issues this raises. If a person wants a space they likely should use "%20", however this just doesn't look good and I believe gets confusing.

I could simply allow the community to create such URL's even though I believe them wrong and then attempt to resolve them in my Grails application with:

someString.replaceAll("%20", " ")

since firefox and I suspect other browsers will convert the spaces to "%20" for the user.

This seems a poor approach though since if someone puts two spaces into the URL or somehow a tab gets in things get ugly fast. A person just has far too many bad issues to deal with.

Forcing the community to use camel case (here I will use lower camel case but one could use upper camel case) would require a URL like:

.../timescale/lateJurasic

where I could split the parameter in Groovy with:

someString.replaceAll("([A-Z])", " $1").trim()

so "lateJurassic" becomes "late Jurassic" which is what is in my database (actually "Late Jurassic" but I can case insensitive search) since that is the "proper" form of the name.

This puts the burden of creating a camel case representation of "Late Jurassic" on the client end of the application (ie, make this someone else's problem, which is always a good thing).

I suspect this is just a case where the best resource identifier in the database is not always the best resource identifier for the REST URL representation. It would seem then that apply a best practice approach one would use a camel case representation. This might be a situation where using upper camel case is better yet since the proper cases on the name is "Late Jurassic" resulting in a URL:

.../timescale/LateJurassic

That is still fine since the .trim() call will remove the leading space still in the above code. I'd love to hear other thoughts on such mappings.

take care
Doug

Javascript comparison of WebKit and Firefox 3 Beta 5 on Ubuntu

2008-04-14T09:19:00.001-05:00

This weekend I had a small pet project I did that gave me a need for a webkit based browser on Linux (Ubuntu 7.04). While I could have used Konqueror I was more interested in a later source base and so downloaded the WebKit r31738 source from http://www.webkit.org. Actually, after a couple simple apt-get install for the development library this source package will compile just fine on a rather stock Ubuntu 7.04 and runs well (Ref: http://live.gnome.org/WebKitGtk ). The default UI shell for the webkit core is ultra basic but all I need for testing against this rendering engine. One can use webkit as the engine for Epiphany (http://live.gnome.org/Epiphany/WebKit ) if you want a nicer wrapper.

I was curious to try this new engine (more specifically it's javascript aspect) against my current favorite browser Firefox in the form of Firefox 3 Beta 5 (http://www.mozilla.com/en-US/firefox/all-beta.html ). A tried both the Dromaeo test suite (still in early release, review notes at: http://wiki.mozilla.org/Dromaeo ) and Sunspider from: http://webkit.org/perf/sunspider-0.9/sunspider.html. My platform is rather paltry Athlon 64 3200+ and 2 gigs of memory running the afore mentioned Ubuntu 7.04.

Firefox 3 Beta 5:
http://dromaeo.com/?id=5552 (3282.80ms)
http://dromaeo.com/?id=5560 (3287.60ms)
SunSpider: 6794.0ms

WebKit
http://dromaeo.com/?id=5556 (3260.60ms)
http://dromaeo.com/?id=5563 (3225.40ms)
SunSpider: 5794.8ms

In the Dromaeo test there is for all practical purposes no difference between the two javascript engines. Sunspider showed a 1.17 factor favor toward the Webkit engine. There has been a lot of recent new traffic over acid3 testing and other performance metrics (http://ajaxian.com/archives/where-is-firefox-on-acid-3-here ). The Firefox ecosystem around its amazing set of add-ones (https://addons.mozilla.org/en-US/firefox/ ) is without question a key factor for me. The Mozilla foundations strong effort on standards, highly open and extensible platform win over a few percentage points from a Javascript benchmark. The acid3's focus on DOM scripting is important to me and so I would like to see that arrive in the Gecko engine though some of the comments of Mike Shaver (http://shaver.off.net/diary/2008/03/27/the-missed-opportunity-of-acid-3/ ) and Rob Sayre (http://blog.mozilla.com/rob-sayre/2008/03/26/acid3-is-basically-worthless/ ) regarding this do make sense to me.

I'm a loyal Firefox user and nothing I have seen from the Webkit camp has yet to give me any pause, though I am glad they are there. We need competition and pressure to drive development forward. It's always good to compare and I am still in a situation where as a developer I need to have various engines around for testing. I also played around with the new Next-Generation Java Plug-in (https://jdk6.dev.java.net/plugin2/ ) and was very impressed with it as well. Especially the easy javascript to applet communication the new plugin delivers.

take care
Doug

Grails datasource in resource.xml

2008-03-26T14:09:00.001-05:00

So I have the need for a couple extra datasources in my grails application. This turned out to take a bit more research than I first thought it would, though likely just due to my lack of much heavy duty spring experience so far. Basically I just want access to another database resource that is NOT my main datasource used by the hibernate ORM. My domain classes for this application are separate from my need to access this other datasource which I would do just via SQL calls.

A quick review of the Nabble list gave me a couple of leads:
http://www.nabble.com/dataSource-on-ApplicationBootStrap-td11441836.html#a11441836
http://www.nabble.com/multiply-datasources--td10038844.html#a10038844

I created the following entry in resource.xml located in GRAILS_APP/web-app/WEB-INF/spring directory. Create this path and file if it doesn't already exist. The full XML document is like:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">

  <bean id="dataSourceJanus" class="org.springframework.jdbc.datasource.SingleConnectionDataSource">
        <property name="driverClassName" value="oracle.jdbc.driver.OracleDriver"/>
        <property name="url" value="'jdbc:oracle:thin:@xxx.xxx.xxx.xxx:1521:janware"/>
        <property name="username" value="NAME"/>
        <property name="password" value="PASSWORD"/>
  </bean>
</beans>

though you may only need the bean entry if the file already exists of course. Thanks Alex (again) for finding my insanely stupid typo in this file. I stared for hours (sad to say) at it.

The only tricky part is that you can not directly reference the new datasource ( here called dataSourceJanus in my example) in say your controller class. Rather you will need to implement ApplicationContextAware. So you will have something like:

//  Needed for datasource 
import org.springframework.jdbc.core.JdbcTemplate
import groovy.sql.Sql


// Spring
import org.springframework.beans.BeansException
import org.springframework.context.ApplicationContext
import org.springframework.context.ApplicationContextAware
import org.codehaus.groovy.grails.commons.GrailsApplication

class FacetController implements ApplicationContextAware {
    
    GrailsApplication grailsApplication
    ApplicationContext appCtx

    def void setApplicationContext(ApplicationContext arg0) throws BeansException {
        appCtx = arg0;
    }

...

This will get you your application context object. Now it's rahter straight forward to use:

def showFacets = {
        def jt = new JdbcTemplate(appCtx.dataSourceJanus)
        def sqlString = "select latitude_degrees, longitude_degrees from hole where leg like '101' and site like '631' and hole like 'A'"
        response.contentType = "text/plain"
        jt.queryForList(sqlString).each {        
            response.outputStream << "${it.latitude_degrees} ${it.longitude_degrees}"
        }
}

That is just a dead simple method to call the datasource (bold) and writes directly to the output stream. Not the org.springframework.jdbc.core.JdbcTemplate import in the previous code sample along with the other imports used for the application context. Obviously you will likely return to a view or render a JSON serialization or some such in your method rather than write directly to the outputStream.

If you place def jt = new JdbcTemplate(appCtx.dataSourceJanus) outside the scope of the method you get a null reference at start up. So it seems this should be scoped inside the method that uses it. However, this logic might be better moved into a service class anyway depending on how much you will use it.

Some additional good reading on this includes:
Datasources: http://docs.codehaus.org/display/GRAILS/DataSources+New
Using JNDI resources instead: http://docs.codehaus.org/display/GRAILS/JNDI+Data+Sources
Spring Bean Builder: http://grails.org/Spring+Bean+Builder
Application Context: http://grails.org/Services

If you review and follow some of the thread in the Nabble site, you will see that seems one could in fact use dynamic methods with this new datasource if you do a few extra steps. I have not done this but it appears to involve making some entries in the hibernate.cfg.xml file. Reference http://www.nabble.com/How-to-get-multi-dataSource-in-grails--td12291409.html#a12291409 for some comments I found on trying to do this. There does seem to be some issues I am hearing about having/using dynamic methods on multiple datasource though, so I am not sure how this all plays out.

For my needs however, just having an SQL resource that I could use the very nice Groovy SQL capacity with.

enjoy
Doug

Flot plotting in Grails:

2007-12-06T12:04:00.000-06:00

I came across the FLOT package (Ref: http://code.google.com/p/flot/ ) while reading Ajaxian (Ref: http://ajaxian.com/archives/plotting-in-jquery ). Flot uses the jQuerry package (Ref: http://jquery.com/ ) to do client side plotting. I wanted to see how this works when dropped into Grails. As expected the results are quite nice.

The set up is rather easy. Obtain the jquery.flot-0.1.js and jquery-1.2.1.min.js files (your versions will obviously change with the march of time) and place these in your web-app/js directory of your grails project.
You will need to reference these in your GSP page HEAD tag with something like:

  <g:javascript library="jquery-1.2.1.min"/>
<g:javascript library="jquery.flot-0.1"/>

At this point your are basically done. You can review the URL's at the top of this page for better information straight from the source. Basically, you will need to place into the BODY tag of your GSP something like the following:

 <div id="placeholder2" style="margin-left:100px;width:600px;height:300px;"></div>

<script id="source2" language="javascript" type="text/javascript">
 $(function () {
   var d1 = [];
   for (var i = 0; i < 14; i += 0.5)
   d1.push([i, Math.sin(i)]);

   var d2 = ${array};

   $.plot($("#placeholder2"), [ d1, d2 ]);
   });
</script>

Here is the only interesting aspect of all this. The line var d2 = ${array} is referencing a list from the controller:

  def jflotPlot = {
      def array = [[4, 6], [5, 7], [9, 2]]
     [array: array]
}

Likely your controller will get more interesting data. ;) This is just a really quick and dirty copy paste job of the examples from the above references into a Grails app. Flot can already at version 0.1 do more types and plots and zooming and other very slick features (see the demos at the flot site ). Integration with Grails is quick and easy.

enjoy
Doug

Grails + ROME + geoRSS

2007-11-14T10:39:00.000-06:00

Grails + ROME + geoRSS

If you are looking to do some RSS/ATOM feeds in Grails you likely are looking at either at the examples that Glen Smith has done (ref: http://blogs.bytecode.com.au/glen/2006/12/22/1166781151213.html ) or to work with the Feeds plugins (ref: http://grails.org/Feeds+Plugin ) by Marc Palmer. Both are excellent places to start.

What I want to talk about there is adding geospatial data to a RSS feed in the Grails framework by building off this work. The above efforts leverage off the ROME package from http://rome.dev.java.net. For geospatial elements there is the GeoRSS Module for ROME by Marc Wick (ref: http://georss.geonames.org ).

Adding this ability to a Grails project is rather easy. Obviously you need to have the Rome and GeoRSS jar files from these referenced sites in your project lib directory. From there you can do something like the following. Please note these examples are based on the work of these previously mentioned individuals and the credit is theirs. I just glued things together.

So likely you will end up with something like this in your controller:

  // some import for this..   (watch me polute my namespace) ;)
import com.sun.syndication.feed.synd.*;
import com.sun.syndication.io.SyndFeedOutput;
import com.sun.syndication.feed.module.georss.*;
import com.sun.syndication.feed.module.georss.geometries.*;


    def supportedFormats = ["rss_0.90", "rss_0.91", "rss_0.92", "rss_0.93", "rss_0.94", "rss_1.0", "rss_2.0", "atom_0.3"]

    def rss = {
        render(text: getFeed("atom_0.3"), contentType: "text/xml", encoding: "UTF-8")
    }

    def all = {
        def format = params.id
        if (supportedFormats.contains(format)) {
            render(text: getFeed(format), contentType: "text/xml", encoding: "UTF-8")
        } else {
            response.sendError(response.SC_FORBIDDEN);
        }
    }

    def getFeed(feedType) {

        def ageModels = AgeModel.list()
        def jdb = Sql.newInstance('[ WE ARE MAKING SQL CALLS SO THE CONNECT STRING GOES HERE.  GORM WOULD BE NICER, BUT NOT AN OPTION FOR ME')
      
        def entries = []
        ageModels.each {ageModel ->
           List functionData = jdb.rows("select latitude_degrees, longitude_degrees from hole where leg like ${ageModel.leg} and site like ${ageModel.site} and hole like '${ageModel.hole}'")
           def locationString = functionData[0][1] + ", " + functionData[0][0]

            def desc = new SyndContentImpl(type: "text/plain", value: "Entry for leg " + ageModel.leg + " locate at " + locationString + ageModel.rating + " " + ageModel.status);
            def entry = new SyndEntryImpl(title: ageModel.leg + "_" +ageModel.site + ageModel.hole + " - " + ageModel.user,
                    link: 'http://localhost:8080/janusAmp/rest/lsh/' + ageModel.leg + "/" + ageModel.site+ "/"+ageModel.hole,
                    publishedDate: ageModel.date, description: desc);

             // Select the style of encoding you want to do.
            // def geoRSSModule = new W3CGeoModuleImpl()
            // def geoRSSModule = new GMLModuleImpl()
            def geoRSSModule = new SimpleModuleImpl()
            geoRSSModule.setPosition(new Position(functionData[0][0],  functionData[0][1]));
            entry.getModules().add(geoRSSModule);
            entries.add(entry);
         }

        SyndFeed feed = new SyndFeedImpl(feedType: feedType, title: 'Recently added age models',
                link: 'http://www.chronos.org', description: 'List of age models from Janus data',
                entries: entries);

        StringWriter writer = new StringWriter();
        SyndFeedOutput output = new SyndFeedOutput();
        output.output(feed, writer);
        writer.close();

        return writer.toString();
    }

The only thing different than what Glen, shows in his blog is the addition of the bold lines to add in the geoRss element. The result... geospatially enabled RSS feeds in the Grails framework (and anywhere else for Groovy and Java of course)

A few things to note:
a) While Google Earth and Google Maps should read this output I had some issues with the resulting files. Running things through the validator at: http://cite.opengeospatial.org/test_engine/georss_validator/ one gets some errors related to the namespaces. I noticed that the examples for RSS feeds at http://georss.org/ even had some issues. While the ATOM version validates ok the RSS based feed errors out in the same way that my generated feeds do. Likely this is an issue with ROME and if I track down the issue I will follow up on it. Until the feeds validate these feeds do not appear to work in Google Earth or Maps.
b) However, you can generate the rss version 2 feed with the SimpleModuleImpl() from the georss jar file and that will translate correctly with the rss to kml style sheet found at http://www.kovacevic.nl/hacks/kml/georss2kml.xsl (ref: http://www.kovacevic.nl/blog ).

RSS to KML via XSL in Grails:

So you can drop into your controller something like:

  def kml = {
        def xslPath = getAppPathService.serviceMethod().toString() + "georss2kml.xsl"
        def kmlxsl = new File(xslPath).getText().toString()
        def rssxml = getFeed("rss_2.0")

        def mywriter = new StringWriter()

        def factory = TransformerFactory.newInstance()
        def transformer = factory.newTransformer(new StreamSource(new StringReader(kmlxsl)))
        transformer.transform(new StreamSource(new StringReader(rssxml)), new StreamResult(mywriter))

        render(text: mywriter.toString(), contentType: "text/xml", encoding: "UTF-8")
    }

you make also need the following in your code for XSL support:

//  for the XSLT transform
import javax.xml.transform.TransformerFactory
import javax.xml.transform.stream.StreamResult
import javax.xml.transform.stream.StreamSource

into your controller where AppPathService is (not mine.. but I don't have the reference for this.. bad me)

 import org.codehaus.groovy.grails.commons.*
import org.apache.commons.logging.*

import org.springframework.beans.BeansException
import org.springframework.context.ApplicationContext
import org.springframework.context.ApplicationContextAware

class GetAppPathService implements ApplicationContextAware {

   GrailsApplication grailsApplication
    ApplicationContext appCtx

    boolean transactional = true

    def void setApplicationContext(ApplicationContext arg0) throws BeansException {
        appCtx = arg0;
    }
    def serviceMethod() {
        String applicationPath = "${appCtx?.getServletContext()?.getRealPath("/")}"
        return applicationPath
    }
}

What the KML does is load in the XSL file (it's looking at the root of the response which will be your web-apps directory. You can move and alter this as you wish. Of course you could simple place the XSL in the controller too like at http://groovy.codehaus.org/Processing+XML+with+XSLT which is what kml method is based on. I like this this way a bit better. At this point you should be generating KML from this and Google Earth and Google Maps should load and monitor this URL just fine.

I do want the get the GeoRSS feed vaildating though and will work on that effort as noted.

enjoy
Doug

Checking REST and RSS style feeds

2007-11-12T14:16:00.001-06:00

Continuing to do work with REST in Grails as well as RSS feeds. One issue is testing some of these URL's. In my last post I spoke about a groovy client using Apache's http client package which is quite nice for doing testing with. I also ran across this Firefox extension (RestTest) which is quite handy.

I also found this to a nice tool for testing RSS feeds with since it's not hard to get into a situation these days where your browser will either try and show you the RSS styled up or even redirect you to Google Reader or some other RSS reader. You can put the RSS/ATOM URL into this extensions interface and simply retrieve the XML of your feed for inspection during coding.

On the topic of RSS/ATOM and Grails check out the Feeds Plugin at the Grails site.

Groovy REST client with Apache httpclient (setting accept request-header value)

2007-10-23T14:12:00.000-05:00

Been doing quite a bit with REST style services in Grails. In particular trying to follow closes REST best practices approaches. One of these approaches is to inspect the HTTP Accept header values. (Ref: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html).

While creating these services a person really needs a simple client that they can alter the accept request-header field with. I created one using the Apache httpclient package (http://jakarta.apache.org/httpcomponents/httpclient-3.x/).

Simply download and place the commons-httpclient-*.jar file into your groovy lib directory (or somewhere you class loader will find it) and generate a Groovy program like:

import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.UsernamePasswordCredentials;
import org.apache.commons.httpclient.methods.GetMethod;

def url = "URL TO SERVICE"
client = new HttpClient()
get = new GetMethod(url)
get.setRequestHeader("Accept", "text/xml")
client.executeMethod(get)

println get.getResponseBodyAsString().toString()

It's a simple client and one can alter the ACCEPT header value easily to check and validate REST services response to various settings of accept request-header.

Using CHRONOS Timescale Converter Service Calls

2007-09-17T21:42:00.000-05:00

The timescale converter web application accessible through the CHRONOS portal (Ref: http://portal.chronos.org/gridsphere/gridsphere?cid=tools_tsconvert) uses publicly available web service calls to the timescale service described by the WSDL document located at http://services.chronos.org:9090/axis/timescales.jws?wsdl. These services are available for use on any platform.

Simple clients to these end points can be created in any modern programming language. Examples using Python and Groovy, two popular dynamic languages, are given below. Examples using Ruby or C# would be quite similar.

Example using Python

#!/usr/bin/python
#
import re, string
from SOAPpy import SOAPProxy

server = SOAPProxy("http://services.chronos.org:9090/axis/timescales.jws")
time = 20.0
while time < 40.0:
result = server.convertTime("GTS 2004", "Berggren 95",time)
print time , result
time = time + 0.1

Example using Groovy

import groovy.net.soap.SoapClient

def proxy = new SoapClient("http://services.chronos.org:9090/axis/timescales.jws?wsdl")

def serviceClosure = {
time -> return proxy.convertTime("GTS 2004", "Berggren 95", time)
}

for (float f = 20;f< 40.0; f=f+0.1) {
println serviceClosure(f)
}

A slightly more advanced example of a real-world application of these services is the implementation of the service in a Java-based web application server. In this case, a user would use e.g. the Xfire (http://xfire.codehaus.org/Client+and+Server+Stub+Generation+from+WSDL) or Apache Axis (http://ws.apache.org/axis/java/client-side-axis.html) packages to generate a library from the WSDL file. The resulting library could then be easily called from various locations in the application with only a few lines of code.

The first step would involve the creation of the stub classes in a WSDL-to-Java process. We will use the Apache Axis package in this example but the process is similar for Xfire or C# style environments. The initial stub classes are generated through a call like:

java -cp axis.jar:commons-logging-1.0.4.jar:commons-discovery-0.2.jar:axis-ant.jar:log4j-1.2.8.jar:wsdl4j-1.5.1.jar:jaxrpc.jar:saaj.jar org.apache.axis.wsdl.WSDL2Java -o . -d Session -p org.chronos.ws http://services.chronos.org:9090/axis/timescales.jws?wsdl

The result of this call is a set of Java source files that would then be compiled:

javac -classpath axis.jar:commons-logging-1.0.4.jar:commons-discovery-0.2.jar:axis-ant.jar:log4j-1.2.8.jar:wsdl4j-1.5.1.jar:jaxrpc.jar:saaj.jar org/chronos/ws/*.java

and the resulting class files collected into a jar file:

jar -cvf timescale.jar org/chronos/ws/*.class

The resulting jar file can then be used to greatly simplify the creation of clients in Java or any other Java byte code compatible language like Jruby, Jpython or Groovy.

An example of Java client that gets the color scheme for the Geological Time Scale is:

public class wsClient {
public static void main(String [] args) throws Exception {

// Make a service
org.chronos.ws.TimescalesService service = new org.chronos.ws.TimescalesServiceLocator();

// Now use the service to get a stub to the service org.chronos.ws.Timescales_PortType ts = service.gettimescales();

// Make the actual call
System.out.println("call " + ts.getColorScales());
}
}

A Groovy client that uses the jar file (here using the batch convert method for time conversion) is:

import java.text.DecimalFormat

// Make a service
def org.chronos.ws.TimescalesService service = new org.chronos.ws.TimescalesServiceLocator();

// Now use the service to get a stub to the service
def org.chronos.ws.Timescales_PortType ts = service.gettimescales();

// Make the actual call
def batchResults = (ts.convertTimeBatch("GTS 2004", "Berggren 95", 0.toDouble(), 60.toDouble(), 1.toDouble()));

DecimalFormat df2 = new DecimalFormat( "#,###,###,##0.000" );

// Closure for 2 place formating
def decf2 = {
value -> return new Double(df2.format(value)).doubleValue();
}

for (item in batchResults) {
println decf2(item)
}

These examples illustrate the process involved in the creation of clients using only the WSDL URL. Once created, jar libraries like those at the end of the process can be dropped into application server class paths and used in frameworks like Grails (http://www.grails.org), Seam ( http://www.jboss.com/products/seam) or JSR-168 portal environments like Gridsphere (http://www.gridsphere.org). Any application or tool with network access can invoke these services in a similar manner.

All CHRONOS services a similar pattern and can be utilized in web-based or stand-alone clients that have access to the network.

Grails SQL date formating lib

2007-09-04T13:57:00.000-05:00

Been working with Grails more and created a little tag lib I thought I would post up for others. The date that comes out of the domains is a rather ugly SQL date style so I create a tag lib with the following code:

 def formatSqlDate = { attrs -> 
    def String startdatetime =  "${attrs['targetDate']}"
    def DateFormat odf = new SimpleDateFormat("yyyy-MM-dd hh:mm:ss.S");
    def DateFormat df = new SimpleDateFormat( "MMM d, ''yy" );
    out << df.format(odf.parse(startdatetime))     
 }

Then I can simple reference it with:

<g:formatsqldate targetdate="${domain.dateObject}">

A person can mode the date format string all they want and also could put in various cases or a flag to select a format style if they wished.

Terminal shell in eclipse

2007-07-27T10:06:00.000-05:00

The state of working with Grails in Eclipse is not the best for me. Using the external tool call has been a pain and I have been using an external terminal shell. Today I went looking for a terminal shell solution built into Eclipse and found the Target Management Project.

You can install the components for this via the Eclipse update manager with the information from: http://europa-mirror1.eclipse.org/dsdp/tm/updates/.

Once this is done use the menu sequence Window -> Show_View -> Other and select the Remote Systems folder. From there I added "Remote System Details" (not really needed) and "Remote Shell". Using the triangle menu for this latter element I selected the default local connection.

After all this though.. failure.. it exec's each command out to the shell and as such there is no way to re-attach and kill a process easily. If you run grails run-app it's disconnected to one has to ps its PID and kill it. (not optimal).

After some looking I found sshView (http://www.eclipse-plugins.info/eclipse/plugin_details.jsp?id=1187). This installed but gave some very strange behavior and never was able to get it to successfully work. You can access it in the same sequence as above and look for sshview.

In the end there is easyShell. However, all this does is launch the shell I was using when I started this journey. Yes, you can configure it to open up in the directory the file is in and you can get to rather easily through the contextual menu. In the end not really worth the time and effort and still no good built into the IDE view shell solutions that I can find at least.

Using Grails GSP to provide alternating colors in a table

2007-07-19T15:44:00.000-05:00

Using Grails GSP to provide alternating colors in a table

I saw something related to this a while ago on the net but was not able to find it again. So I just came up with my own approach.

I wanted a table that had alternating rows highlighted. There are many ways to approach this. This solution is what I am going to use since it all falls on the view side of things and as such is rather easy to code. It is based on the code examples from http://grails.org/GSP+Tag+-+set

<table width="50%" align="right" border="0" cellpaddinng="0" cellspacing="0">
<g:def var="counter" value="${1}" />
<g:each in="${lastElement}">
<tr>
<td style="background:${counter % 2 == 0 ? 'white' : 'grey'">
<g:showElementResults id='${it.id}'/>
</td>
</tr>
<g:set var="counter" value="${counter + 1}" />
</g:each>
</table>

It is the "${counter % 2 == 0 ? 'white' : 'grey'}" that does all the work. The only other interesting elements are in bold. Simply change the two colors to get the banding effect you want. A more advanced version of this could set a style name or some other CSS element to improve the approach. I am sure there are several ways to do this. This seems to work for me. The "<g:showElementResults id='${it.id}'/>" code is not relevant here. It's just a taglib I use to create the formatted text of the cell based on an id and the rest is the just the <g:each> tag boilerplate.

Keyboard shortcut for grails external tool in Eclipse

2007-07-12T10:20:00.000-05:00

Somehow I think this should have been much easier to do. Since it didn't seem to be I am placing it here.

Using Grails in Eclipse (IDE integration information here), I wanted to make a shortcut for the external tools command which is used so much in this set up.

To do this you need to go to preferences (under the window menu), and select Keys -> Modify (Tab) -> Run/Debug (Category pull down) -> Run Last Launched External Tool (Name pull down) then assign the key in the key sequence and be sure to click the "Add" button.

Once all that is done you have a keyboard shortcut for the external tool.

(Please.. I'd like to see netbeans and grails integration)

Perm gen size and Grails

2007-06-26T10:14:00.000-05:00

I've been continuing to work with the Grails framework and found an interesting issue. When working with the Tomcat application server I run into several issues related to out of memory errors with Grails applications.

Attempts to resolve this with -Xmx512m or some other setting for the memory heap size failed to work. This seems to be due to the large number of elements (controllers, domains, etc) in my grails app that load classes. I believe this is related to issues of lots of use of reflection (not sure). It does appear that this memory is alive during the entire life of the application (not garbage collected). After installing several plug-ins and increasing the size of the application quite a bit this occurred. It also seems to be partly related to the creation of an additional "Context Path" for an application in Tomcat.

I found some reference to Tomcat 6 having better "perm size" management on the net vs Tomcat 5 series. Though nothing of a definitive nature. I saw it seems related since it removing the extra "Context Path" for this grails application seemed to resolve things. Perhaps it is an issue with grails applications in multiple Context Path's but the perm gen size increase did resolve it cleanly.

My current options line for catalina.sh:
JAVA_OPTS='-Xmx512m -XX:MaxPermSize=256m -server -Djava.awt.headless=true'

CBS bought me (well, last.fm)

2007-05-18T07:59:00.000-05:00

So I didn't want this blog to become abandoned, and in fact I have several items throughout the last weeks that I have felt worthy of blogging about and have not. I think I just need to discipline myself to post my thoughts when I have them and not "I'll do that later".

But yesterday when I heard CBS bought last.fm (and being a last.fm user) it got me to thinking and motivated to post this thought. I have just been bought... along with my data. Will CBS keep it, take it, toss it out? Indeed if you look around I have relationships with Google, Amazon, Last.fm, Digg and other sites. All this data is valuable to me and what might happen to it in a merger, take-over, bankruptcy?

Do we need a Federal DATA Insurance Corp.
I had this thought a few weeks ago when talking to Josh, regarding my use of Amazon S3 to backup my data. I was looking at various on-line data backup sites. There are many and for my small amount several I could get free (add supported). However, I liked the Amazon S3 due to it's use of services and quite frankly the size and apparent strength of Amazon.

We began talking about the amount of data a person has on the net. I have data I would call valuable with Google (mail, RSS, this blog and a bit more), Amazon (data via S3) and Yahoo (they own del.icio.us now).

However, if any of these were to go away what would happen to my data? Gone? It got me to thinking about the Federal Deposit Insurance Corp. Established to provide some assurance that money (or at least some) would be safe.

Though the analogy is obviously not one to one it does make me wonder if there isn't some general consumer SLA that needs to be established with on-line data "deposit" sites. A Federal DATA Insurance Corp. to provide the public with some level of trust, confidence that data stored on-line in places like Flicker, MySpace, S3, Google, etc are retrievable when/if a company goes south.

IDE for jRuby

2007-03-04T09:08:00.000-06:00

I went looking for the current state of IDE's for Ruby and in particular Ruby on Rails. In fact even more specifically for jRuby since I am not very interested in Ruby or ROR unless it is hosted on the JVM.

I found RadRails and RDT. Both of these are based on the Eclipse IDE platform. The former apparently based on the later. I installed and attempted to use both with jRuby. I had little success in this effort and from what I can gather jRuby support is yet to come for these packages.

I grabbed one of the Netbeans 6 nightly builds and was presently surprised to find jRuby there. A little more searching reveled this site which has done a far further exhaustive review of various Ruby IDE's than I have seen so far. Tor Norbye's blog section on Ruby also provides news straight from the source on this effort.

February Errata

2007-03-01T11:59:00.000-06:00

So it has been some time since I made a post. Been busy but I hope to manage my time better in future so that I can post my thoughts more often. The main reason I do this is to benefit myself in terms of aligning what I see and experience in the community into a coherent philosophy.

I wanted to touch on several things in the last few weeks I have been interested in. Actually sitting down and writing this was sparked by this news item (Adobe to take Photoshop online at Cnet) that caught my eye in my RSS feeds.

It caught my eye due to a discussion I had with Josh, where we debated a bit this very concept. Word processing and excell spread sheets aside it seemed unlikely that the on-line application model as exemplified by Google docs would work for something like GIMP or Photoshop. It is interesting to look at this in light of things like Joyent though. Correl is looking to take a variation of Word Perfect into a model that uses both on-line and off-line elements in collaboration with Joyent. This mix of on-line and off-line aspects to an application is very interesting to me. It is all part of the development that is taking place where API's like XUL, XAML etc are getting attention in the same context as GTK and QT. However, how much application API's matter is up to debate. Is it only the movement and management of information in an open and extensible manner that really is important?

One element of that did catch my eye was Remix which uses Flex. This is more interesting to me as it brings up the issue of rich client development. I have been watching XUL, XAML (Windows Presentation Foundation) and things like FLEX and Laszlo for some time now. Mix in things like the rich client platforms from Eclipse and Netbeans and you have the making of new "internet aware application API's" forming. This would be great. GTK, QT, Cocca etc. aside what is more interesting to me is the ability to make "application" that come to the user over the net.

Grails Update
Grails has a new major number out and a major shift in the way it deals with incorporating third party features. A very nice plug in approach. The jump from .3 to .4 was a bit painful as you have to un-weave the code you have done in the old approach for the new. I updated an Acegi security based application from .3 to .4. The biggest pain is removing the controller, domain, and other classes that were required prior to the plugin approach. Though this was a bit tedious it is well worth it for the nice new plugin based system. Hopefully such a major effort will not be required in the future. I suspect the Acegi package is one of the bigger ones to migrate.

Seems they also changed the way to they do the creation of URL's and several of my relative links got broken. I should be using more of the Grails link tag (http://grails.org/Tag+-+link) more than using relative URL linking.

This blog visualization is cool
This is neat.. http://twingly.se/ScreenSaver.aspx useful? Not so sure, but for sure neat. If I ran windows I would likely install it and play with it some.

Yahoo Pipes
Yahoo Pipes came out, this one one of those things I was interested in blogging about. I even retrieved/reactivated my Yahoo account just to play with this. While I found it very interesting, I can not say I found it highly useful. This idea implemented as an API I could embed in my app server, that would be far more interesting.

There are several other packages like this out there too. xFruits and even www.yourminis.com.

Don't get me wrong, the idea behind Pipes is very much along the route I would like to see things going. Mechanisms to collect, relate and cross reference data generated on the net. I'd just rather have it as an API for my applications rather than a third party service provide to/for me.

Get things done
I was only tangentially aware of the GTD approach. The book has spawned several packages on-line and I reviewed a few of them listed in the post at Center Networks. One, Tracks, caught my eye and I actually installed and got ruby working (which can be a pain with regard to DB access at times). (rant) The Ruby on Rails database API and interface is archaic in comparison to Java JDBC let alone persistence aspects (/rant)

I got it set up and working and played around with it some. It took a while to get this done due to the poor, IMHO, ruby database approach. Anyway, Tracks is interesting, but I think I was more interested in the experience it gave me with Ruby on Rails than anything else. I may try and play with it some more in the future though and revisit it as a utility and approach to time management. On a side note the Joyent package mentioned above is a Ruby on Rails application. The note some interesting news coming from them based on the collaboration with Sun. One might speculate on them switching to jRuby in the near future. jRuby plus the Java VM could be a very powerful combination for an application like this. I am still a Grails person until jRuby evolves more. When it does I will give it a try again.

XML UI's revisted (XAML, XUL and AJAX)

2007-01-27T10:17:00.000-06:00

Back in October I made a post about WPF and my concerns around it. Now the European Committee for Interoperable Systems (ECIS), has raised similar concerns. To be fair some of the points raised by the people at the respectable Ars Technica site are valid on this point. Their article on this development raises many issues that indicate XAML (WPF) is not quite a full scale attack on standards (not yet anyway). A review of WPF at Wikipedia.

This was interesting news for me to wake up to since I have been thinking about XML based GUI's lately in relation to the Grails development I have been doing. Grails recently added OpenLaszlo as a build option for interfaces. (Ref: http://grails.codehaus.org/OpenLaszlo+plugin)

The combination of XHTML and JavaSript to make user interfaces via AJAX that one can embedd into XHTML is currently all the rage. One can contrast this with XUL as yet another plus JavasScriptUI toolset. The process of embedding XUL UI's into XHTML might not be so clear to me at this time. Though perhaps XHTML in XUL is more likely anyway. I need to investigate this and also how CDF might play into this. Blending AJAX and XUL approaches might be an interesting gadonkin.

In the same manner that the grails generate-views command can be used to make XHTML interfaces the grails generate-laszlo is used to scaffold artifacts for Laszlo. So why not a grails generate-xul command? Indeed, as the Ars article points out it is hard to imagine anything that could compete with the ubiquitous presence of HTML. However, for web applications the ability to auto generate different UI's (XHTML, XUL, OpenLaszlo, etc) has some appeal. If this is done automatically from the domain classes with common class extensions to a set of scaffold'ed artifacts then a developer can simply worry about the application and the elements a UI needs and leave the final implementation of the UI up to the user. If the user wants XUL, then give that, if the user wants OpenLaszlo then give that. This might have more appeal yet as the web moves toward things like the iPhone, or the OpenMoko.

Also of note in that regard is the development of things like Wazaabi which brings XUL to the Eclipse RPC environment which can be used develop rich client applications. A friend of mine has used Eclipse RPC in his efforts related to work in Antarctica. :)

In conclusion I believe that the generation of richer UI's delivered across the web to browsers or brokered out to supporting apps by browsers is a developing trend. I'm interested to see how the Grails application framework could be used to generate XUL interfaces that are used from within Mozilla based browsers like Firefox. Combine this with the Mircoformats based semantics I have written previously about and I start to see a component framework built around Web standards and protocols. I need to build an example of some of this to truly investigate it.

Microformats

2007-01-14T09:22:00.000-06:00

I was interested to see some news items that on the surface don't seem related but to me have a lot to say about trends.

Mozilla has started talking about Firefox 3 (http://www.readwriteweb.com/archives/mozilla_does_microformats_firefox3.php) and in particular has mentioned microformats and a broader role for the browser as broker of information and service on the net to other applications. As a long time UNIX person I believe in the vision of small dedicated programs that do what they are meant to do and do it well tied together via piping. So the idea of data described by microformats, discovered or retrieved by broker and routed (piped) to other more dedicated applications has appeal to me.

Also of note was Songbird talking about plans for an API for site developers. I hope this evolves into a microformat or RDFa that allows a music centric site developer to place data directly into their web page that allows browsers like Songbird or even machine searching and indexing of music by the likes of Google or Singing Fish to use structured data from within a page or site.

This could get very exciting with things like Apple TV and The Venice Project (which is also using Mozilla technology) coming up and extending this possibility into video. With browsers getting more extensible and smarter about how they can handle structured data site developers will actually be able to see a reason (ie benefit) to doing something (anything) with metadata. Which to this point has has little to no use or benefit (IMHO).

404... your app no longer works

2006-12-29T08:19:00.000-06:00

How many more buzz words could I get into to the title? Been a while with Christmas this month, but there is something I have been meaning to think/write about. Google says it will do no evil. So why then am I some what concerned when I see things like the news at: http://code.google.com/apis/soapsearch/. While I might not be surprised to see the SOAP end point go away, replaced by a ReST end point, I am a bit shocked to see an AJAX replacement.

This Google search box is definitely exactly what I have been working with myself (and many others). The ability to place a software component directly into a remote DOM. It is also very analogous to the OpenDoc approach I have been advocating for sometime.

However, an AJAX component is not a service. It is not something that I can invoke like a SOAP or ReST end point. Obviously (I think) that a dissection of what is occurring with Google Search AJAX component would reveal a ReST based response system, likely returning XML or most likely JSON(P). However that doesn't mean that the average user will be able to use it or even access that ReST end point (if it is there).

This whole issue with services is interesting. A development group might say they are going to create all their service components as open ReST or SOAP end points. For them to then eat their own dog food is important. That is they should use what they offer others. However, this can have issues like performance and scaling factors. These are issues that might be avoided in a group just used native environment services like (for Java) RMI, Jini, beans, EJB, etc. The lure of SOAP and ReST is that others can use them regardless of platform. However, we don't have an environment right now where many service consumers rely on or trust remote data services to build web apps on. Aggregated RSS/ATOM feeds don't count as a service.

In fact, most groups create and use their own services to build their final value added result, not for others. In that case however, the argument for SOAP or ReST is even harder to make. Indeed that very argument favors the use of components in the primary language(s) of the development group.

Yes, the argument can be made that in the case of Google, driving or presenting the brand is more important from a business POV than "doing good" by giving out a SOAP (or ReST) end point for people to use with potentially no reference to Google at all. However, I am not going to dwell on the business model aspect of this action in this post (perhaps tangentially I will).

No the real wonder I have is since there is no real market for services, will the SOAP and ReST efforts just wither and die for now? Until a need (a want) for the ability for sites like Slashdot and Digg, or TV listing sites or Mac rumors sites or whatever to actually work together for the benefit of each other (and as a result the users) we wont see much need for services. This is unlikely to occur in our current environment where the business model for sites is to get readership and eye balls across adds or to the fewer "pay for premium access" sites. It may occure in other groups less driven by economics and more by altruistic endeavours like science, social good, etc. Even then though, how many groups are willing to risk the operations of their web app on the existence and reliability of services provided by completely autonomous groups?

Had UDDI or any other directory service actually succeeded in gaining use perhaps some of this might have been avoided (perhaps... not likely though). However, that didn't happen. I am curious about an approach that involves more server side logic where an "agent" like application downloads logic and data into a local cache. From there execution of local code can produce results based on the remote logic and data. This has many issues though since many disciplines have so much data that transferring the all the data is not feasible (or logical).

Don't get me wrong. Services are not dead and things like SOAP and ReST may find more affection from things like mobile devices, web aware game consoles (for things like Wii weather) or the recent Google / NASA efforts that might show up in Google Earth than they might from server farm based efforts.

Additionally, ReST based service end point information embedded into a page perhaps via some microformat syntax might be interesting if more value added browsers (rich internet apps (RIA)) like Songbird become more popular. In that example a web site devoted to music might inform a browser like Songbird about its local offering via ReST end points rather than Songbird scrapping the sites content and page DOMs.

Perhaps services are just waiting for that killer app (RIA or agent) to come along and actually make people want all this.

Ref:
http://developers.slashdot.org/developers/06/12/20/0155238.shtml

A groovy week on grails

2006-12-02T08:50:00.000-06:00

So the Ruby on Rails people asked the group that started the "Groovy on Rails" effort to stop using the term "on Rails" (whatever). That is why they have the name Grails now. They didn't ask me and I have to say that this week of building a first end to end app for production use with Grails was quite groovy.

First, I do like the coding by convention approach that allows less code and less configuration to be used. I also like that what configuration is there is in the form of classes not XML. We made an app for the ANDRILL effort and it should see the light of day soon (ie be reviewed and deployed).

During the course of this development I was very impressed with how the various packages in Grails: Hibernate, Quartz, AJAX (via prototype in my case), Spring (obviously) and the GORM objection relation mapping all work. I am very keen to make up a Quartz job and then use the jabber resource to make a job that looks for changes or additions to the database and then IM's me. That is more for me, the real version would likely report via email.

We did run into some issues (Grails is only version .3 after all). An issue with binary uploads in a none scaffolded controller (I assume the generated static code is somehow out of sync with the dynamic scaffold generated code). Also had an issue with using the remotelink tag for AJAX calls vs using the remoteform tag for the same effect. We resolved this by just not using them :) but I need to report this to the Grails people to see if it is a known issue or something I was doing wrong. Getting used to the scope of certain classes and methods took some getting used to as well, but that is more just my ignorance with how the plumbing was working at the start of this effot.

Having full access to the all the wonderful Java classes makes Grails a no brain-er for me over Ruby on Rails. I have to admit I am a bit sad that Sun seems to be in love with Ruby (via jRuby which ironically is also a codehaus effort at http://jruby.codehaus.org) and pays little if any attention to Grails so far (hopefully this will change). Yes, though jRuby the Ruby syntax gets access to the classes as well. We will see how this all works out. I was interested to find that one of the key people in the Apache Geronimo effort is also one of the key Groovy people as well. Ruby has some very cool aspects to it, but I think the JVM is a better host for it than as a stand alone app directly on top of the OS.

In a final experiment (ref: InfoQ Grails + EJB and a Graeme Rocher blog entry) we used entity classes (not entity beans) that we generated via Netbeans for a database and dropped the POJO's into a grails app and were able to get read and write access to the db via the grails app using these entity classes as our domains with simple generated controllers and views. We had some issues with primary key mappings, but nothing too hard for us to work around. This was interesting to me since I could use the entity classes along with Hibernate to do the data access work and then Grails as an lightweight flexible dev layer on top of that. Grails lacks sophisticated buses like Service mix and others, but I am not certain I need or want those at this time (or that I can't do what I really want inside the Grails framework anyway).

coding by convention vs prior convention

2006-11-26T13:19:00.000-06:00

Been a while since I posted. I have still be doing some work with Grails and Java EE5. I completed out my test for UI's via AJAX as well but I thought I would put down some thoughts on coding by convention.

I am new to the Rails approach and I have to say I quite like it. The idea that the file system, file names and such carry meaning and that the framework uses these to look for things that might otherwise be designated in XML configuration files is quite nice.

Until of course you create a domain image. I created the domain image in Grails. This has the effect of telling Grails that any URL with /app/image in it should be processed by the controllers, services and such associated with that domain. Unfortunately this also means that img tags with src attributes that use /app/image in their path don't load images from a directory in that path, but that pass the request on to the domain controller.

It's a small point perhaps, but it does make me wonder what else I might run into. Domains with names like image, pdf or I am sure many other (conventional names) just might interfere with the convention of the framework.

Working on contrasting EE5 and Grails (Rails)

2006-11-06T09:01:00.000-06:00

It has been a while since I did a post. I have been busy at work on some new architecture designs and intend to make a good entry contrasting Grails and Java EE5 efforts soon. However it has been a while and thought I would post up some errata thoughts I have had. It has been very interesting to see how the two efforts (Grails and EE5) deal with similar goals compared and contrasted to each other. Trying to do the same thing in two different frameworks has definitely made it easier to see how each is addressing needs. Of particular interest is comparing how grails deals with call backs from ajax widgets to server side logic compared to the Java Jmaki effort. I've got a longer post with my lessons from this that I am working on for a later date.

I must say I have come to really like closures. A programming concept I have not used much to this point. Groovy (which Grails is based on) does closures and I also found an interesting page on closures for Javascript. Ruby also obviously does this well.

I selected Grails over Ruby on Rails mostly due to the access I would have to the JVM combined with the Rails approach. Also since I was wanting to contrast against a pure EE5 approach this made more sense. By pure EE5 I mean an entity bean approach combined with JSF. With the hiring of Nutter and Enebo by Sun Microsystems, Jruby (another codehaus.org effort) could becoming an interesting option too since then "Ruby" will have access to the Java API's. It has that access now I believe, however I don't believe the Ruby and Rails aspect of Ruby is fully over to the JVM yet. I do like the coding by convention vs. the XML configuration approach however. I love XML, but think we have done too much with it in places where we could just use the coding language we have available. I definately will be looking at Ruby in general and the Ruby on Rails community more.

A very interesting interview is given by Tim Bray on InfoQ. It's a great listen for anyone interested in dynamic languages, RAILs concepts, ReST vs WS and some of the issues surrounding them.

AJAX and server side scripting

2006-10-24T08:51:00.000-05:00

I ran across this page at the sun developer site. It has a very nice run down of various AJAX design strategies. (Ref: http://java.sun.com/developer/technicalArticles/J2EE/AJAX/DesignStrategies/?feed=JSC). This is a good read not just for Java developers but anyone. The principles in it apply toward the development of any architecture in any language implementing AJAX.

In reviewing this document I came across a project again that interested me and spent some time playing with it on our app server (glassfish this time). This is the Phobos project (Ref: https://phobos.dev.java.net/). It basically allows for the use of javascript (and later other scripting languages like hopefully Groovy) to be used on the server side. JSP and JSF use tags to embed Java code into pages that are rendered server side. Phobos extends this to javascript with the concept of "ejs" files which embed javascript into the page for server side rendering. The benefits to an AJAX implementation of javascript on both side are obvious. I do not know yet how well the server side javascript has access to the Java classes. Future integration with JMaki might also make future use of JMaki widgets in Phobos apps easy. How might one instance this into a remote domain though?

I am still curious if one could simply make a GET call to an app server that would render custom javascript in JSONP manner and return it to a script call of a page. In truth couldn't this be a servlet. A servlet responds to a GET call and can return just about anything. So one would seem to be able to make a servlet that returns JSONP. If the Phobos server could return JSON or JSONP as well as EJS rendered HTML that might open some interesting possibilities for remote access to widgets. This may be worth some investigation and comparison to some of our early GWT application experiments.

Why Portals Must Die

2006-10-15T08:00:00.000-05:00

A few days back I read a couple articles on some efforts by Google. These were an article on Googles Gadet effort and Google /BEA mash up talks.

While interesting in their own right and likely the kind of thing being done by Yahoo, Microsoft and others it got me to thinking about a mantra I have had going for some time:

Portals must DIE.

This is not to say portals themselves in concept are bad things. Only that they are often used or implemented poorly. I oversee a JSR-168 based portal myself and direct development of applications under that. The intent of portals is obvious. It provides a development group a unified display and authentication API set, among others, to work with. This helps to focus development skill and look and feel issues. A great thing compared to developing against a myriad of API's. Of course portal API's themselves don't talk to each other so little was gained for efforts outside the local enterprise (or enterprise group). In the Java camp the JSR-168 helps to focus efforts around a common portlet spec and that is great. Web services for remote portlets (WSRP) might have been a wonderful thing had I ever seen one example of it in the wild.

It is just that portals don't allow for dynamic mixing of content from various sites into my own space. I can't easily (yet) pull maps from Google, stock tickers from Yahoo, search interfaces from 3rd party databases sites and simply drop them into my page and have them work with authentication and styles (CSS) resident there. I want to be able to do this, so portals must die.

I believe that the common HTML DOM or a CDF derived instance is the common glue (as I spoke about in my OpenDoc posting). Combine this with remote AJAX applications that have some capacity for awareness of secure federated identity and we begin to see the a portlet concept in this environment. Secure "inter-AJAX portlet" communication could be done via a messaging middle-ware though I want to explore what Javascript allows in terms of intercommunication between various functions in a web page to see if that is possible. Look and feel issues could be addressed through systematic CSS utilization. In a manner similar to GWT.

To me the future portal is a simple DOM with portable code that can be pulled across the net to create its UI through local DOM object manipulation. Combine this with federated identity and CSS based styling for logging in and look and feel issues respectively. Let the whole thing talk to a services oriented architecture for logic execution. Hopefully the efforts of the W3C Rich Web Clients efforts will go well.

.NET 3.0 an attack on standards?

2006-10-15T07:54:00.000-05:00

So I read my first article on .Net 3.0 this morning over at informit.com. I then headed out to the Netfx3 site and did a little reading on the Communication, Presentation, Workflow and Cardspace foundation that are part of .Net 3.0.

Microsoft has obviously turned their back on standards like Web services, SVG, efforts like SAML, WS-Federation and the Liberty Alliance. I am sure they have done in this in the name of "user experience". However, this will be user experience only on Microsoft Windows.

Do I trust that things like WPF/E (the "Everywhere" cross platform version of WPF) will be coded to, made easy to code to, be used at all by many developers? Of course not, Microsoft has no interest in doing that. As a company they have fostered a culture based around ignoring standards and ignoring where the network is going. Even as powerful as they are, I have no doubt they will fail, but they have the potential to cause huge amounts of damage to peoples data along the way.

The core interconnecting standards of the net are under attack. Microsoft will use its influence, FUD and any other tactics to encourage this. Then they will wrap themselves in the flag of "competition is the cornerstone of capitalism" implying that they are the moral high ground holders.

It is obvious that Vista and .NET 3.0 will bring the most concerted effort yet by Microsoft to undermine Internet standards and cross platform environments.

CDF + Google gadgets = OpenDoc ?

2006-10-06T08:53:00.000-05:00

So recently Google was noted for putting out "gadgets" that could be syndicated across the network (ref: Cnet ). The direct link to this being at: http://www.google.com/ig/directory?synd=open/.

This is basically javascript that you can instance in your own web page. There is nothing spectacular about this. However, Google has done a nice job of making it easy (easier) to do this. I am a big supporter of this syndication concept even though there are likely several issues still with this idea especially around security. (One Ref at Greg Murrays Blog)

Now combine this with something like the W3C CDF efforts. CDF allows a developer to place multiple XML formats into a common document.

The collision of all this reminds me of something from Apple from way back. Those that know me know I still haven't gotten over OpenDoc. systems could begin to However, CDF + remote web apps like AJAX Esqdevelop the kind of application that currently is still restricted to the true rich client applications. I don't know where web based applications will stop in terms of being reasonable approaches. Email, RSS aggregation, content management and many other types of communication based efforts have done very well on the browser platform. Lately word processing, spreadsheets and other attempts have started to crop up. I doubt we will see an AJAX based GIMP (or ever want one). For the most part, if I am working on data or creating data local like in a GIMP image there is no need to send that over a network. However, Writely has shown that even word processing can benefit from network aware ability, in its case in the form of real time collaborative editing of documents. (Apple Knowledge Navigator)

Imagine though a document with a SVG canvas being written to by an embedded ajax application communicating over XmlHttpProxy and then locally updating the SVG canvas through inter DOM communication. How about an OpenGL canvas? (Java 3d and applets is perhaps a bit of a digression for this posting, but not much).

I am encouraged and interested by where an open format, open standards based approach will go that mixes interactive applications with the DOM and graphics languages like SVG and perhaps even OpenGL.

It's about the data

2006-09-26T10:59:00.000-05:00

I have been trying to figure out what would make me happy. I don't like the net as it is today, it feels to ethereal. Like anything I make, a blog post, a OPML file anything is just going to end up like some old 9 track tape I find some day and can't read or use.

Yea, I could download this stuff and store it somewhere (I do) but must I? Should I? I find that while I have my dug articles from Digg, how many of these links will be there in a year? I'd rather just cut and past them into a personal store. Like the scrapbook firefox extension which I love.

I some ways I would like to see it combined with something like the Simile Piggy / Semantic bank efforts. Perhaps those are what I like and I am just not using them enough yet. At least they are using standards I philosophically agree with like RDF and XHTML and such.

Lately I have been listening about Jini on the Javapossee podcasts and been wondering about how it could be used in data grids vs. the compute grids it seems focused on now.

Now if I could just figure out a portal/dashboard approach I like I would be set. Wonder if KDE's plasma effort will have an easily extensible UI apsect to it ala kramaba (sp) that will make it the wonder desktop. I like the concept of the containers I see so far in it. What I need is a tag based files systems (who needs folders?) and a tagged based UI to go with it. :)

Firefox Application Framework

2006-09-21T11:11:00.000-05:00

For some time I have watched the XUL effort at Mozilla. It is interesting as more and more people talk about the creation of "rich apps" (network centric apps.. whatever) to watch various approaches rise to the top. The strong efforts by the eclipse and netbeans camps to form their solutions about their respective architectures is great. There is also the efforts by M$ and their XAML goals eventually. I don't really think XAML is all that "Internet" when you have to run it on a windows machine though.

However, I have been interested to see that XUL, which I felt was only going to be a solution for Mozilla's own apps like firefox, thunderbird and sunbird, gain some traction in the community. The Songbird (http://www.songbird.org) effort is likely the one with the highest visibility, but Democracy TV (http://www.getdemocracy.com/) is also popular. I recently saw on a on-line episide of Command N where they talked about Cletx (http://www.celtx.com) which I guess is also based on XUL. They say it's a firefox app, but I assume the meant XUL.

It would be nice if we had a true XML UI language (yes, perhaps XML is a bad method to represent UI's ... that is another topic) that was a standard. Not that M$ would bother to support it but XUL can't claim to be a standard yet for anything but Mozilla apps.

There are tons of XML UI languages for the Java VM true, but I am not sure they leverage well of the developer community till they get some kinda of buy in from Sun itself. Perhaps with AJAX there is no need. Javascript + DOM + XHTML and go from there, but you loose some of the spiffyness of a true UI (widgets and all). I haven't seen SVG care much for doing it and I refuse to go the flash route.

So I don't know.. if you want to make a true Internet application I guess there is not set of standards to work with other than AJAX approaches.

Web 2.0 semita (footprint)

2006-09-19T11:57:00.000-05:00

So I am trying to figure out what my "Web 2.0" footprint looks like and how this compares to my personal computer footprint. The core elements of my web footprint are: (beyond this blog):
Del.icio.us (http://del.icio.us/fils), Odeo (http://odeo.com/profile/Fils), Tag cloud and Blog lines (don't much care for Blog lines for RSS reading)

I have found I don't care for any on-line RSS tools. Prefering instead my Liferea on Linux, though obviously it is my OPML file that is the most important to me.

For news I see I use: Slashdot, Digg, Newsvine, Last.fm for music and Boxxet (the later more interesting in concept than useful in execution) . For all I have accounts and I should resolve out the URL's for them and get them posted here.

I find tools like Writely and box.net to be rather nice and I have started to use them some.

Why do this

2006-09-01T11:49:00.000-05:00

So I am not sure why I just did this. I am not really much of a blogger (at least I don't think I am). However, I have started to try and figure out what it is about web 2.0 that bugs me... so I have to do it to resolve why.