Wednesday, October 3, 2012

Writing a terminal app with MeteorJS

Since Javascript is nowadays dominating more and more my daily web developing activities, I've been more interested in extending its usage to server side. A few days ago, while reading articles about Node.js, I found out a new excellent JS framework build on top of Node.js.

MeteorJS is a new open source framework build on Node.js. It's developed for concurrent browsing experience for web apps that are being simultaneously used by number of people. Building concurrent web apps like live chats, Google docs etc. can be achieved in a friction of time.

By default, Meteor uses build-in MongoDB for data storing which is quite easy to adapt. In Meteor's official examples, Handlebars is used for templating system but also alternative approaches such as AngularJS are supported. Meteor uses its own package manager, Smart Packages, providing support for favourite JS add-on's like jQuery.

If you feel like getting more familiar with Meteor, I suggest reading an excellent blog post by Andrew Scala about Meteor Fundamentals and Best Practices.


Running shell commands with Meteor



I got a nice idea for an app to learn the basics of Meteor. The web app simulates the usage of terminal by having a command line as input field and a div block for shell output. The app is considered only for testing purposes and to demonstrate the functionality of Meteor in practice.

If you're new to Meteor, download the SDK and create a new project. Replace the HTML and JS files with ones disclosed below and run the app.


<head>
  <title>MeteorJS terminal</title>
</head>

<body>
  {{> terminal}}
</body>

<template name="terminal">
 <pre>{{ window }}</pre>
<input id="command" type="text" value="{{ last_cmd }}" />
 <input type="button" value="Run" />
</template>

app.html
// A collection for stdouts
var Replies = new Meteor.Collection('replies');

if(Meteor.is_client) {

 // Start listening changes in Replies
    Meteor.autosubscribe(function() {
        Meteor.subscribe('replies');
    });
 
 // Set an observer to be triggered when Replies.insert() is invoked
 Replies.find().observe({
  'added': function(item) {
   // Set the terminal reply to Session
   Session.set('stdout', item.message);
  }
 });

 // Show the last command in input field
 Template.terminal.last_cmd = function() {
  return Session.get('last_cmd');
 };
 
 // Show the last shell reply in browser
 Template.terminal.window = function() {
  return Session.get('stdout');
 };

 // Add an event listener for Run-button
 Template.terminal.events = {
  'click [type="button"]': function() {
   var cmd = $('#command').val();
   Session.set('last_cmd', cmd);
   
   // Call the command method in server side
   Meteor.call('command', cmd);
  }
 };


}

if(Meteor.is_server) {
 var exec;
 
 // Initialize the exec function
 Meteor.startup(function() {
  exec = __meteor_bootstrap__.require('child_process').exec;
 });

 // Trigger the observer in Replies collection
    Meteor.publish('replies', function() {
        return Replies.find();
    });

 Meteor.methods({
  'command': function(line) {
   // Run the requested command in shell
   exec(line, function(error, stdout, stderr) {
    // Collection commands must be executed within a Fiber
    Fiber(function() {
     Replies.remove({});
     Replies.insert({message: stdout ? stdout : stderr});
    }).run();
   });
  }
 });

}
app.js

Tuesday, March 9, 2010

GAE/J datastorage persistence with Objectify framework in practice

I am quite a newcomer for Google App Engine platform and just getting familiar with the basic concepts. Google App Engine is a platform for developing and hosting web applications in Google-managed data centers. One of the most important features in App Engine is using the DatastorageService where developers can persistet their application objects in BigTable clusters. Many who have worked previously with JDO and JPA might find the Datanucleus implementations worth trying but I come from PHP background which lacks those high level abstractions anyway. JDO, and especially JPA is designed for managing traditional relational database models which fights against the design philosophy of Google Datastorage. Moreover, both persistence interfaces are developed over a long period and provide backward compatibility increasing the historical burden of their implementations.

Google Datastorage is built on BigTable which is addresses highly scalability and geographically dispersed locations. In constrast to RDBMS'es, Google Datastorage has other principles one must take into account when persisting objects in the cloud. Google provides DatastoreService, but it has more machine-friendly than human-friendly query interface1.

Fortunately, there is at least three frameworks providing higher abstraction for using the DatastoreService. Objectify, Twig and SimpleDS address the complexity of DatastoreService by providing a human-friendly query interface for applications objects' persistence.

Selecting between those three took me a few hours as I was browsing through their discussion groups and evaluating the quality of Wiki. Especially, effort the developer Jeff Schnitzer has put on speaking in favor2 of Objectify and defending its design principles against other candidates made me an impression that the project is really worth trying :)

Let's get started with Objectify


The following tutorial is written for developers who have no previous experience with Google App Engine. If you feel you're more experienced, just leap the steps you feel not suitable for your purposes.

This stupid app demonstrates a simple one-to-many relationship between a car and its owner, and how to manage their persistence with Objectify.



Relational domain model


1. Install Eclipse and
Google Plugin for Eclipse

2. Create new GAE project

Name your project and package as you wish but deselect Use Google Web Toolkit and then click Finish

3. Download Objectify 2.0.2 and copy the jar into lib-path and add it also to build path


4. You can test run your application and get the "hello world" printed on browser screen

Everything seems to be ok, so let's proceed with some real code

5. Let's create an entity class Person which stores the car owner instance
import javax.persistence.Id;
import com.googlecode.objectify.annotation.Entity;

@Entity
public class Person {
@Id private Long id;
private String name;
/* setters and getters omitted for brevity */
}

Notes:
  • @Entity is not obligatory for Objectify but good for distinguishing entities easily
  • @Id is the only required parameter and can be Long, long or String. I prefer Long, because id is auto generated if entity does not exist

6. Create Car entity class


import javax.persistence.Id;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Parent;

@Entity
public class Car {
@Id public Long id;
private String license;

@Parent public Key<Person> owner;

private Car() {}
public Car(Key<Person> owner) {
this();
this.owner = owner;
}
/* setters and getters omitted for brevity */
}

Notes:
  • @Parent informs that the entity is in many-one relationship with Person (unlike in JDO/JPA, only children entities annotate their parent entities). Parent must be a type of Objectify Key data type.
  • Car constructor must be created with Key<person> argument to set the owner
7. Now we're going to implement some logic. Open ObjectifyCarsServlet.java and replace the code printing "Hello, world"



public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws IOException {

resp.setContentType("text/plain");
resp.getWriter().println("Hello, world");

}



...with...



public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws IOException {

resp.setContentType("text/html");

String html = "<form>Owner id (blank for new)<input name="id" type="text"> "+
"Car license <input name="license" type="text"><input type="submit"></form>";
resp.getWriter().println(html);

if( req.getParameter("license") == null ) {
return;
}
if( req.getParameter("license").length() == 0 ) {
resp.getWriter().println("License not set");
return;
}

}


Notes:
  • You should have basic form with car id and license fields when you restart the application and browse it
  • Owner id's are auto-generated by Objectify so you can not give distinctive name by yourself
8. Then... we must register Objectify entities. Write a static block for ObjectifyCarServlet-class:


@SuppressWarnings("serial")
public class ObjectifyCarsServlet extends HttpServlet {

static {
ObjectifyService.register(Person.class);
ObjectifyService.register(Car.class);
}

public void doGet(HttpServletRequest req, HttpServletResponse resp)
...


Notes:
9. Now let's make some logic for GET-requests. Paste the following code block bottom of doGet()-method:


Objectify ofy = ObjectifyService.begin();

Person owner = new Person();
if( req.getParameter("id") != null && req.getParameter("id").length() > 0 ) {
owner = ofy.find(Person.class, Long.parseLong(req.getParameter("id")) );
}
else
ofy.put(owner);


Notes:
  • Create Objectify object, ofy, for persistence manager
  • Then, we either find an existing owner or create a new one
10. Continue pasting with...

Key<Person> ownerKey = new Key<Person>(Person.class, owner.getId());
Car car = new Car(ownerKey);
car.setLicense(req.getParameter("license"));
ofy.put(car);

Notes:
  • Declare new ownerKey data-storage key for the owner object
  • Create a car instance for the owner
  • Persist the car instance with Objectify.put()
11. Finally, we display all cars stored in data-storage:

Query<Car> q = ofy.query(Car.class).ancestor(ownerKey);
resp.getWriter().println("<pre>Cars for owner id "+owner.getId()+":");
for( Car c : q )
resp.getWriter().println(c.getLicense());

Notes:
  • ofy.query(Car.class) searches all Car.class entities from the database
  • ancestor(ownerKey) omits all Car entities which don't have the requested owner from the result set (it took me quite a while to find out that Objectify Wiki lacks an example of using Query.ancestor() method)
  • Query is a collection type so it can be iterated through and strip down to single entities
There you go, the result could look something like this with browser after populating a few entries to the data-storage...

Saturday, July 11, 2009

Doppler image animation with PHP and ImageMagick

Gnome Weather panel provides a radar map using an image URL to show a current weather forecast. Radar map has a support for animated gifs which drove me to develop a little PHP script to make a doppler animation.

I found out that Foreca provides a set of dopper images showing the next 7 hours forecast in Finland. Images have well formed filenames, so there's an easy way to make a script to download them periodically.

Dependencies: ImageMagick and a truetype font

// Nice little script I made courtesy to Gnome Weather Report panel's radar image
chdir(dirname(__FILE__));
exec('rm *.gif');
// download doppler images (foreca provides next 7 hours ahead)
for( $i = 0; $i < 7; $i++ ) {
$hour = date('H')+$i;
$hour = $hour%24;
$hour = str_pad($hour, 2, "0", STR_PAD_LEFT);

$date = date('Ymd').$hour;
$file = 'fin-'.$date.'.gif';

// thanks to taloussanomat :)
$url = 'http://http.foreca.com/xfer/taloussanomat/precipanim/'.$file;
exec( 'wget '.$url );

$date_str = $hour.':00';
// make a date image
exec( 'convert -size 50x20 xc:transparent -font '.dirname(__FILE__).'/tahoma.ttf -fill red '.
'-pointsize 14 -draw "text 5,15 \''.$date_str.'\'" date.png ' );
// add date image on doppler image
exec( 'composite -gravity southeast date.png '.$file.' '.$file );
}

// generate animation (1s delay, infinite loop)
exec( 'convert -delay 100 -loop 0 fin-*.gif doppleranimation.gif' );


The script can be added to crontab, to make the doppler animation refresh once an hour. Working animation can be seen here http://panic.fi/~tero/weather/doppleranimation.gif