Using MongoDB with Morphia and Groovy

As I mentioned in this post, I live coded a blog site using Morphia and MongoDB in my workshop session at MongoSF. I used Grails as the platform to quickly scaffold the application. One might wonder why I didn't use the Grails plugin. I used Morphia because it aligns with traditional Java paradigms and didn't want to have too many new items by using a full Groovy stack.

Setup

After installing MongoDB, you will need a version of the Java driver and the Morphia library. Make sure to use the latest snapshot.

Creating the mapped objects

Morphia uses Java annotations to describe how to persist objects in the database. The following annotations are available:

  • @Id - marks the MongoDB id field to be autogenerated
  • @Entity - marks the class to be persisted as an object and optionally the collection
  • @Embedded - tells Morphia to embed this object in another
  • @Reference - analogous to MongoDB DBRef
  • @Indexed - prepare an index for this property
  • @Serialized - store as BSON
  • @Property - indicates that the property name in MongoDB will be different than in the object
  • @Transient - doesn't persist the property

As you may already know, Groovy adds a few properties to objects. You will get an error that it won't persist the unmapped fields but it doesn't inhibit execution. Our BlogEntry class contains a title field, date it was created, content, a MongoDB document id and an embedded list of comments.

import com.google.code.morphia.annotations.*

@Entity
public class BlogEntry implements Comparable{
	@Id  private String id
	
	String title
	Date dateCreated = new Date()
	String content
	
	@Embedded
	List<comment> comments = [ ] 	 	  	
}

Similarly, our Comment object contains properties for the name, dateCreated, and content. Please note that both places that refer to Comment need the @Embedded annotation.

import com.google.code.morphia.annotations.*

@Embedded
public class Comment {
	String name
	Date dateCreated = new Date()
	String content
}

While you can marshal/unmarshal objects from POJO/POGOs to BasicDBObjects and persist them to a collection by hand, it is much simpler to use a data access object(DAO) which provides all the CRUD functions for you.

import com.mongodb.Mongo
import com.google.code.morphia.*

public class EntryDAO extends DAO {
    public EntryDAO(Morphia morphia, Mongo mongo) {
       super(mongo,morphia, "entries")
    }
}

In this snippet, I initialize a DAO object, create BlogEntry and Comment objects, persist them to the DAO, and read them back out to prove they were stored. Please excuse the Java-ness of the code but I didn't want to blow people's minds with too much Groovy.

import com.mongodb.*
import com.google.code.morphia.Morphia

mongo = new Mongo()
morphia = new Morphia()
dao = new EntryDAO(morphia, mongo)

entry = new BlogEntry()
entry.setTitle("Test Entry")
entry.setDateCreated(new Date())
entry.setContent("This is a test entry.")

comment = new Comment()
comment.setDateCreated(new Date())
comment.setContent("Test comment")

entry.comments = []
entry.comments.add(comment)

dao.save(entry)
entries = dao.find()
entries.each {
    println it.properties
}

With a running instance of MongoDB, you could run all these bits in a Groovy console and see it in action. In the next post, I'll cover how to integrate these classes into a Grails application.

 

Java at MongoSF

As has come to be expected the 10gen-ers have put on a great conference. It's almost like a MongoDB world domination tour with the NoSQL Live in Boston this March, MongoSF and MongoNYC this month and MongoEU this summer. There seemed to be bigger numbers of Ruby and Python devs but that wasn't so surprising given that they tend be early adopters compared to Java devs. Because of that I believe the path to reach Java developers is to show them that they don't have to throw away all their code to use MongoDB. My morning session capitalized on that and used Grails to show how you could quickly wire up persistence using DAOs with Morphia and MongoDB.The afternoon session that was a bit more high level primer for what was covered in the morning. 

 
The schwag was well thought out too. Speakers got really snazzy embroidered MongoDB polos. There was an update of the mugs from NoSQL Live featuring the MongoDB branding on the one side with 10gen branding on the other. 
 
Slides from the conference are here. Videos from the sessions should be coming soon. Check the 10gen blog for updates.
 
 
It was also great news to learn that there are several books on MongoDB set to release this fall. 
 
 

Presenting at GeeCon 2010

I'm happy to announce that I'll be in Poland this May to present at GeeCon. I'll be giving a preso titled "Game Programming with Groovy." While it's isn't mentioned in the title, there will definitely be a Griffon component.

GeeCon takes place on May 12-14 in Poznan, Poland. You can find out more information about the conference or register here.

NoSQL Live 2010 recap and MongoSF

Last week, I attended NoSQL Live in Boston. The conference was organized by 10gen and Hashrocket. Besides the normal plights of a conference (wifi), it was really well put together event. Check out the 10gen blog for other posts and media from the event.

Due to a last minute cancellation, I gave a lightning talk titled "Using MongoDB with Groovy." (slideshare) I highlighted how Groovy's metaprogramming and post-Java nature makes it a philosophical match for the post-relational and schemaless nature of MongoDB. Though there was video streaming of the event, a server crash caused those files to be lost. However Christian Scholz recorded MP3s of the event available at the 10gen link above.

10gen has decided to have another one-day conference in San Francisco on April 30th. This time the focus for all sessions will be MongoDB. I will be giving a talk on Java development with MongoDB. You can find out more information here or register for the event here. Early bird pricing of $50 ends on April 9th.

Griffon at GR8 USA 2010

I'm happy to announce that I will be speaking at GR8 USA. GR8 USA is a one-day conference on all things in the Groovy, Griffon, and Grails ecosystem. The conference will take place on April 16, 2010 in Bloomington (close to Minneapolis). Venkat Subramaniam and Guillaume Laforge will be giving the kickoff and keynote speeches.
 
I'll be leading 2 sessions at GR8 USA: Porting your legacy desktop apps to Griffon and Enterprise Griffon.
 
Registration and signup details are available at http://us.gr8conf.org/
 
Early bird registration is only $175 before the 28th of February.

Deckchair Groovy JSON store

Deckchair is a Groovy lightweight JSON document store ported from JS(Lawnchair). In a lot of desktop applications, there is a need to store some sort of information. It's usually not a massive amount but enough that the current options aren't exactly suitable:
 
• JNLP Persistence API
• Java Preferences API
 
The JNLP Persistence and Java Preferences APIs don't really allow you to structure your data and JNLP Persistence doesn't exactly work in difference contexts.
 
Deckchair uses JSON documents to allow you to structure your data without regard for how it will be stored by the backing adaptor. Currently the only implemented adaptor is Derby but others will be coming. Deckchair is fairly easy to use and is API compatible with all the signatures of Lawnchair
 
Sample Usage:
def d = new Deckchair(['name':'temp', 'adaptor':'derby'])
d.save([name:'James', age:28])
println d.all()[0]
 
You can download the source here

Replace Amazon S3 with MongoDB GridFS and Grails

Mostly because this blog started on a hosting plan that didn't really allow file server access from web applications, I got in the habit of putting files on Amazon S3. While I love the ease of throwing something on S3, I don't particularily like the cost given the small amount of data I'm serving per month.

My recent foray into MongoDB got me thinking about using it to serve files. You can do just that with MongoDB's GridFS specification. When a file is stored in GridFS, it is represented by a metadata object and one or many chunks which store a subset of the data. Chunking the files helps with searching but could help replication and sharding presumably. Files in a GridFS store are just like any object in the database. You filter by any of the metadata properties and get chunks on demand or out of order.

Setting up your application

You could hardcode config settings if you run mongo on a different host or port but I put the following settings in Config.groovy:

 
mongodb {
    host="localhost"
    port=27017
	dbName = "whatever"
}

I added the following to my UrlMappings.groovy file so expose a download path to our files.

"/downloadFile/$filename" {
    controller = "files"
    action = "downloadFile"
}

Wiring your application to MongoDB

Next I created a GridfsService to interface with MongoDB. To get the settings from Config.groovy injected, I subclassed InitializingBean and did my configuration in afterPropertiesSet. Keeping things simple, I'm just allowing saving a file, deleting a file, and getting a files list.

void afterPropertiesSet() {
    this.mongoSettings = grailsApplication.config.mongodb
    mongo = new Mongo(mongoSettings.host.toString(), mongoSettings.port.intValue())
    if (mongoSettings.bucket == null) {
        gridfs = new GridFS(mongo.getDB(mongoSettings.dbName.toString()))
    } else {
        gridfs = new GridFS(mongo.getDB(mongoSettings.dbName.toString()), mongoSettings.bucket.toString())
    }
}
boolean saveFile(file) {
    def inputStream = file.getInputStream()
    def contentType = file.getContentType()
    def filename = file.getOriginalFilename()
    
    try {
        if (gridfs.findOne(filename) == null) {
            save(inputStream, contentType, filename)
        } else {
            println "Removing old file and uploading new file"
            gridfs.remove(filename)
            save(inputStream, contentType, filename)
        }
    } catch (Exception ex) {
       throw ex
    }
    return true
}

def save(inputStream, contentType, filename) {
    def inputFile = gridfs.createFile(inputStream)
    inputFile.setContentType(contentType)
    inputFile.setFilename(filename)
    inputFile.save()
}

def retrieveFile(String filename) {
    return gridfs.findOne(filename)
}

def deleteFile(String filename) {
    gridfs.remove(filename)
}
    
def getFilesList() {
    def cursor = gridfs.getFileList()
    cursor.toArray()
}

The last piece is to create the controller we referenced in UrlMappings.groovy. The FilesController contains actions for uploading, downloading, and deletion. For brevity's sake, I'm just showing the actions for downloading and uploading.

def downloadFile = {
    def filename = params.filename
    println filename
    def file = gridfsService.retrieveFile(filename)
    if (file != null) {
        response.outputStream << file.getInputStream()
        response.contentType = file.getContentType()
    } else render "File not found"
}

def upload = {
    def f = request.getFile('myFile')
    println f
    if (!f.empty) {
        if (gridfsService.saveFile(f)) {
            redirect(action:'uploadComplete')
        } else {
            
            flash.message = 'Error occured during upload, please try again.'
            redirect(action:'uploadFile')
        }
    } else {
        flash.message = 'An empty file cannot be uploaded.'
        redirect(action:'uploadFile')
    }
}

At the moment, the above code only supports a single bucket but you could easily alter the service and configuration to support more. GridFS support will be a part of an upcoming Grails MongoDB plugin.

Hibernate and JPA with Griffon

Whilst tradional GORM is forthcoming and there is a preview release of the MongoDB plugin with some GORM-like features, there might be a reason why you might need to use Hibernate but not GORM. You might want to slowly evolve an application and work inward from the Swing bits. Using Guice and the warp-persist extension, you can quickly wire Hibernate support into a Griffon application.

I started with the warp-persist-samples DAO example located here and copied all the source files to the Griffon app. It demonstrates normal pre-GORM data persistence patterns with a data access object for each data type.

I put the persistence.xml file into the resources/META-INF directory of the Griffon application and the following in my controller to instantiate the Guice injector and connections to the database (after creating the projectClient object in the model:

 model.projectClient = Guice.createInjector(
				new MainModule(),
				PersistenceService.usingJpa().across(UnitOfWork.TRANSACTION)
						.transactedWith(TransactionStrategy.LOCAL).forAll(
								Matchers.any()).buildModule()).getInstance(
				TestClient.class);
model.projectClient.go();

warp-persist uses guice 1.0 and the griffon plugin uses 2.0 so the plugin isn't needed to enable Hibernate support. While the resulting application lacks GUI interaction, it shows how quickly basic Hibernate support can be added and why that is no longer a barrier to Griffon adoption.

Video of Griffon session at RuPy 2009

The video from my Griffon session at RuPy has been uploaded. I know I said it before but I have to mention again how awesome the organizers were. I'm hoping I can get back to Poznan for GeeCon next May.

 

Check out the video from the session here

MongoDB plugin for Griffon

A couple of weeks ago, I showed how you could make MongoDB a bit more Groovy. In this post, I'll be covering a preview release of the Mongo DB plugin. Behind the scenes, this plugin uses Guice for Dependency Injection. 

Getting started:


Install the mongodb plugin:

griffon install-plugin mongo-db


In your conf directory create a file named MongoConfig.groovy listing the database connection details, like this:

mongo {
    host="localhost"
    port=27017
    dbName="mine"
    collectionName="testcoll"
}

Add the following bit to your Application.groovy file:

guice {
    modules = [ "MongoModule"]
}

MongoModule.groovy is provided by the plugin. Getting a collection Collections are the approximate equivalent to a RDBMS table. Collections will be wh2ere you do most of your work. We can get a reference to a collection by adding @Inject MongoDBCollectionFactory factory to our controller and calling injector.injectMembers(this). Guice uses the settings in MongoConfig to properly wire our Mongo and DB instance giving us a CollectionFactory. Let's make a collection called posts and populate a couple of documents:

def coll2 = factory.getCollection("posts")
coll2.insert(['title':'Whatever', 'postcreated':new Date(), 'comments':[
    'text':'A sample comment','datecreated':new Date(109,10,13)]] as BasicDBObject)
coll2.insert(['title':'Whatever1', 'postcreated':new Date()] as BasicDBObject)
coll2.insert(['title':'Whatever3', 'postcreated':new Date()] as BasicDBObject)

Dynamic Finders

Just before the factory returns the collection, it decorates it with code to generate GORM-like dynamic finders with operators like LessThan, LessThanEquals, GreaterThan, NotEqual, etc. Let's query our collection for all posts with a specific title:

def cursor3 = coll2.findByTitle('Whatever')

The return value is a DBCursor that we can iterate though. In addition to dynamic finders, the MongoDB plugin allows you to access child documents. This is extra useful because Mongo usually stores related documents as child documents of a parent. An underscore between two property names indicates a property on a sub-document. So if you wanted to find the comments created on a specific date. You'd use a query like this:

coll.findByComments_DateCreated(new Date('2009-10-10'))

This plugin, codenamed MonGorm, is a preview release so everything isn't fully baked. Comments/feedback welcome (emails/tweets better than comments on this post)