March 13, 2016

Groovy by example: JSON handling (part 2 of 2)


Pages: 1 2

Serializing / marshalling Java objects to JSON

Rather than building a JSON structure manually using builder invocations, it may be easier to model the structure as a Java class model and simply serialize an instance to JSON.

A similar use case is when you have an existing in-memory Java object and you want to save it to disc (you may call it a “savegame”). Here, the JSON format is a sensible choice as it is standardized, lightweight, and highly human-readable. If these factors are important in your scenario, I would recommend the JSON file format rather than, e.g. XML or some binary format.

From a Groovy implementation point of view, this is a whole different story. Groovy’s capabilities to serialize objects to JSON are there, but they are very limited.

As we just saw, we can print any object in JSON format by initializing JsonBuilder with the object:
Customer customer = new Customer("Max")
    .address(new Address("First Street", "Los Angeles"))
    .purchases([
        new Purchase(100, Date.parse("yyyy-MM-dd", "2016-03-13"))
    ])
    .discounts([(DiscountType.FIDELITY): 5])

new JsonBuilder(customer).toPrettyString();
However, the default result may be unpleasant: For instance, the date format is fixed, and time zone information is lost. There’s no straightforward way to customize JSON output from JsonBuilder. For instance, JsonBuilder will use properties rather than fields as the data source which may lead to undesired side-effects.
{
    "discounts": {
        "FIDELITY": 5
    },
    "address": {
        "street": "First Street",
        "country": null,
        "city": "City of Los Angeles" // undesired
    },
    "purchases": [
        {
            "date": "2016-03-13T00:00:00+0000", // undesired
            "price": 100
        }
    ],
    "name": "Max"
}
Yes, you could write your own solution, but why bother? There are several JSON serialization tools available for Java (and thus Groovy), and they are industry-proven and very fast. Building it yourself violates DRY design.

Here, I recommend Jackson. So far, I found it very easy to use, and it works great with generics and Groovy. But really you can use any JSON serializer of your choice. Just don’t rely on vanilla Groovy in this use case!

For Jackson: Import the maven dependency:
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.7.2</version>
</dependency>
And simply print your object structure:
ObjectMapper mapper = new ObjectMapper();

mapper.setDateFormat(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"))
mapper.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.NONE)
mapper.setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY)

mapper.writerWithDefaultPrettyPrinter().writeValueAsString(customer)
Jackson JSON handling is easily customizable. As shown in the example, you can e.g. globally change the Date format and whether to use fields or properties (getters) for serialization (the former is typically recommended as you are literally saving the object’s internal state). For more precise control, Jackson also comes with a vast amount of annotations which you can use on your model to modify JSON (de)serialization on a per-field or per-class basis.

In the example, the output would be:
{
  "name" : "Max",
  "address" : {
    "street" : "First Street",
    "city" : "Los Angeles",
    "country" : null
  },
  "purchases" : [ {
    "price" : 100,
    "date" : "2016-03-13 00:00:00"
  } ],
  "discounts" : {
    "FIDELITY" : 5
  }
}

Deserializing / unmarshalling JSON to Java objects

You’ll typically want to have two-way serialization / deserialization of your model, i.e. “load” a saved state like this one into in-memory again:
{
    "discounts": {
        "FIDELITY": 5
    },
    "address": {
        "street": "First Street",
        "country": null,
        "city": "City of Los Angeles"
    },
    "purchases": [
        {
            "date": "2016-03-13T00:00:00+0000",
            "price": 100
        }
    ],
    "name": "Max"
}
Yes, you could use Groovy’s JsonSlurper for this:
new JsonSlurper().parseText(TEST_CUSTOMER_JSON_INPUT) as Customer
But again, this solution is far from perfect. First of all, there is again no customization. Most severely, however, Date or enum parsing is not supported and generics are not recognized:
customerJson.address.city // "City of City of Los Angeles"
customerJson.address.@city // "City of Los Angeles"
customerJson.purchases[0].class() // LazyMap
(customerJson.purchases[0] as Map).date as String // "2016-03-13T00:00:00+0000"
customerJson.discounts.keySet()[0] // "FIDELITY"
For instance, in the example where Purchase instances are held in a List<Purchase>, this becomes a List<Map> during deserialization. Because Groovy’s syntax for accessing properties of objects and keys of maps is the same, this may not matter in a truly dynamically compiled environment. But if you want to @CompileStatic, this becomes a major issue.

Thus, again I recommend using an existing JSON serialization framework like Jackson for this use case.
ObjectMapper mapper = new ObjectMapper()
Customer customerJson = mapper
    .setDateFormat(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"))
    .readValue(TEST_CUSTOMER_JSON, Customer)
This really is all you need and you get full support for Date and enum parsing and generics:
customerJson.address.city // "City of Los Angeles"
customerJson.address.@city // "Los Angeles"
customerJson.purchases[0].class() // Purchase
Date.parse("yyyy-MM-dd HH:mm:ss", customerJson.purchases[0].date) // "2016-03-13 00:00:00"
customerJson.discounts.keySet()[0] // DiscountType.FIDELITY
And it works really well with Groovy.

Conclusion

Groovy comes with great out-of-the-box support for everyday JSON parsing / writing tasks. However, it clearly is not a full-fletched JSON serialization framework. Yet this isn’t a deficiency as there are excellent third-party JSON serializers available. The important part is that you realize which use cases are best implemented using vanilla Groovy, and for which ones using an external framework makes sense.

Personally, I really enjoy working with the Groovy / Jackson duo, most certainly from an ease-of-development perspective.

In the accompanying GitHub repository, you’ll find JUnit tests showcasing JSON parsing and writing with Groovy and Jackson in action.

Please let me know in the comments section whether you agree with this article or if you have a different opinion on this topic. Also let me know if I missed any important aspects of Groovy JSON handling.


Pages: 1 2

Groovy by example: JSON handling (part 1 of 2)



You’ll find a lot of simple examples of dealing with JSON in Groovy on the web. In this article, however, I’ll try to provide a use-case driven collection of recipes, showing you when to apply which specific technique to solve your JSON-related problem.

This article covers:
Throughout this article, I will use a simple example domain model which serves as the structure for our JSON data. Here are the individual Groovy classes (AST transformations omitted):
class Customer {
    String name
    Address address // to-one-relationship
    List<Purchase> purchases // to-many-relationship (generic)
    Map<DiscountType, Integer> discounts // map (generic, with enum keys)
}

class Address {
    String street
    String city
    String country
    
    public String getCity() {
        return "City of $city"
    }
}

class Purchase {
    int price
    Date date
}

enum DiscountType {
    FIDELITY,
    EMPLOYEE,
    EU,
}

Parsing / reading JSON

In the most simple case, you just want to read an excerpt (maybe even just a single property) of a JSON String.

Here’s a example JSON String:
[
    {
        "name": "Max",
        "address": {
            "street": "First Street",
            "city": "Los Angeles",
            "country": "US"
        },
        "purchases": [
            {
                "date": "2016-03-13 00:00:00",
                "price": 100
            },
            {
                "date": "2016-03-13 00:00:00",
                "price": 200
            },
            {
                "date": "2016-03-13 00:00:00",
                "price": 300
            }
        ]
    },
    {
        "name": "Sarah",
        "address": {
            "street": "Second Street",
            "city": "San Francisco",
            "country": "US"
        },
        "purchases": [
            {
                "date": "2016-03-12 00:00:00",
                "price": 200
            }
        ]
    }
]
You can parse this String easily using Groovy’s JsonSlurper which supports Groovy’s property access syntax (GPath notation), resulting in highly readable code.

Get the root element (here it’s a list):
def root = new JsonSlurper().parseText(TEST_CUSTOMERS_JSON)
And then access child nodes:
root[0].name // "Max"
Because node collections are iterable, you can also invoke Groovy’s iteration methods on them:
root.collect {customer -> customer.purchases.findAll {it.price > 200}}.flatten()*.price // [300]
Note that this method of accessing a JSON structure is designed for use in dynamically compiled code only. Otherwise, above code segments would get seriously bloated as you would have to cast to Collections or Maps in order to make the compiler happy.

Writing JSON

Groovy also comes with built-in tools to write dynamically or statically defined JSON using the elegant Groovy builder syntax.

Create a JsonBuilder:
def builder = new JsonBuilder()
And then build the structure:
builder.call(
    [ 
        builder (
            name: "Max",
            address: address(builder, "First Street", "Los Angeles"),
            purchases: (1..3).collect { num ->
                builder (
                    date: "2016-03-13 00:00:00",
                    price: num * 100,
                )
            }
        ),
        builder (
            name: "Sarah",
            address: address(builder, "Second Street", "San Francisco"),
            purchases: [
                builder (
                    date: "2016-03-12 00:00:00",
                    price: 200,
                )
            ]
        )
    ]
)
whereby address(…) is
private static address(def builder, String street, String city) {
    return builder (
        street: street,
        city: city,
        country: "US"
    )
}
There are a few advanced techniques you can use to make your builder more powerful:
  • Use structural programming facilities to build a structure dynamically (in the example, we build Max’s purchases dynamically from an int range).
  • Use helper methods as “templates” to keep your code DRY compliant (in the example, this is implemented by the address(…) method).
(I have explained a bunch of these techniques in another blog post which deals with Groovy builders in general.)

Finally, pretty-print the output JSON String:
JsonOutput.prettyPrint(builder.toString())
The example will result in the JSON output depicted in the previous section.

Transforming / manipulating JSON

In this case, the input as well as the output is JSON. There’s no need for an intermediate Java object representation of the model.

This really is most easily achieved by subsequently parsing JSON, manipulating the returned structure, and then writing it as JSON using the techniques discussed above.

You can use any Groovy facilities to do the model restructuring.

This code example will change the “US” country code to the “USA” code, and add additional currency information to a customer:
def root = new JsonSlurper().parseText(TEST_CUSTOMERS_JSON)

root.findAll { customer -> customer.address.country == "US"}.each {
    it.address.country = "USA"
    it.info = [currency: "USD"]
}
In the final step, we initialize the JsonBuilder with the manipulated JSON node structure and simply print it again.
def builder = new JsonBuilder(root)
JsonOutput.prettyPrint(builder.toString())


Pages: 1 2

March 1, 2016

One year later...


Dear reader,

Exactly one year ago, I started this blog, codebulb.ch, as a place to collect and share my knowledge and experience as a Java software engineer.

I’m happy to find through your feedback that the articles in this blog proved interesting and useful for a worldwide audience. I really enjoy maintaining this blog both as a means to record my own knowledge, but also as a motivation to delve even deeper into subjects I want to learn more about myself.

You may have noted that of late, the thematic focus of my blog shifted slightly from in-depth experience reports in my main area of expertise (JSF, Java EE, Groovy) to short tutorials and exploration of up-and-coming technologies (AngularJS, Node.js), and to short presentations of open-source projects I am working on. I really find it much more challenging and interesting to study new subjects, and to report my journey on my blog rather than discussing all over what in general is already common knowledge.

In the last few weeks and months, I have particularly enjoyed working on my open source projects, and I plan to continue doing so over the next weeks, even if this means that I will for a moment put less focus on writing blog posts.

Let me however stress how much I enjoy running this blog, and that I have no plans of stopping it or changing its technological focus in the long run.

I thank you for your continued interest. Please check out my GitHub repository to see my latest open source projects, and follow me on twitter for any updates.