Category ‘Projects’

How Google’s Geocoding solves Address Validation

For the e-commerce business, it is very important to have a valid and correct address base. Sending packages to incorrect addresses causes significant losses, since packages are not delivered and sent back. In the case of the Kickz online shop, we faced exactly this problem. Thus, we thought about how to improve and force users to enter correct addresses. Learn how we used Google’s Geocoding service to solve this problem.

Read the rest of this entry »  |   Share

Rapid Prototyping with Spring Roo

Spring Roo, a new contender to Grails and Ruby on Rails, provides flexible scaffolding for Spring based applications. We used Spring Roo in a web project to quickly provide working software in order to discuss features and gather feedback. This article shows how we generated an early prototype and transitioned to early development and then to production code.

Read the rest of this entry »  |   Share

Non-Standard Ways of using Lucene

For our recent online shop project, we required a full-text, multi-criteria product search. Lucene, the popular Java search engine, is an ideal candidate for this functionality. But in order to meet the high performance requirement, we had to extend its usage beyond standard full-text search. This posting describes our solution including index switching and using Lucene as a simple NoSQL database.

Read the rest of this entry »  |   Share

Why Functional Tests don’t belong in a Build Environment

The previous part discussed why a unit test for a class should be written by the developer of that class, and why a functional test should be created by an independent tester. This posting argues that functional tests should not be part of the build process of the product, but instead should be developed and executed separately. For this, I give guidelines for setting up an independent validation system.

Read the rest of this entry »  |   Share

Portlets with XForms in Liferay

This posting discusses how we integrated an web application with XForms as one of the main view technologies into a Liferay portal. We were using the Orbeon 3.8 XForms engine, which was introduced in the previous part of this series, and Spring Portlet MVC. Since we were facing quite a few problems and issues regarding Orbeon and Liferay, I go to great length discribing our working solution.

Read the rest of this entry »  |   Share

Producing High-Quality Test Data

This part addresses the question what makes test data valuable for functional tests. You will understand the important concept of extreme and special values, and how to obtain test data that is highly compressed and also attains a high test coverage. The article also explains our novel idea for constructing a generator for such high-quality test data.

Read the rest of this entry »  |   Share

Developers, Don’t Write Functional Tests!

Over the past few years I have noticed that the distinction between functional tests and unit tests has blurred in a lot of projects. I think that using the features of modern testing frameworks like JUnit and TestNG to push functional tests into unit tests is the wrong approach, because it shifts the focus of the test from the test perspective to the development perspective. In this blog post, I explain in detail how I have come to this conclusion.

Read the rest of this entry »  |   Share

Form Validation with Rule Bases

A lot of our web applications contain a large number of forms with hundreds of fields and complex cross-field constraints. mgm’s quality assurance team uses rule bases and automatic form validation to verify the correctness of these apps. This blog series discusses the challenges in generating test data for this verification and explains our automated process for producing masses of test data by utilizing the rule bases.

Read the rest of this entry »  |   Share

Test-driving XForms with Orbeon

In this second installment, we conclude our evaluation of XForms implementations and explain why we ultimately preferred the Orbeon engine. I walk you through a XForms sample to explain essential concepts. My team has spent quite a lot of time understanding the Orbeon architecture and I discuss our findings, including separate-deployment configuration and state handling.

Read the rest of this entry »  |   Share

Managing distributed Solr Servers

We use the open-source search server Solr for real-time search on data stored in a Hadoop cluster. For our terabyte-scale dataset, we had to implement distributed search on multiple Lucene index partitions (shards). This article describes our solution to manage 40 independent Solr instances without human interaction, including transparent failover and automatic backup of the Lucene index.

Read the rest of this entry »  |   Share

Building Complex Multi-Platform Projects with Hudson

When building multi-platform projects where C++ is involved, developer teams are confronted with the task to compile and run the project on different platforms with different compilers. Besides, maybe just a part of the project is written in C++, other parts are written in Java and an automated build process has to be established including both languages. In our project we have found a solution based on Hudson.

Read the rest of this entry »  |   Share

Realtime Search for Hadoop

In the previous part of this article series we focused on the efficient storage of log data in Hadoop. We described how to store the data in Hadoop’s MapFiles, and we tweaked the configuration settings for increased data storage capacity and greater retrieval speed. Today, we discuss how to perform near real-time searches on up to 36.6 billion log messages by a clever combination of Hadoop with Lucene and Solr.

Read the rest of this entry »  |   Share

Storing log messages in Hadoop

In part 1 of this article series we described the various challenges of dealing with large amounts of logging data in a heavily distributed software ecosystem. After evaluating different approaches, we quickly selected Hadoop as the technology of our choice. In this article we will describe how some pitfalls we had to solve when using Hadoop to store log messages.

Read the rest of this entry »  |   Share

See You, SQL – Hello Hadoop

Our team has developed a system for storing and processing huge amounts of log data using Hadoop. The challenge was to handle Gigabytes of log messages every day and enable querying the 30+ Terabyte archive with instantaneous results. In this first part of our blog series we explain our motivation for using Hadoop and contrast this solution with the traditional relational database approach.

Read the rest of this entry »  |   Share

Why XForms might be a Winner

mgm develops several large scale web applications containing hundreds of forms, e.g. the e-government portal ElsterOnline Portal. Design, implementation, test, and maintenance of these forms implies significant effort. We are currently evaluating different technologies – including XForms, Wicket and JSF (ICEfaces) – that could help us to reduce these efforts, to improve quality and set the framework for our next generation web forms-centric applications.

Read the rest of this entry »  |   Share