Using Domain Specific Languages to Implement Interactive Frontends
For many years we have dealt with the challenges that frontends with interactive forms pose w.r.t. validation, test data and quality. Describing the requirements in formal Domain Specific Languages (DSL) became the way of choice to create a specification that gives a twofold benefit: first, the customer understands it better, and secondly, the software engineers use the specification not only to implement more resilient software, but also to improve quality assurance. This new series will explain how we do it and why we think it’s the best approach.
We develop many applications for e-government, finance and insurance. These usually rely heavily on the presentation and evaluation of data processed within interactive frontends. Legal and business reasons demand that the resulting applications meet very high standards of reliance and safety. For us, this meant that we had to find a way to work close with the customer to understand the domain while simultaneously improving coverage and efficiency of development and QA.
We met this challenge by using Domain Specific Languages to involve the customer in the creation a specification basis for the domain which could be used by our engineers not only as requirement description but also for the implementation of tools for automatic validation or even code generation. The work spent on the definition, development and maintenance of specification language tools proved to be worth it as quality improved and effort decreased. E.g. while we were analyzing and reporting test coverage in terms of project specific formalized test data, we were glad that we could rely on a resilient specification basis.
The proven benefits of formal specifications at the comprehension level of customers encouraged us to bridge the gap in more mgm projects: we motivate more customers to introduce a formal requirement process and more engineers to a domain specific, though formalistic view of requirements so they can benefit from the improvements.
This first blog article introduces our domain specific formal language approach. The upcoming second part will focus on quality benefits derived from formal specifications.
Setting the Scene
The quality of complex interactive applications always includes a high level of conformity to the respective customer’s requirements. It is not enough to have a well-functioning system; the system also has to do what the customer expects! This holds for both, explicit and implicit (sometimes never formulated) requirements. We believe that the only way to accomplish this is to learn from the customer and to struggle for exact and comprehensive requirements consistent with the customer’s domain.
Requirements modeled by Domain-Specific Formal Languages
We believe that the need for exact requirements can be met by using formal languages to describe requirements. To model and implement what the customer needs (and not what we believe that he or she needs), the specification must be formulated using the terms of the respective domain and be comprehensible for the customer. We achieve this through the adaptation and extension of our existing language family to support the customer’s particular domain.
Requirements modeled for Frontends – at least
It is common sense to separate front-end aspects (such as input and output handling for forms) from the full business logic modeled somewhere in the back-end. The main purpose of front-ends is to deliver to the back-end information which is guaranteed to be sound and consistent. The back-end implementers should not be bothered by interactive aspects and user level data consistency.
mgm technology partners’ approach to form-centric applications was to implement a framework supporting specification languages along the full software development chain: editors, code generators, documentation and test data generators for the supported language family.
- We use and promote formal specification languages for front-ends and part of the business logics.
- We involve customers in the requirement process, especially through including formal specifications for their domains.
- We tailor the specification framework to each of the customer’s specific application domain.
This yields a high return (for customers and mgm) in terms of delivery time and quality and, in addition, reduces the implementation and quality assurance effort.
The benefits of using formal front-end specification techniques are manifold:
- Consistency of the customer requirements: It becomes very likely, that the system accepts and denies exactly what the customer expected (including implicit requirements which would have never been formulated in absence of formal specifications).
- Reduction of programming effort and risk: Since formal specifications allow for automatic code generation, a great deal of programming effort and risk simply vanishes.
- Increasing functional quality standards for the front-end: Obviously, generated code is consistent to specifications (once the code generator is mature and tested). Moreover, due to the formal description of legal inputs, test suites describing valid inputs are automatically generated.
- Driving the tests of the back-end: Since the front-end specifications define the set of correct inputs, they serve as basis for tests for the back-end. Automatically generated test suites can be run which prove the quality of the back-end in a well defined setting. Test coverage goals for the back-end can be formulated in terms of the existing formal front-end specifications for the front-end. Test coverage becomes measurable in terms of front-end data and front-end use cases.
In this blog post we focus on the topics 1) and 2) described above. We do so by presenting a flavor of the used specification languages. Aspects 3) and 4) will be discussed in a second blog post.
Specifications supporting customer-related formal Requirements
In the following we will demonstrate how to derive a formal specification using mgm’s specification language.
An User Interface Example
Imagine a simple bill writing system based on a simple (e.g. web graphical) user interface where we will also need to consider both, calculations and consistency constraints.
Let us assume that a number of bill positions with a net unit price can be typed in, each of them with its own multiplicity (quantity) of at least 1. The system calculates the net price for each position as well as the baseline including the default or specific VAT (value added tax).
The input form might look as follows (we ignore GUI details in this example):
Input fields are marked yellow and fields which are to be calculated by the system (and thus are “read-only” for the user) are marked blue.
A completed form for two positions, with reduced VAT would look like follows:
Formal Specification for the Example
The specification language is now used to describe the properties of the fields, i.e. their types, the relations between their values and other constraints regarding consistency and completeness in a formal way. This approach is the core of the formalization of requirements.
Let us look at some formal specifications for the example above.
Depending on the domain, fields have domain-specific types at user level. These types can be specified as numbers, currencies, date data, strings, enumerations and truth values. Most of them also include a specification of the allowed field length. For instance
AlternativeVat is a positive numeric value with 2 digits which leads to the specification:
Some fields (e.g. all position related fields in the example) might have multiple instances. This is also specified along with the type definition. Multiplicity is specified as follows by using
multi Position: String(25) multi UnitPrice: EurosAndCentsDigits(8) multi Quantity: PositiveInteger(3) multi PosFullPrice: EurosAndCentsDigits(10)
Fields values which are either subject to calculation by the system or which are given or by the specification are marked as
constant, respectively. Both kinds are not editable by the user (they are “readonly”).
calc NetAmount: EurosAndCentsDigits(10) constant NormalVat: PositiveNumberDigits(2) = 19 calc AllVat: EurosAndCentsDigits(10) calc multi PosFullPrice: EurosAndCentsDigits(10)
Functional rules specify values in a functional setting.
For the fields
GrossAmount one gets straight-forward specification formulas:
AllVat = If FieldValueSpecified(AlternativeVat) then AlternativeVat/100*NetAmount else NormalVat/100*NetAmount GrossAmount = AllVat + NetAmount
The semantics of these rules is intended to be functional rather than imperative, i.e. a field value is not influenced by any state aside from the occurring field values (Similar to formulas in spreadsheets or functional languages). Note that functional rules can also be used to specify mappings within the user interface, e.g. from drop-downs to textual representations.
Constraint rules describe constraints which have to be satisfied to ensure consistent inputs. They consist of
- a conditional clause specifying the constraint, which the field values have to satisfy, and of
- a fail-clause including a message which shall be issued if the constraints are not satisfied.
In this example, only specific VATs can be considered: normal VAT, half normal VAT or no VAT at all.
constraint AlternativeVat == 0 or AlternativeVat == NormalVat or AlternativeVat == NormalVat/2 => failed: "VAT can only be normal, half normal or zero"
Note that the semantics is similar to assert statements: the failed-clause expresses feedback in case the conditional clause is not fulfilled.
Note further the differences between functional and constraint rules:
- Unlike functional rules, constraint rules specify constraints (operationally expressed: checks) rather than computations. For syntactic differentiation, we use “
==” here rather than “
- Functional rules do not have fail-clauses since they enforce values for fields rather than checking their relationship.
Rules for Multiple Instances
The language supports multiple instances of fields (and rules therefore) along two multiplicity dimensions – “All” and “Each”.
All-multiplicity for functional rules
For computations such as for the field
NetAmount multiple instances have to be considered
NetAmount = Sum(PosFullPrice.all)
.all” next to
PosFullPrice stands for an “all instances” semantics. We call this “all” multiplicity.
Each-multiplicity functional rules
Each instance of
PosFullPrice is calculated in a homogenous way by multiplying the price with the quantity.
PosFullPrice.each = UnitPrice.each * Quantity.each
This functional rule yields for each row (i.e. instance), respectively and is denoted with the postfix “
.each“. We call this each multiplicity.
This defines that the rule holds for each multiple instance of involved fields. Intuitively, one can view this as multiple copies of the rule for each instance.
Each-multiplicity for constraint rules
Consistency constraints for multiple instances of fields are expressed here as well.
In the example, it must be guaranteed, that each position (row) is fully specified. This is expressed with the specific predicate
FieldsCommonlyDefined (using “each” multiplicity).
constraint FieldsCommonlyDefined(Position.each, UnitPrice.each, Quantity.each) ==> failed: "All fields (Position , UnitPrice, Quantity) must be specified if one is specified "
All-multiplicity for constraint rules
Obviously, “all” multiplicity is usable in constraint rule as well. In the example, since the system should not print empty bills, one has to specify multiplicity greater than zero at least for one of the fields referring to positions.
constraint AtLeastOneInstanceExists(Position.all) ==> failed: "Please specifiy at least one position"
This completes the example showing some of the specifications means. The full specification is given in the appendix. Since it is complete it can be used for automatic code generations for computations and constraint checking.
The aim of this short example is neither syntactic accuracy nor completeness. It is merely an illustration of important aspects. Please remember that each language of the Specification Language family is tailored specifically anyhow.
Summary: Characterization of the Specification Language Family
In a nutshell, the specification language family described above defines valid inputs and front-end computations for web applications or other form based systems. The formalism can be characterized as a subset of typed predicate calculus defined in terms of the customer domain. For the sake of simplicity and comprehensibility, we do not extend to full predicate calculus or to higher order logic. The pragmatic expressiveness of the language family is more important than the theoretical power of predicate calculus.
Here is a summary of the aspects which can be described.
- Typed field: Valid and invalid field value using field description.
- Field-value-constraints: valid and invalid values for related fields.
- Existence-constraints: validity and invalidity of existent and non-existent input value depending on values or existence of input value for other fields.
- Both kinds of constraints can be interwoven.
- Computation of field values based on other field values. This is expressed by rules including:
- mapping of external presentations to internal ones (such as mapping drop down selections to values),
- constant values which show up in the user interface,
- conditional values.
- Multiplicity: fields may be defined to occur in several instances, controlled by the specification. In terms of constraint control this allows to express iterative aspects on a higher level than for single fields. There are multiplicity aspects expressible by the specification language family (not needed in this example). This, however, is beyond the scope of this paper.
And here’s the full specification example:
# Fields and Types: multi Position: String(25) multi UnitPrice: EurosAndCentsDigits(8) multi Quantity: PositiveInteger(3) multi PosFullPrice: EurosAndCentsDigits(10) AlternativeVat: PositiveNumberDigits(2) constant NormalVat: PositiveNumberDigits(2) = 19 calc NetAmount: EurosAndCentsDigits(10) calc AllVat: EurosAndCentsDigits(10) calc multi PosFullPrice: EurosAndCentsDigits(10) # Functional Rules: AllVat = If FieldValueSpecified(AlternativeVat) then AlternativeVat/100*NetAmount else NormalVat/100*NetAmount GrossAmount = AllVat + NetAmount NetAmount = Sum(PosFullPrice.all) PosFullPrice.each = UnitPrice.each * Quantity.each # Constraints: AlternativeVat == 0 or AlternativeVat == NormalVat or AlternativeVat == NormalVat/2 => failed: "VAT can only be normal, half normal or zero" FieldsCommonlyDefined(Position.each UnitPrice.each, Quantity.each) => failed: "All fields (Position , UnitPrice, Quantity) must be specified if one is specified " AtLeastOneInstanceExists(Position.all) => failed: "Please specifiy at least one position"
Development and Testing issues
Due to formal specifications, both, software development and quality assurance become less cumbersome, more comprehensible, easier to track, more scalable and safer. We conclude by shortly sketching these aspects.
Avoiding Implementation Efforts and Risks through Code Generation
Similar to programming languages, specification languages substantially simplify software development. Code generated from specifications eliminates a great deal of complexity and leads to less error-prone systems. Both functional rules and constraint rules can be used as input for generators to generate fully operational code. This is a highly efficient way to automatically produce components for the validation of data in a complex solution environment.
Test Data Generation and Test Coverage
Since the used specification languages describe well defined inputs they are an ideal prerequisite for test data generation: consistent test cases are generated directly from the specification due to
- the existence of type definitions for fields,
- the existence of constraint rules describing the relationship between fields,
- and the existence of calculation formulas for calculated fields values.
Test data generation occurs with few exceptions fully automatically. For specific test cases it is also possible to automatically generate inconsistent data by deliberately enforcing wrong data. This is done by negating constraints and re-running the test data generator.
mgm technology partners has developed a test data generator suite based on the used formal languages. The tool is used for the generation of big and complex test cases allowing for extensive and well defined test coverage. Some insights can be found in our blog articles “Form Validation with Rule Bases“, and in “Producing High-Quality Test Data“. Test coverage and other quality assurance topics issues will be highlighted in the upcoming second blog article.