How does the DRY Principal relate to a heavily layered architecture?

Today I’ve been thinking about how the DRY Principal (Do not Repeat Yourself) applies to an SOA or heavily layered architecture. A colleague of mine pointed out that our architecture violates this principal quite a lot, we didn’t really get the chance to dig into it, but I’ve been thinking about it quite a lot since.

The scenario that we’re dealing with is a shared database (shared with legacy apps). A shared back-end WCF service (split into many services, each with a service, business and data access layer). And several client application. With an architecture like this I think it’s very easy to fall into duplication, but it’s also worth differentiating between logic and definition.

As far as logic goes I think we’re pretty good at DRY; any real business logic goes in the WCF services and the clients are (in theory) pretty thin. When it comes to definitions I think we’re repeating ourselves a lot, but I’m not sure if this is necessarily a bad thing, because as soon as two areas of code share the same definition of something they’re both bound to that definition and therefore to each other (at least in a small way).

So let’s take an example of a definition of an aircraft that’s ‘repeated’ in several places, and in each place see if we can justify ‘repeating’ the definition:

1. Database – The aircraft exists in the database, let’s take this as our initial declaration
2. Data access – In the WCF service the aircraft exists in a dataset xsd – this acts as the mapping definition between the database and the business entities and is pretty much autogenerated anyway.
3. Business entity – In the WCF service the aircraft is represented as a POCO business entity, this is seperate from the data access so that the business logic is decoupled from the pesistence technology. It’s also a much richer representation of the data that can be manipulated in code.
4. Translator – Although not strictly it’s own definition a translator class translates between the business entity and the DTO, and ends up having to ‘know’ about the definition of an aircraft
5. DTO – At the service layer the aircraft is represented as one or more data contracts. This is the contract that the WCF service exposes, therefore it’s a good idea to seperate it from the business entity so that the internals of the service can change without necessarily modifying the contract. Also a DTO is just for transferring the data, whereas the business entity can be a much richer representation.
6. Client business entity – Some of the clients convert the DTO into a locally defined business entity, this is because some of the clients have quite rich behaviour themselves and decoupling from the DTOs (that are shared with the service and other clients) is advantageous.
7. Client side translator – again a translator is needed to convert between the client side business entity and the DTO.
8. View – Sometimes the UI needs a slightly different representation of an aircraft to display slightly different things, this occasionally leads to one last definition of aircraft.

Looking at the list it does initially look like we’ve repeated the definition a few too many times, and I’d probably concede that there are a few too many definitions; however none of the definitions are a straight forward repeat, each definition is there to provide decoupling, and is usually slightly different from the other definitions, even in cases where the definition is the same this is just ‘coincidence’ – as requirements change the different layers can diverge in what they think an aicraft looks like.

There are some places where I can see potential for reducing duplication, I think the two translators could be replaced with an automapper. The dataset could be replaced with a proper ORM, and possibly one that used convention over configuration.

So in summary I think that the DRY Principal is ‘good’ but that it needs to actually be thought about, and there are times when you want to ‘repeat’ yourself for another reason such as achieving decoupling.

Translation Tester, part 3 – Requirements

In the 3rd part of the Translation Tester series I intend to come up with some initial requirements for the product. This will then be used to create a product backlog and drive development via ‘Test Driven Development’. I’m going to try and express the requirements as ‘user stories‘.

The users

First a quick introduction to the users; I realise that I should probably have some real users providing my user stories, and in due time I hope to add to my initial user stories with real end-user stories.

The Traditional Developer is a developer who writes code and then writes some unit tests afterwards to test parts of their code.

The Test Driven Developer uses tests to drive the design and development of their code

The term Developer will be used to refer to all developers.

The Development Manager manages a team of potentially changing developers over a period of times, potentially on several projects.

The Build Engineer manages and monitors a product’s automated build.

The Stories

So without further ado lets start writing some stories, although they’ll probably go from simple to complex no prioritisation or ordering is intended at this stage.

As a Developer
I want to test my translator classes
So that I have confidence that they work

As a Developer
I want to specify that a property should not be translated
So that I can have minimal classes

As a Developer
I want to be able to clearly see properties that were excluded from the translation
So that as requirements change I can easily identify the changes to be made and bug fixing should be easier

As a Developer
I want to know when my code change has broken another area of the code
So that I can have confidence in making changes without the risk of regression

As a Developer
I want to reduce the amount of ‘boilerplate’ code for testing translators
So that I can work more efficiently

As a Developer
I want to be able to specify complex translations where necessary
So that as much as possible of the translation can be tested

As a Developer
I want the tests for my translator to specify the desired output, not how the output is achieved
So that my tests are more robust, and don’t just duplicate the production code

As a Developer
I want each test to test one aspect of the translator
So that a failed test clearly indicates the reason for the failure and other failures are not hidden

As a Developer
I want to exercise the translator with a wide range of inputs
So that I can test the translator handles a wide range of inputs correctly

As a Developer
I want to be able to use a mocking framework to test that a call was made as part of the translation
So that I can test parts of the translation that do not easily allow for state based testing

As a Developer
I do not want to be forced to use a specific mocking framework
So that I can work the way I (and my team) work.

As a Development Managers
I do not want to be tied to any other dependencies (such as on a specific unit testing framework)
So that I can choose what tools the team will use and change my mind in the future

As a Build Engineer
I want to automate running of the translation tests
So that I can be notified of a failure without any manual intervention

As a Test Driven Developer
I want to be able to start from a small skeleton translator and test class
So that I can get quick feedback on my development

As a Test Driven Developer
I want each small test to indicate the work that must be done in the translator to make the test pass
So that I can use the tests to drive my development.

Translation Tester, part 1 – The problem

Introduction

This post is the 1st in what may become a series of posts detailing the development of a test support tool for testing ‘Translators’. I’m planning on blogging the development of this tool for a few reasons. One reason is to try and gauge if there is any interest in the tool (may impact whether I bother to continue development and maybe even extending it to a releasable open source product). Another reason is so that if anyone out there thinks there is a tool out there that already does this they can tell me and save me a whole load of effort

Some useful information before I state the problem…

It may be worth me starting by explaining what I mean by a ‘Translator’; When you have a layered architecture in a software product/suite you often have de-coupled representations of concepts for each layer rather than having a common set of representations for all the layers. This means that whenever you cross a layer boundary you need to convert from one representaion of the data/concepts to another, the class that does this conversion I would call a Translator.

Entity Translator diagram

Modern software development seems to be more and more layered, and with some concepts like Domain Driven Design (DDD) and The Onion Architecture advocating decoupling different parts (or layers) of an application from each other, it seems to be more and more prevalent.

The problem

The problem I’ve found with Translator classes is that they can be pretty difficult to unit test reliably, flexibly and effectively.

1. Unit tests often seem to over-specify the behaviour and are pretty much just directly duplicating the translator code.
2. Writing unit tests for translators can be long and laborious with seemingly little benefit, it’s almost mindless coding. However a lot of bugs seem to be due to translators, so they obviously do need to be tested.
3. When a new property/field is added to one of the types being translated from/to the test will often continue to pass, and there will be no indication that the translator is not attempting to translate the field.
4. An untranslated property often doesn’t cause an immediate problem (compilation failure, unit test failure, immediate run-time failure). Rather the missing data will show up some time later as a hard to diagnose bug.
5. When a bug is identified it can often be traced back to a translator not mapping a field, when you have several layers each with each layer boundary having a translator there can be a lot of places to look!
6. Over time translator tests go out of date, and not even due to any changes of the class being tested changing (the translator), rather that one of the types being translated has changed.