August 30, 2015

Why “creative” EE developers are bad for business



In this article, I’d like to address an issue I’ve met in most companies I worked for so far: It’s apparently a common anti-pattern for management to expect and encourage their top enterprise developers to come up with preferably “creative” solutions. Here, I’d like to show why I think adhering to “common” solutions is actually preferable.

What is “good” software?

“Good” software, i.e. software which is a valuable asset for the development company that creates it, will typically adhere to these two characteristics:
  • It fulfils the requirements imposed by the customer, both functional and non-funcional.
  • It is maintainable.
For typical enterprise application development / customer software development, the “maintainability” factor cannot be stressed enough. Experience shows that maintaining existing parts of a software system (changes, bugfixes, additions based on existing functionality) takes up more time over the lifespan of a product than adding completely new functionality. Leading opinions in the field of software development prove this.

So, as a thought experiment, is it better to create a nearly-perfect product but with diminished maintainability, or a highly maintainable product with diminished overall functionality? Of course, neither of these situations is ideal. However, in case of a high level of maintainability, you still have the possibility to change, to adapt. This is absolutely essential especially in typical custom software development where “development” spans over many months, is hardly ever finished or if you plan to support your product afterwards. Especially in the case of agile software development, maintainability is key to be able to adopt to changing requirements.

What is “maintainable” software?

High software maintainability can be achieved by:
  • Increasing source code readability: making it easy to find out what a piece of code does.
  • Increasing source code extensibility: making it easy to adapt existing software and extending it.
  • Providing good documentation
  • Providing good software tests
In this article, we focus on source code readability and extensibility aspects. There are many articles elsewhere on the Internet which go into more details about how to provide “good” documentation or tests.

Both source code readability and extensibility are facilitated by adhering to common best practices for the programming language and the environment in use, such as globally accepted source code conventions and well-known design patterns.

This is the actual “engineering” process in software engineering: Identifying a common pattern to find a specific solution for a concrete problem. The more generic the underlying idea is, the easier it is for any third person to later grasp the concrete adaptations of the solution applied to match the concrete problem.

What is a “good” solution?

A good solution is based on generally accepted best practices and well-known patterns, but adopted just enough to match the specific problem. It should solve the concrete problem, not more (“overengineering”), and not less.

A solution which is based on commonly accepted best practices is expected to be generally accepted as well, which is critical for any piece of software developed in a team. Even more important, though, is the aspect of distribution of knowledge: a solution which is commonly accepted and commonly known facilitates any aspect of software maintenance management because less specific documentation, training, or even design discussions are needed because we use what has already been taught on and agreed on by the team and even potential new members of the team.

To sum up, such a solution has these characteristics:
  • It requires no or little specific documentation or training as the underlying ideas are well-known.
  • It requires less research and testing as the underlying design / technology is proven to be robust and effective.
  • It diminishes the need for discussions and lessens the risk of conflicts within the team as the underlying pattern is generally accepted as a “best practice”.
  • It makes the software / project / company more independent on the individual engineers which develop the software because knowledge about the underlying ideas is (ideally globally) widespread. Depending on individuals is a huge risk no sane company should take.
Let me give you some examples of applying a “good” solution vs. applying a “bad” solution. (These are real life examples of what I’ve seen in practice.)

Library usage

Example: Creating a custom Java library e.g. for Date conversion vs. relying on an existing (3rd party) library like e.g. Joda time.

There’s a huge, undesirable shift of responsibility towards your project: By building a library from scratch, you (i.e. your development team) is responsible for accuracy (testing!) and documentation of the library all by itself. As the library is developed “in-house”, no potential new member of the team is expected to have any prior knowledge of the library, thus the need for training is increased.

Think about it: Apart from developing the actual business software, can you really effort to stem an additional project to build an entire utility library? Probably not. In this situation, you better rely on existing products, using them as “black boxes”, i.e. without worrying about their accuracy or other aspects: as long as they prove their correct functionality through tests, you can just rely on their team to make its job. Ideally, you could even find a library which offers business support.

Technology usage

Example: Creating a custom solution for “dynamic” web UI generation through “static” technology vs. relying on existing solutions like e.g. PrimeFaces Extensions DynaForm.

If you feel like you have to “bend” existing technology near the point of “breaking”, e.g. when code readability and (IDE) tool support is diminished, you’re most likely on the wrong way. It then typically helps to either search for help online, hoping that someone knows a neat solution to solve the problem more or less elegantly within the existing technology; otherwise, you should even consider breaking out of the existing technology and build a sub-system based on another mature technology / library.

Using technology in a way which seems clever at first sight, but for what it is clearly not designed will most likely quickly get out of hands as soon as more advanced requirements come up and the solution gets more complex. At that point in time, it shifts from creating a sub-system to creating your own library, as described above, but typically without ever having clearly separated technology and business aspects of the solution, making it very hard to test and maintain, and requiring specific knowledge of both the business aspects as well as the application of the solution itself in order to work with the code base.

Having built your own little “framework”, and especially if it’s brittle, is especially frustrating for new developers, unable to bring in their own experience with mature technology, but required to dive in into your custom solution which will most likely not stand the test of time anyway.

Middleware usage

Example: Creating a custom mini-rule engine vs. relying on an existing rule engine product like e.g. Drools.

Re-inventing the wheel is always a no-go in software engineering. If there’s a solution which matches your requirements, go for it. It’s tested, it’s documented, it’s efficient, and there’s potentially even professional support, maybe it’s even open source. Even if you feel like the solution exceeds what you actually need as functionality, you’re on the safe side: If requirements change or extend, they are still most likely covered by the 3rd party product. Everything which holds true for relying on existing libraries or technology applies here as well.

But what if there is no “good” solution available?

Experience shows that failing to find a mature solution to what seems to be a common issue typically shows a lack of proper research rather than lack of actually available solutions. A technology ecosystem such as JEE is a huge field, and your team is not expected to hold knowledge about every aspect and every 3rd party extension out there. In this situation, always search for external help first, either online or through professional consultants / trainers.

If you really find yourself in a situation where you feel like you have to come up with your own solution there’s a trick which seems counter-intuitive especially for commercial software development companies at first, but which may pay out in the long run: Develop your solution, decoupled from any “business-specific” aspect, and make it open-source, publishing it on the Internet. Some of the many advantages of this approach include:
  • You get (free) feedback from external developers
  • You may even receive (free) contributions from external developers
  • You spread knowledge about your solution across the general public
  • You may even receive acknowledgement and gain reputation for the fine piece of software developed at your company

What is a “good” developer?

A good developer will acknowledge the need to create “good” software and that it needs to fit the definition described above, or a similar definition.

I really like the idea of distinguishing between a “programmer” and a “software engineer”. The former is someone who writes code which really requires no specific abilities or education. The latter, however, will, just as described above, transform “generic solutions to match a specific problem” which typically requires profound knowledge of the given technical environment and abilities acquired by personal experience.

What is the job of a “lead developer”?

A “lead” or “senior developer” will be even more aware of how to achieve maintainable software. He will help design software based on common best practices and advocate use of general design patterns within the team.

His role has thus the following characteristics:
  • He takes a “leading” role, but without making the team or the software product dependent on him.
  • He brings in his knowledge and experience, and these qualities will shape the outcome of the software product.
  • He takes responsibility for the input and advice he provides to the team.
This definition of a “lead developer” role perfectly matches an agile team (e.g. in Scrum), where no single developer is in charge of any part of the solution, but decisions are made in the team, and responsibility is held by the team in its entirety.

According to this role definition, it’s not the lead developer’s job to come up with his own solution to specific problems. As discussed above, this would actually harm the project, at least in the long run. Rather, the lead developer will help the team identifying common best practices to transform them into a matching solution to the concrete problem.

Conclusions

Encouraging developers to come up with “creative”, non-standard solutions increases dependency on individuals while decreasing maintainability and the overall life expectancy of a software product.

Advocating common conventions, best practices and refusing to re-invent the wheel, in contrast, are essential virtues of a good software developer, and essential to a lead developer. A software project adhering to these rules is more likely to create maintainable software, which defines return of investment in the long run. Still, the “Not invented here” anti-pattern is, in my experience, quite wide-spread in software development. As long as lead software developers are judged by the number and quality of “inventions” they make in-house, this is not likely to change. Of course, finding other ways to judge the actual performance of a developer by measuring his qualities as a true “software engineer” presents a challenge to software development management.

Eventually, I think, financial success proves how relying on open, generic and commonly agreed solutions pays out. As a software engineer, I like being part of a team which has come to this conclusion already.

Do you agree with what I wrote in this article? Do you have any additions from your own experience? This is my first proper “opinion” post on my blog, so please let me know what you think about it in the comments section below.


No comments:

Post a Comment