If working it is, reinvent it you shouldn't!

The Second System Effect

More than 40 years ago, in his influencing book (The Mythical-Man month), Frederick Brooks said about what he called: The Second System Effect:

The second-system effect proposes that, when an architect designs a second system, it is the most dangerous system they will ever design, because they will tend to incorporate all of the additions they originally did not add to the first system due to inherent time constraints. Thus, when embarking on a second system, an engineer should be mindful that they are susceptible to over-engineering it.

Unfortunately, I have learned this lesson the hard way, and used to see developers repeat my fault again and again!

So why do they tend to re-invent every wheel they are working on???

Mess and Myth

Joel Spolsky, one of the influencing guys in the industry (and the co-founder of stackoverflow, the CEO of StackExchange, BTW) answered this question in one of his most popular articles. 

There's a subtle reason that programmers always want to throw away the code and start over. The reason is that they think the old code is a mess. And here is the interesting observation: they are probably wrong. The reason that they think the old code is a mess is because of a cardinal, fundamental law of programming:
It’s harder to read code than to write it.

Said Joel!

Legacy code may contain messy code, lengthy methods, too much conditions that you can't figure out why on earth they are there? But the answer is very simple: these are bug fixes developed over years! and when you try to rewrite the application from scratch you are trying to throw away all the knowledge/experience in the code!

Why thinking that re-developing messy code from scratch will lead to better application is simply a myth?

Let's continue with Spolsky...

When programmers say that their code is a holy mess (as they always do), there are three kinds of things that are wrong with it.

1. First, there are architectural problems. The code is not factored correctly. The networking code is popping up its own dialog boxes from the middle of nowhere; this should have been handled in the UI code. These problems can be solved, one at a time, by carefully moving code, refactoring, changing interfaces. They can be done by one programmer working carefully and checking in his changes all at once, so that nobody else is disrupted. Even fairly major architectural changes can be done without throwing away the code. On the Juno project we spent several months rearchitecting at one point: just moving things around, cleaning them up, creating base classes that made sense, and creating sharp interfaces between the modules. But we did it carefully, with our existing code base, and we didn't introduce new bugs or throw away working code.
2. A second reason programmers think that their code is a mess is that it is inefficient. The rendering code in Netscape was rumored to be slow. But this only affects a small part of the project, which you can optimize or even rewrite. You don't have to rewrite the whole thing. When optimizing for speed, 1% of the work gets you 99% of the bang.
3. Third, the code may be doggone ugly. One project I worked on actually had a data type called a FuckedString. Another project had started out using the convention of starting member variables with an underscore, but later switched to the more standard "m_". So half the functions started with "_" and half with "m_", which looked ugly. Frankly, this is the kind of thing you solve in five minutes with a macro in Emacs, not by starting from scratch.

Resume Driven Development

I have noticed another hidden reason why developers tend to refactor working software? They want to try a new technology to write it in their resumes! even if it is not mature enough to be used in production code!

This behavior becomes more dangerous when they want to try these immature technologies in a working software! Which is one of the main reasons of software failures!

Exceptional cases

This post is aimed mainly at reinventing working/legacy applications from scratch. If it works, you shouldn't replace it, rather refactor pain parts gradually. Sure, there will be some cases that re-inventing the legacy application is the correct decision, but most of the times it is not, rather it will cost you a lot.

There are some exceptional cases that reinventing the wheel is worth doing (again, unless you are working on a working application):

• You may reinvent parts of a working application not the whole application.
• You may follow DIY principle (Do It Yourself) if alternatives are not feasible, for example your application depends on an expensive library that your budget can't afford, they you go for building it yourself.
• When you are starting a new application (not refactoring an existing one)
• When you want to learn something new.

Jeff Atwood, the other co-founder of stackoverflow, wrote a nice article about such exceptional cases, check it!

Software development status quo - 1: general insights (as of February 2016)

In this post I will select some results from surveys/studies/reports/statistics about the state of software development.
The intent of this post is to help developers understanding what is happening around them, and what are technology trends so that they can adapt and keep competitive.

For shared or different results between resources, I selected a single resource as a reference and let other results to readers to find differences themselves.

General

Popular Programming languages[1]

* Note: Have a look at this other article which summarizes findings from different important resources in a single post.

Most Loved Technologies[2]

Most Hated Technologies [2]

Most Wanted Technologies [2]

Web Development

Server-Side 

Server-Side Technologies[3]

Web Servers [3]

Client-Side

Client-Side Technologies[3]

Javascript Libraries[3]

Text Editors [2]

* Note: Text editors not IDEs

Text editors by occupation [2]

Source Control [2]

Mobile Development

Operating Systems Market Shares [5]

Mobile Application Markets [5]

Mobile Developers [2]

Agile Methodologies [4]

---

[1] IEEE Spectrum: The 2015 Top 10 Programming Languages
[2] Stackoverflow holds an annual survey about the state of software development. These results as per 2015 survey.
[3] W3Techs - World Wide Web Technology Surveys, as per February 2016
[4] VersionOne - 9th annual State Of Agile Survey, 2015
[5] MobiForge - Mobile Software Statistics, 2015

What is the role of software architect?

AFAIK, there is no consensus on the roles or activities every software architect should do! Even different levels of architect titles (Application Architect, Software Architect, Solutions Architect, Systems Architect) has no formal job description and I saw them used interchangeably.
All architect's roles are determined by the organization she is working for!

In this post I will try to describe the most important duties that I believe an architect should be concerned about. Most of what you will read in this post are arguable opinions, but here is how I see it...

Eisenhower Decision Matrix

[Source: Wikipedia]

I am a big fan of Stephen R. Covey and his awesome book: The 7 habits of highly effective people.
One of the concepts that was popularized by this book is Eisenhower decision matrix, where duties' priorities are divided into 4 categories:

1. Urgent and important
2. Not urgent and important
3. Not important and urgent
4. Not important and not urgent.

If you want to achieve outstanding results, then you should focus in the second quadrant, in the important but not urgent duties, at which I believe most software architect duties fall in!

The Software Architect Role

Now, the role definition IMHO:

A Software Architect is a chief programmer who is the owner of every technical strategic decision in a given organization, by making or taking such decisions.

The previous definition may seem a little bit weird, but I will illustrate it in detail in the following words...

Technical Decisions

The architect can analyze alternatives to make the decision whether it is better to buy or build a software solution.
If he sees that buying a solution is better, he should set selection criteria, analyze alternatives that match these criteria, and finally select the best fit from different vendors.
If he sees that building the solution in house is better, he should select technology stack, and do other related duties mentioned later in this post.
Managerial decisions like setting the budget, assigning resources, deciding the scope, ..etc, are out of architect responsibilities, although she may participate in them sometimes.

Making or taking decisions

Most of the time, the architect will take technical decisions like the ones mentioned in the previous example, but sometimes his recommendations may contradict with some business or management constraints, for example, he decided that a given tool is very important for the organization, but it is a very expensive tool that the project budget cannot afford, then the project manager will reject the architect's recommendation, and the architect have to adapt to the new constraints by looking for open source alternative for example.
In such cases, the architect is making the decision not taking it.

Strategic decisions

An architect is concerned mainly about strategic technical decisions rather that tactical decisions. By strategic decisions I mean expensive decisions that cost weeks to months to change in future, like selecting technology stack, setting the application architecture, what parts of the solution should be abstracted for future replacement?, ..etc.
By tactical decisions I mean cheap decisions that cost few hours to few days to change in future, like refactoring a given class, deciding to use certain design pattern, ..etc. An architect may participate in such decisions sometimes, but this is not his main concern.

Architecture owner

I have seen brilliant solutions come out from the least experienced developers. So, don't underestimate your team skills; whatever experience the architect may have, there are many many gaps in her knowledge!
I like to be called "Architecture Owner" instead of "Architect", the architecture owner facilitates the emergence of the architecture instead of forcing it. Architecture contributions are welcomed from all team member, but the architecture owner is responsible of accepting or rejecting them.
The Architecture owner's concerns are not limited to greenfield applications, but she should inspect brownfield applications as well, find design flaws, and lead the refactoring processes.

Quality attributes and the second quadrant

As I said before, most of the architect duties lie in the second quadrant of Eisenhower Decision Matrix, that is: the important but not urgent tasks.
Among these important but underestimated software features: quality attributes, or non-functional requirements, like: Conceptual Integrity, Maintainability, Re-usability, Portability, Security, Performance, Scalability ..etc.
Quality attributes is a huge topic that deserve a separate article, or more!

Specifying the coding standards belongs to this category of duties as well.

Should architect write code?

Definitely!

I know the answer to this question is debatable, but I hate talking from ivory towers and proposing infeasible solutions!

The architect should get his hands dirty to be able to decide which technology/solution is feasible and which is not.

From the other hand, I believe the competitive advantage of architects come from their technical expertise and their hands on experience, once an architect start to write code less, his competitive advantage will fade!

To be fair at closing this important point, the architect should not spend most of her time writing code, I would say she should sometimes write code, not always. For example, to develop a Proof of Concept (POC) for a certain idea, to develop a solution for a challenging problem, to develop a tough part of the system end-to-end..etc. That is why I consider her a chief programmer!

Process owner

Every project has its characteristics, and there is no single process that fits all projects.

One of the architect's roles is deciding the best process/methodology that fits the project.
In agile environments, the architect may be seen as agile coach, or coach of coaches!

Other duties

The architect role intersects with other roles, he can sometimes do some secondary duties that are assigned to someone else in the team.

An Architect is a leader, he may review developers' code, he may teach them, he may pair with them,...etc.

An architect is a designer, he may do some detailed design, and refactor some parts of the system.

An architect may interface with higher managers, and with customers.
The architect may do some managerial tasks like following-up project statuses, assign resources, ..etc.
I have created a dumb shape to illustrate where the architect's role fall between other roles in the team.

A final word: The architect in agile environments

An architect in agile environment will be concerned about strategic decisions upfront, and then be the architecture owner who is responsible of emerging the architecture incrementally.

Now, Your turn

I think this article is important and what I've said should not be by any means complete, and I hope from readers to enrich this post by sharing their thought in comments.

   

Want to know more?

I think the following books may help:

97 things every software architect should know -- Creative Commons license

Software architecture for developers

Learn technologies just in need not just in time

Technologies come and go, and if you try to cope with all new technologies you will get dissapointed — if not totally failed!
Even if you try to follow a single technology, like .NET or javascript for example, you will not be able to follow all their advancements!
On the other hand, if you buy in every new trend and try it directly in your production code, then be prepared for disasters!  One of the popular reasons of software project failures is depending on immature technologies!
I have heard some geeks said that they didn't use any product of Microsoft till the first service pack is live!
Even if we take their words seriously, we will not be able to apply this advice on all new technologies. And it seems that the solution may be to avoid learning new technologies altogether!

But how to keep competitive in this dynamic industry?

1- Spend most your time learning core concepts.

When I interview a candidate, most of the questions I used to ask related to technology agnostic concepts. Concepts that will not fade with time and differentiate geeks from amateurs.
For example, I rarely ask a candidate if she knows OOP concepts or what she knows about design patterns, instead I give her a simple problem that could be simply solved by applying a certain pattern.
Among the concepts that I believe any serious developer should invest most of his time in:

• Object Oriented Programming (OOP)
• Object Oriented Design (OOD)
• Design Patterns
• Database design
• Algorithms and data structures
• Mastering a general purpose programming language

2- Learn technologies just in need not just in time!

Someone gave this advice before, and it helped me too much during my career path!

In brief, you don't need to learn every new technology, and you can't! But when you need a solution for a certain problem, then it's time to learn a technology that solves this problem, or do it yourself!

But how do you know their is an existing technology that solves your problem?

3- Keep updated with emerging technologies

Don't learn new technologies till you need them, but keep monitoring the trends and know what problems are solved by which technology. I will tell you how I do this right now...

• Register in an RSS feeds aggregator, I am using Feedly
• Subscribe in important development sites' RSS feeds. I am following a bunch of sites/blogs, but here are the most useful IMHO:
• DZone.com (Don't miss their cheat sheets, or what they call RefCards)
• InfoQ.com (Don't miss their Mini-books)
• I created an account on Quora some days ago, I can't remember why! But they magically send very interesting content to my email, and it seems they have a strong recommendation system that they customize the content they send me day after another! So, go and create an account on Quora, follow some topics their, upvote some answers you feel good, and you will be impressed of the content they will send you, I promise :)
• Medium is a great source for geeks as well, visit it frequently.

4- Select technologies that worth knowing about, not learning

Actually, I rarely read a complete article from my feeds! but I keep passing through the titles, and I do this many times, every day! When I notice some technologies gain traction, that is, I see their  names repeated frequently, I  know this in the correct time to know what this technology is all about. I don't learn how to use this technology, rather, I learn what problems it solves and may read some reviews from developers who already use it.

This advice has a good side effect. Many times you will be asked in technical interviews about technologies you don't know. Good answers here is to provide an alternative. For example, if you are asked about NHibernate, which you never used, you can answer simply that you know Entity Framework instead! This gives the impression that you know what you are talking about, ... and it works!

5- Select technologies that worth learning

This advice is not for you unless you feel confident about your level in the first advice.

When you follow the previous advice (#4), you may find that certain technology gain more traction that it got adopted by giants, many companies has vacancies for it, ..etc. Then it's time to start learning it. For example, Microsoft created TypeScript. People kept talking about it for a while. I applied the previous advices on it till Google announced that the next AngularJS version will be built using TypeScript. Now, it's obvious that this is the correct time to adopt TypeScript (I didn't took the decision od learning TypeScript till now, just for priorites)

This is the best advice I can give for the busy developer to keep competitive in the market.

If you feel it useful, kindly help spreading it with your friends

Hello World!

Hello World!

Although I have two blogs that I used to write on some years ago, I discontinued writing on them for many reasons that I hope not happen again.
These blogs were not dedicated to software, neither they were in English, which is the case in this blog; I will write mainly about software in English!

The busy developer...

Have you ever faced a problem, and then found a solution the hard way?

Have you ever learned something after hard times figuring out how it works?

Have you ever seen an article that simplifies a concept you wished you can understood it?

Have you ever found a simple solution for a problem you have solved before using zillions of lines of code?

Have you ever felt disappointed from your inability to cope with the fast pace of new technologies?

Have you ever....

The list is endless, but let's ask one more question: have you ever need to document or share some good piece of knowledge?

I know you are a busy programmer, as am I, and for all the questions above I tried to help with this blog.

What to expect?

Don't expect frequent, and long articles. Actually, as we agreed, I am - as you are - a busy developer (to be honest, I am a software architect now), and I have limited time which I need to divide between my full-time job, my family, my self-learning, my side projects, and finally my blog!

One of my popular obstacles to write frequently in my previous blogs, is that I was trying to write long articles, which I have no time to do..I am a busy developer! And I decided to overcome this obstacle in this blog; I will share with you even very short articles that worth sharing in social networks not on blogs!

English everywhere!

I am not native English speaker, Actually my mother tongue is Arabic, which I am very proud of, but I decided to start this blog in English for two reasons:

1- To gain more audience

2- To enhance my English skills

And I hope you find it useful.