The issue is that we all love complexity, even when we say otherwise:
https://medium.com/outsystems-engineering/visual-programming-is-unbelievable-heres-why-we-don-t-believe-in-it-ad52ba24c98

I agree completely. I’ve been writing software for over twenty years and it has changed significantly over that time. Many more hipsters joining the programming ranks. They have to maintain an aura of coolness by bringing in the latest tools and trends, despite the underlying lack of utility of many of those trends. The other problems I see are letting inexperienced people architect solutions, how programming is taught in colleges (too much dogmatic OO), and authors trying to make a buck writing books that advocate over-architected solutions to problems.

Oops my diagrams were interpreted as XML and stripped.

8 years back, I happened to work on a work-flow tool (which is internally built in Java). The tool had drag and drop components even for a “for loop”. So if the business requirement is known, just use the components to build the logic, deploy it and you are done.

Apparently, it sounds so simple but believe me it was complex, for the reason that for a simple “for loop”, a developer with Java background can simply write for(int x = 0; x < 10; x++), which is more simpler than using the component.

Being Simple is always good than being complex, after all the maintenance is much easier with less cost for simpler implementation.

Great article Justin. The 20+ years that I have been in this industry, I’ve heard and seen so many religious arguments for one technology versus the other (anyone remember the introduction of Java?). While I do have my own ideas about what works best, ultimately the technology is situational to a certain extent. It just depends on what you are trying to achieve and the customer’s needs is ALWAYS the most important. Simplification is really the goal as I think most people would agree. Controlling costs should always be at the top of your list when building systems. As well as, balancing that happy medium with the customer of cost versus complexity. While they may want software for complex processes, this often opens a dialogue for process improvement or modification in an effort to reduce software development costs. Once a plan is put in place, then frameworks can be discussed to address the requirements. It’s important to note that in the consideration of frameworks, who will have to maintain it in the future? You or the customer? Again, yet another point to come to an understanding about with your customers because you may need to pick technologies that play to the strength of the their own teams.

DRY, KISS, YAGNI and SOC are probably the most beneficial and important principles in software engineering to master. Part of that mastery is knowing when not to go too far as well. You can oversimplify and overcomplicate but it’s probably easier to fix oversimplifications with some exceptions (missing data because you stored the result of a + b which you imagined was simpler rather than a + b but later a requirement comes in and you need a independently). To master them isn’t purely a matter of study. Practice makes perfect. Simplest way, make your thing work then after that work out how to simplify it as much as possible.

You start the bar a bit low at DRY. DRY is one of the most fundamentally important principles in producing good software. It tends to be measurable or easily demonstrable and while like anything people can take it too far a high level of DRY is nearly always good and you have to take it to quite an extreme for it to be too much compared to other things. Gains are usually very quick and obvious with DRY and not so commonly debatable. You have a handful of thresholds and exceptions that should be easy to both conform to and demonstrate the pros and cons of precisely. As you raise the bar things become increasingly easier to adhere to problematically or inappropriately. When and how to DRY tends to be obvious and easy to learn to do automatically with little overhead in terms of delivery especially when well practiced. As I said there are exceptions but it’s not really that ambiguous or complex. There’s a particular concern with DRY. It’s so easy to copy and paste creating ten times the code. You can inflate things enormously very easily, changing one or two things and creating a spot the difference nightmare. You really need to adhere to some basic level of DRY otherwise there’s no limit as to how much code you can have. DRY also helps with your architecture. It’ll naturally reveal shared dependencies doing half the work for you in terms of organization. It’s not just about preventing inflation.

SOC is very important but easier than DRY to do unnecessarily or without gain. People can get it wrong not only in respect to what to separate but how. Is it always good to have presentation distant from logic it’s intimately dependent on? Do you separate by file, class, method, etc? For small programs unlikely to expand is the separating needed?

Talking about composition versus inheritance you’re moving much more into the zone of does it really matter. Wait for inheritance to become a problem. Wait for composition to become a problem.

You’re starting right at the top of the food chain. This isn’t what’s happening with overcomplexity. If your complaint consists of DRY and SOC then I don’t think you’ve seen the worst of it. Your argument might be better placed many many years ago when potentially excessive standards concerned more bare minimum basic principles which generally if not applied idiotically (naively, religiously, excessively) tend to deliver worth with very little extra effort and tend not to be excessive. At least for any code endeavor that’s not going to be tiny. Your starting position isn’t quite what I might call high investment, high standards or high maintenance unless people are being overly zealous like a grammar nazi. Those are just normal standards. Same as consistency. If you don’t stick to these standards to an appropriate degree you can expect to see productivity dive as well as quality in respect to correctly functioning software over a relatively short time span (even within a year).

Over time what I’ve the bar go up and up for programming on a LAMP stack in terms of minimum complexity needed regardless of the actual requirements of an endeavor. None of these things are strictly bad but if you do then for the sake of doing them and even worse entirely by the book in the most strict sense then what you have is ten to a hundred times the complexity needed. The worst thing is that I quickly switched from Java to PHP to get away from this madness.

Today you need to do: DDD, TDD, BDD, anything with DD in it, all the “best practices” (anything anyone claims is a best practice from whatever soapbox they’re given), all the standards, power tools (ultra IDEs, etc), design patterns (all of them all the time), SOLID (all the time), OOP (all the time even though PHP is multi disciplinary), Agile (both XP and Scrum at the same time), TDD, Unit Tests (maximizing coverage), integration testing, REST (I guess that’s progress over SOAP same as JSON replacing XML), UI testing, full integration of tools for the full SDLC pipeline, Event Sourcing, Command Bus, CQRS, Queues, Microservices, Cloud, DTOs, Virtual Services, NoSQL, distributed everything, in memory databases, docker, template languages even though PHP is one, huge frontend and backend frameworks, ORM, functional programming, etc.

The list never ends. Many of these are useful in specific niche circumstances and when not over done. I’ve used a good many of these and taken it upon myself to apply them but when I’ve had problems where they’ve been relevant and usually one at a time. What I tend to find is I look at this list and think well that’s daunting, all of that. If I’m a good programmer and even I find having to do all of that a bit much then you have to ask what about the rest. What you’ll find is that they’re not coping with all this stuff. They’re just learning to repeat it by rote. They’re not solving problems they’re just making code. You don’t have to understand these things and apply them appropriately or well, you just have to do it.

Apart from OOP you’ll not see programming in that list. Though even when it comes to OOP you see people spending more time orienting objects than programming.

There are a number of drives for this. Part of it’s the Java effect where a lot of people have been trained in that but end up going into something else. Much of it is the illusion that it’ll make things simpler. A lot of training wheels stuff like TDD or perceived productivity enhancers. Also the illusion that it’ll contain bad programmers and force then to write good code. Another notion that it’ll future proof the code (which is practically premature optimization). Often cases people are just copying big names in technology because they somehow think using the same technology stack will lead to magical success. This is well known as Cargo Cult. Rosetta code is another sought after benefit attempting to bridge the gap between developer and requirements or code and the business. Many of these things are the titles of CS courses or tech presentations which tends to tell you these people aren’t really avid programmers but avid students.

All laudable goals but the lofty the promise the less likely it’ll be fulfilled. People have been creating successful applications without all the works and fundamentally the difference is quality of programmer not so much quality of tools and methodology (which often turn out to be far from perfect).

In nearly all cases you’ll end up getting the opposite. You have your programmers spread out. I call this anything but the code. It’s almost malicious. Finding excuse to do anything else other than the code that needs to be done to meet the requirements. It puts a huge mountain of work in front of even the most basic programming task. You’re really onto something when you point out that there’s stuff that should be secondary to the immediate business needs. You can’t always sacrifice the future for the present but you should be still spending a large portion of your time on present concerns and not staring into a crystal ball. Even experienced developers have significant limits to their clairvoyance.

Increment slowly. Don’t add all the things then have to remove things. Solve problems as they arise. If you’re going to solve a problem preemptively then make sure you have the experience to really be certain about how that problem is going to play out and that the solution will fit.

None of these things come with a magical ROI. In some cases the thinking behind that couldn’t be more wrong. As in that adding more and more will make things more maintainable when minimalism is one of the most fundamental principles of maintainability. Anyone with a bit of common sense would know that as a rule of thumb more layers of indirection, more objects, more lines of code, more to maintain. Adding something as complex as a million lines of code framework or toolkit to your project might not make it simpler. Adding complexity to gain simplicity is never a sure thing. Good programmers will instead apply things on a case by case basis.

Sometimes these things are promoted on the basis that if everyone uses the same thing then it’ll make it easier to have some kind of widespread compatibility with developers that only have to learn one thing or already learnt it while ignoring that you’re adding something new to learn in the first place. Your solution is your problem.

Subjective and highly tentative gains are regularly presented as well as claims that these are proven. Unlike a lot of people only reading blogs I’ve read one of two of those research papers and they’re not saying or proving much of anything. They tend to apply to specific cases, are purely anecdotal. It’s hard to rule out conflict of interest.

Maintainability is drilled in at university. I remember that but spotting the problem isn’t as easy as solving it. You ask someone why something is maintainable or easier to read and when you get to the bottom of it you’ll find it’s because someone else said that. It’s not tested, measured or understood. An education only goes so far in making up for a lack of experience and practice.

Quite often people don’t understand the basics behind these solutions. You need to test your code and make sure it actually works. There are many ways to make sure your code is testable. It doesn’t have to be one way. Divide and conquer is a common principle for making code maintainable but you don’t do that down to the individual line and there are a number of strategies to compartmentalize. Breaking things up in the wrong place can make things larger rather than smaller. Start from the top down. Divide by two, then if need be divide by two again. Don’t divide by a thousand straight away. Let things grow and when they’re actually starting to get too big then break them up.

I’ve seen first hand some of the mess this makes. A template language is used in the belief it’ll stop bad developers putting logic in views that doesn’t belong there and that it’ll provide security. Now you have ten problems. A bad developer will always find some way to break out of the cage or abuse their tools. You now have to learn two languages. Your pipeline is now much more complex. You template language likely wont always get it right with its best guess escaping and will try to escape for everything rather than what’s needed by the context (some people seem to think context based escaping going by extension solved everything). Developers that can’t write secure code in the first place can only be helped by tools up to a point and if anything it’ll deprive them the learning opportunity. Your template language is a crippled language often missing even basic functionality. There’s no rule stating that a view always needs to be simple.

What ends up happening then is that because the template language is so minimal people end up pushing the complexity into the frontend using javascript libraries in place of their template language and often they don’t even realize they’re doing it.

A darker side to this is that once you get over all of these learning curves most of them really offer little benefit. It’s a kind of gnosticism. It’s not programmer friendly but anti programmer.

I have to raise a point of contention with you on the toolkits. Some experienced developers will object because they don’t want to learn knew things if what they’ve learnt works. Some however have the experience to know these things aren’t delivering on their promises. Quite a few things even when they do they add other costs that far exceed the benefits of saving a few lines of code. I’ve been saying for a decade now that libraries tend to achieve a better outcome and people are finally starting to warm up to this idea as they’ll gaining experience with their frameworks and toolkits. It’s not perfect but it’s a start.