Now, new and next

Frank Wammes shared a self-assessment about thought leadership than made me think of what I write on my blog Disruptive Architecture.

The tagline for the blog is "When you need to make huge changes to your organization" which is about going from now to new. But what about next in the increasingly uncertain future?

I've been experimenting with generative AI since two years ago, and watching the progress of some of the tools since then. From something barely usable to an indispensable tool today.

As a consultant, I work with many different companies, and generative AI is more of new than now.

The question for me as an Enterprise Architect are the longer implications of this.

I'm trying to build a rather complex software platform for film productions, and with AI, I can build something much faster to try out ideas. This was not possible two years ago when I begun dabbling with the concept.

For many of my clients, this is not now, maybe new and more towards next. Very far far away from the daily work in the agile teams.

If I look at software development mostly done with AI, as new, what is then next?

In a follow up article, I will try to look into a crystal ball, and write about next using the film site project as an example.

Rather obvious

It's rather obvious when you know, but from my viewpoint, very few architects understand the need for information modeling. Even fewer know how to do it the whole way, from seed to loaf.

Your transformation program will surely fail if you forget to design a proper information architecture.

The starting point is a business glossary for the relevant part of the business. Then a canonical information model and a logical interpretation of this. Finally API's and schemas.

Business model -> Information model --> Data model.

If you don't know the craft, then you are in uncharted territory and will get lost.

Definition of done

"Musicians are lazy" is quote from Paul Guy. I believe the same is valid for software developers.

This is why the definition of done is extremly cruical for business critical systems. With optimistic estimates for development, the agile teams won't do more than necessary, i.e. according to the definition of done.

The problem lies in when the definition of done is shallow, and the missing parts affects data quality, security and build technical dept.

I asked Copilot for some input and it came up with twenty different point related to maintainability, team & delivery and governance. The first paragraph in the text was "This Definition of Done is mandatory. A change is not considered DONE and must not be released unless all criteria below are satisfied."

Back to reality.

I'm doing a PoC, lightweight MVP, for my film production related project. A this makes development take much more time if I have a proper DoD. This makes me skip a number of checkpoints for the initial version that will be used to verify the concept. However, with this list I know which debt I've got to resolve in upcoming sprints.

The Defintion of Done is only part of the story. To guide developers (me and some friends) and AI-tools, I'm compiling a playbook for software development & operations. It includes principles, guidelines and patterns, based on best practice, for the context of this application. Without context, you can't suggest a suitable best practice.

Both the structure of the playbook and the content is based on reasoning with Copilot, using references to a number of public sources relevant to this case as input. With a few hours of effort, I've have something usable and good enough to start with.

The more I tinker with AI, the more obvious is it that for rapid prototyping it's already very good. To build large, stable, maintainable systems, you still need to think in advance, before letting AI write your code. The upside is that AI can help you in your thought process.

Good old days

Is software development different today, compared to the good old days?

Today, you often have three main scenarios when starting major programs.

1. Build on what you have

2. Buy something new, COTS and/or SaaS

3. Build something new from scratch

Build on what you usually works well, until there are major changes to your business model, or the technical debt becomes too huge.

For many years, I've seen option two and option three fail miserably. We have our own share of them in Sweden, most recent one being Millienum health care system that was cancelled by two different regions. [edit] Classic waterfall project where the politicians stepped in and closed.

But I've also seen a number of programs developing subscription billing systems going down the drain. The question is why, because they were running agile, not waterfall development style.

Back to late 80's when I got the task to implement a material production system at a manufacturing site, replacing paper. No existing IT-systems, no COTS and cloud computing was somebody’s else mainframe.

The first thing I had to do was to collect a hard hat and safety shoes, so I could go out in the factory and see what really happened in real life. As a manger, I both did a budget and motivate the investment, as well as installing the minicomputers running VAX/VMS and plan a local fibre network.

The requirement work we did was very process oriented, starting with identifying how the people worked at every station in the factories. Then designing a SQL-database and UX for terminals to support the workflow. Used consultants to write most of the code, but we wrote interface code for API communication in C and embedded SQL. It even worked in year 2000, avoiding the millennium bug.

Fast forward until today.

A fair share of the software developers I met recent years have no clue about the business and how things are done. They often don't care about budgets or the cost of development and maintenance. They are highly specialized on a few tools and frameworks, and often competent in their area.

But the problems with major programs starts, but not ends, with requirements.

Lack of requirements, or incomplete requirements. If you starting a new digital business from scratch, and doesn't need to be profitable the first ten years, the your can run agile. If you have an existing business, and try to replace you legacy systems, the skateboard to car analogy doesn't work. Period.

Garbage in - garbage out is next. Information models more or less non-existent in today’s agile world. Very few software developers have the skill or the mandate to a proper information model. The consequences are delays, building up technical debt and lack of data quality.

Today's computers are fast, and this also makes non-functional requirements a forgotten topic. Lack of those will result in bad data quality, technical debt and problems with compliance to regulations.

Finally a word about stakeholder commitments. There are often to many layers between the guy or girl who present the budget to the guy who writes code, or the person who will use the system. Neither is anybody accountable for the whole.

Back in the good old days, we could have the whole team around a large table in the lunchroom, sipping coffee and smoking cigarettes, and I was accountable.

Back to code again

I need a small application to support film production, but the barrier (effort) to write code have been to high. What I will try now if I can use AI for fast development, as I have a documented UI, API and a database schema with test data.

The solution architecture was tricky, but with this design, I assume the coding will be ”easier” than in the 90’s.

Business need

I need a better way to find out who has been participating in each film production, i.e. project, both in front of the camera and behind. In Film production, both the cast and crew are managed in Yamdu, but in separate modules. Then there are projects not in Yamdu, where we have crew and pay wages, e.g. Provide crew & equipment.

There are two reasons for this epic, better CV matching and answers to GDPR requests.

High-level design

The solution should manage a list of projects imported from Finance system, Role descriptions from the CRM system, cast and crew from Yamdu, plus manually entered Individuals with roles in projects.

The SQL database will be in Azure, and the schema will be based on TM Forum SID and their Open API specifications. Access control via Azure Entra and two factor authentication as this data includes PII.

I've created a draft UX for mobile, so this will the basis for a small web application, accessing the database.

.net as development framework, with an aim to use only open source products in the future.

Solution approach

Use of Copilot to set up environment and Claude assist in coding.

First PoC is without integrations, to verify. Then integrations to all internal systems via CSV-files.

My background in software development was more than 25 years ago, and then in Java, C, Fortran, Pascal and script-languages, with Oracle RDBMS (UNIX) and MS SQL-server (Windows).