Software development maze

When you are a software developer, writing code is like walking in a maze. With an agile mindset, this is even more true.

As you don't have the perspective from above, you often end up in dead ends. Result, either go back and redo or create a mess with a chainsaw.

In the nightmare projects, it feels like somebody is chasing you with the chainsaw.

The more experienced you are, the more of high-level view you get, anticipating those dead ends before you choose that path.

When you are an experienced Enterprise Architect, you see the whole labyrinth clearly from above, and will be the pathfinder during the transformation.

Information modeling example: Call sheet

This article is part of my EA case study.

Background

We need to be better of knowing who is participating in film productions, both cast and crew, with the objective to improve both proposals and planning of productions.

Before we develop some bespoke software in the cloud, we need to do some information modelling.

“Simple modeling”

Don't overwork is a good rule, and three architecture artefacts from IAF would be a good start. They are:

• Business term

• Information objekt

• Data entity

Business terms

The callsheet includes information of who should be where and when, and is essential for daily work.

Source: https://beverlyboy.com/blog/20-free-call-sheet-templates/

Every business term in the example above here should to be modelled in an information model, before implementing in a database or used in integration. Bear in mind that is is only one variant of a call sheet.

Information objects

We have decided to use the TM Forum information models as a baseline for our work. Party Management specifications is part of SID, and used in this example to define information objects.

Source: TM Forum Specification Party Management TMF632

A Party is either an Individual or an Organisation. A Party can have zero, one or more PartyRole. A Party can have zero, one or more Contact information. A Contact information could be e-mail, mobile etc.

Using the model, a producer is an Individual, with the PartyRole of a producer, who has a name and a telephone number as Contact Information.

Data elements

Integration with Yamdu

The production system we use, Yamdu, can export a CSV file with all actors, contact information and information related to physical appearance as well as personal preferences.

Each column in the table is an attribute to the data element actor from an integration perspective.

Some information here are categorized as sensitive PII in GDPR and access must be restricted.

An actor is a PartyRole in the in information model. The actors contact information and the agent can also be modeled using standard model.

The challenge here is that TM Forum doesn't support physical appearance, e.g body height, for individuals, and we have to make extensions to the model and API's.

Azure SQL-server

We have setup an Azure environment, with separate servers for development, test, staging and production.

In the SQL server, we have to define tables, views and joins as usual.

The premise is to start with the basic objects in the model, e.g. Individual, Organisation and PartyRole and create tables.

As an Individual can have several Party roles, we should at least create an additional table with two attributes, PartyRole and Individual.

An Individual can belong to more than one Organization so this relationship creates need for one more table with the attributes of an Individual and an Organization.

Another decision is if we should use a separate table for physical appearance with attributes for each of columns in the CSV-file or if it's better to use value-pairs for these attributes. Pros and cons with both approaches.

Non-functional requirements as performance, security, legal regulations etc will also impact the design of the schema.

Conclusion

Without an understanding of the business terminology and use of information modelling, it's very hard to make av flexible IT-solution within a reasonable time.

When running Agile projects, you can't skip this part, unless you like to redo a lot of work at a later stage.

The fact that one person can belong to several companies is one paper a very small requirement, easily overlooked, but will create a huge impact in both UX- and database design. Not mentioning the massive changes in code, in different modules, if implemented afterwards.

You can leverage AI as a tool, but in the end, this is still design with trade-offs. Often the decisions are based on both business and IT-knowledge.

Skills vs skills

Everything I need to learn is how to prompt ChatGTP!

There are skills that are more valuable than others, and some skills doesn't change over time.

In the dark ages, I was really good at Novell Netware 3.11. Today, that skill is of no use. But the experience of total system failures after upgrading business critical Netware servers, and how to recover gracefully is still of great value.

I don't program in assembly language anymore, but computers still work the same way. GPU's are zillion times faster today then back in the days when I learned the math behind the calculations. Those formulas are stll very valid today.

Understanding the business in 80's and do information modeling for relational databases back then is not much different from today.

The skills you need should not be about tools and certifications. Instead, what’s needed is understanding of the problem and which tools to use to solve complex problems.

Garbage in, garbage out

Since last year, I have mainly worked with information governance for several clients.

It's very obvious that top management in many organizations now see information & data quality as something really important. The more we implement and use AI, the more important it gets.

Today, lack of ownership and lack of structure, i.e. Information Architecture, is the common theme for them, and both issues need to be resolved to improve information & data quality.

This is a coin with two sides. You can't be accountable or responsible for something not well defined, and without structure. Neither structure is enough if nobody or somebody is responsible for parts of the structure.

Most organizations need to improve their maturity in this area, to succeed with AI intiatives and traditional transformation projects.

This work is cross departments and cross release trains, thus very hard to solve in classic agile sprints. However, implementation must be done stepwise, as the effort often is huge.

What happens if you don't improve?

Garbage in, garbage out is still the result regardless if you use AI or traditional software.

Do you have a second source?

When I was studying electronics and mechanics in University during the 80’s, we talked about always having a second source for critical components.

This lesson needs to be taught again in the IT industry, as many organizations rely on only one vendor, and often a vendor located in US.

So the question is if I follow my own recommendations about second source for the cloud? This is new article in my EA study saga.

Cloud as a principle

Today most things for Artmann Media is hosted in the cloud, and we can barley run our IT without this infrastructure today.

• Microsoft Office 365 including Teams & Sharepoint

• Yamdu for production planning

• Visma for finance, CRM and project management

• Adobe for post-production

• SquareSpace for webhosting

• iCloud, Backblaze and LucidLink for storage

• DNS and other infrastructure.

For post-production, we have perpetual licenses, and run locally for everything, except Adobe Suite that is our second source in this area.

Most computers are from Apple, so there is a long term dependence here, but we also have Windows servers for certain purposes.

We have sensitive personal data and GDPR impacts our operations, and non-EU cloud providers complicates things.

What are the options?

If we look at the options to cloud, what do we get?

Microsoft Office 365 is used in all processes, so this is probably the biggest hurdle. My plan B since several years back is to run Microsoft Office, including Sharepoint and Exchange locally on-premise in Sweden, with VPN-access and perpetual licenses.

Yamdu is a tool in the cloud for production planning, as breaking down scrips, creating short plans and call-sheets. It also contains sensitive personal data. It's possible to run on-premise, but expensive, so this not an option for us.

Yamdu is located in Germany, and the owner ARRI is a well established German company, run on infrastructure in Germany. The impact from actions related to US government are thus less, and I see the risk as much lower today than with US-based vendors.

Visma Accounting is developed in Sweden and Visma Project is developed in Finland, so I anticipate even lower risks than EU-based services. For both services, I can export general ledger, customers, suppliers and products to use something else.

Adobe Suite. If anything happens, then bye, bye. We have other tools to use instead of their products. They will go out of business if the cloud disappears, so much bigger problem for them.

SquareSpace, same as Adobe.

iCloud, Backblaze and LucidLink are end-to-end encrypted services where sensitive personal data is managed. Worst case, if access is blocked there are other alternatives in EU.

Our DNS is Hover, with head office in Canada and European office in Germany. I also have a backup DNS and e-mail in Sweden for management of critical accounts. Consequences are dire, but risk is very low due to mitigation actions.

What is you plan B?