Are there any advantages with privacy?
View the discussions with Rober Willborg and Casimir Artmann.
This episode was recorded at Mindpark in Helsingborg, Sweden
See more episodes from Architecture Corner at https://youtube.com/c/architecturecorner
The challenge we sometimes have as architects is how to go from a business process to requirements for IT, either for standard software & SaaS or bespoke solutions.
This example is based on APQC processes documentation for Broadcast. The activities are relevant for production process within the Capability Production. The number in parenthesis after each activity is a APQC reference number
Shoot material/create content (13210)
Live feed/live air stream (13211)
Label raw content (13212)
Review and log footage (19891)
Record re-takes (19892)
Record second unit/B-roll footage (19893)
Verify technical quality of recordings (19895)
Transfer recordings to post house (19896)
Disassemble sets and lights (19898)
Return props (19899)
Restore location (19900)
Load out production resources (19901)
Transport production resources from location (19902)
Limited set of business activities within Production
Let's take the first activity Shoot material/create content as an example. We record the individual clips on a memory card or disk, but it’s more information captured during the take. There could be separate audio recordings and we need to keep track of scene, shot & take and other information for use in post-production. We often have a script supervisor who take script notes about continuity between shots and other details.
It is still very common on set to use a physical clapperboard and taking script notes manually, so the question is how to improve with IT?
As we can’t develop our own software for camera and sound equipment are we limited to what the vendors offer for solutions. It’s possible with newer sound recorders and high-end cameras to add metadata to the recordings, so they later on can be used in post-production. This metadata should also be used to label raw content and when review and log footage.
If we list these activities and map the to information objects we see that this is a requirement for an number of business activities.
So for new equipment and software, manage metadata about reel/scene/shot/take is a requirement or an IS-service in both pre-production, production and post-production. If our technical equipment or applications doesn't support this requirement, the consequence will be more time for manual work and a source of errors
We also create some Logical IS Components to assist us in the work. Based on our architecture principles, if possible these components should be align with standard type of solution for this type of business. E.g. Camera equipment, Sound equipment and Non Linear Editors.
If we look at our technical equipment we regularly use, the Sony FS5 & FS7 cameras doesn’t support metadata. Our default sound recorder Zoom F8 supports metadata for scene/shot & take.
If we use mobile iOS applications MovieSlate8 or Logger can we capture script notes in a format readable by other software. With a utility program for MacOS is it possible to match add metadata from a slate or sound recorder to clips from camera, if we have timecode. (not included in picture)
The cameras, Sony FS5 & FS7 doesn’t have time code by default, but we can record a signal from an external device, thus getting a time code for sync. The Zoom F8 sound recorder have support for time code so there is no issues.
The NLE we use (FCP/X) have support for metadata via iXML
What is the lesson learned from this rather technical example?
You need to have business knowledge and understand the information created and used in the different business activities.
You have to be able to identify solutions that supports your information needs (to a resonable cost).
You have to look at several types of information needs so that you don’t end up in a dead alley, only supporting a limited set of needs.
Some of the IT-based solutions for film productions are only available on MacOS and/or iOS, which differs very much from other businesses.
When you have a need for develop your own solutions, this is a different architecture story for next article.
When talking to clients and the architecture community the term microservices is gaining traction and attention. So, what are microservices?
“[…] self-contained piece of business functionality with clear interfaces, and may, through its own internal components, implement a layered architecture” (https://en.wikipedia.org/wiki/Microservices)
The definition above is theoretical and leaves the field open for interpretation on how develop an architecture based on microservices. However, this kind of approach is not new if you have been working with IAF (Integrated Architecture Framework).
Many architecture frameworks take a top down approach when creating the architecture. They start with a high level approach and decompose down to the level required to document what the architecture consists of, much like in the figure below.
The main drawbacks of this approach are:
It’s not that easy to define and analyse solution alternatives, as thinking is guided and influenced by the hierarchical structure that is being created and the political arena in the company.
The structure makes it harder to think out of the box, since each process leads to stow pipes and no room for shared services.
The architect is implicitly combining the ‘what’ and the ‘how’ question, because the definition of the structure along with the definition of the requirements. Making it difficult to isolate the service without running into a redundancy issue with multiple services that do the same thing but are separated by the process. For example Purchasing has the service “review claim” and Sales process also has a “review claim” service. Here there is a risk of creating redundant services in a large and complex organisation.
IAF takes a different approach.
Firstly, we define the services at the level of detail required without explicitly putting them into a hierarchical structure.
Then we define grouping criteria that are based on the architecture principles.
After that we create components by grouping the services into components based on the grouping criteria. The figure below visualizes the IAF approach.
OK – Claudio this sound theoretical can you give me a practical example? Sure: read Casimirs blog http://disruptivearchitecture.info/blog/2019/12/16/ea-case-study-top-down-or-bottom-up that describes a hands-on example of the topic discussed above.
If we follow the traditional way of software development we do requirements (waterfall or agile) then development in iterations.
With architecture, using IAF, after contextual layer, you start to define IS-services, then logical components and finally select physical applications.
But what about existing systems, standard applications and software as a service in the cloud?
Main application landscape for the film production company with both SaaS and applications run locally
We already have a lot of legacy even if we re-started everything from scratch a few years back.
Good luck as a small conpany with trying to influence Apple, Avid or Adobe on their roadmap for their NLE’s.
Finacials and HR are run as SaaS in the cloud (Visma eEkonomi and Tid), with regular updates and customized to a Swedish environment. The benefit for a detailed EA is not very high. However, as this application is the master for project id’s, we may need to integrate to other services.
The question is therefore why Enterprise Application architecture today?
Based on the situation as today for the film produktion company, these are the reasons why we need a lightweight application architecture.
Integrations between SaaS and applications to increase throughput, e.g. faster workflow.
Identify reasons for new SaaS or new applications and support product evaluation
Support development of a very few number of business unique software services, e.g. micro-services architecture.
After Christmas break will I come back to describe how this lightweight application architecture will look. Hint, neither strictly top- down or bottom-up. Instead, little of both.
For you who visited the updated homepage for the film production company, you will se that we doesn’t talk about EA all.
Why is very simple. The audience for this site doesn’t care about EA, they care about film production.
Logical scenarios is a core part of IAF and used to evaluate possible ways of solving several needs together.
Film production and post-processing process is a bit special when it comes to demands on infrastructure, thus this article.
Possible locations for post-production that the process and infrastructure need to support
We have these loosely defined requirements from the business side.
One production creates, as average, between 100 Gb to 1000 Gb of media assets per day
Bandwidth per editor for disk I/O is 250-500 MB/s. Using proxies 100 MB/s.
Relatively few number of users as post-production is highly computer intensive.
Professional NLE’s are not cloud ready yet. This is a very conservative business.
Several editors need access to same media at the same time.
With the type of productions we do, only one editor at a time per timeline / sequence.
The media is either classified as confidential or secret.
The value of one days media is between € 2500 to € 25000 for small to medium size. productions and repeatable shots. Value in larger productions and events can be much higher.
Cost is a factor as nobody want to pay extra for infrastructure.
All this media need to be managed as quick as possible so that the director can adjust next scene/shot/take.
We have several offices around the county and we also need to be able to edit in a mobile production office whetever we are.
The logical scenarios we identified are:
Cloud only
Cloud plus local storage
On-Premise plus local storage
Local storage only
Based on this we start doing a high level evaluation. Normally as a SWOT, but in this case only in text.
Cloud only sounds good and is according to our strategy, but.
Storage cost with Amazon AWS, € 250 / TB / year is reasonable, but when talking about 100 TB of storage in a very near future will the cost be € 25,000 instead.
In addition to storage, you pay for bandwidth to download content, e.g. € 100 per TB. For 250 TB per month will this be an additional € 25,000 per year.
You need at least 1 Gbps internet connection to edit proxis and the cost is € 1000 per year. For other edits you need 10 Gbps connection if they are available. In addition, you need a reliable VPN that adds to the cost.
The limitations in bandwidth and cost for cloud storage makes cloud solutions to expensive right now. In five years time, it may be a different story.
On-premise storage plus local storage
A NAS for storing media costs with RAID6 solution and 18 disks of 10 TB each is a cost of € 12,500. With this, you get 10 GB/s local connection. A fast dedicated media server is twice this cost.
The cloud solution is 12x the cost during three years compared to on-premise storage if you don’t measure installation and maintenance cost. This is € 150,000 for cloud compared to € 12,500 for a local NAS.
Local storage is then € 1000 per seat. We also need a reliable VPN with a bandwidth of 1 Gb so editors in local offices can copy the material to their workstations.
Local storage works only as long as the the editors doesn’t share media on a regular basis or as remaining option if bandwidth is too low at a mobile production office.
Production repository is where we store the timeline for the edit, before NLE’s this was the EDL, edit decision list. Not huge file sizes, updated often and shared between editors. Cloud is the preferred option.
Review repository is where review copies are shared between editors and others, including customers. This is short time storage and not huge volumes. This is why cloud solution is preferred.
Master repository are the delivered versions of the film from post-production. The size of them is much smaller than the actual size of the full media production. The masters will be shared to more parties than the actual editors and therefore a cloud solution is possible, even preferred.
VPN for assuring private communication.
Shared filesystem with SMB and/or NFS is needed for all professional NLE’s.
Backup is mandatory but the question is how to setup this in the best way.
Local backup
Local backup and cloud backup
Cloud backup
Local backup only is not recommended due to risks with theft, fire etc. The benefit with local backup is time to recover, compared to cloud, plus less dependency of external service provider.
Archive Unless implementing an long term archive solution, we need to increase our storage capacity by 50-100 TB per year. This is not sustainable in the long run.
