Fixing the ‘Automation pyramid.’ Does anyone else see that our industry is broken?

Pyramid.jpg
This article refers to the ‘automation pyramid’ concept used in manufacturing 
and production execution processes and is not to be confused with the term 
used in software engineering.

Introduction

There is a relentless rise in the sentiment that the expectations set by Industry 4.0 are far from being realised. Rumour has it that Industry 5.0 is coming, and yet again, we risk contributing a new item to the collection of buzzwords devoid of meaning.

In this article, I am trying to articulate my thoughts about the state of our industry. I will pinpoint the possible flaws; both technological and cultural, which, in my opinion, are preventing our industry from unleashing its dormant potential.

The focus of this article is on PLC systems and programming in general. Solutions must be provided for the ways in which we program and commission automated systems, as I believe this to be the Achilles’ heel of our industry.

There is an approach we adopt to think about industrial automation from a broader perspective, and we call it the automation pyramid. The automation pyramid is a means of simplifying the otherwise complicated structure of production and control automation, into a hierarchical structure. I will use this concept as it gives me the starting point from which I would like to start my exploration.

A brief description

There are different versions of the automation pyramid, but at this point; I shall stick with the following one:

text
Figure 1

This automation pyramid highlights the different layers of automated production execution.

In ascending order there is:

  • the field level (sensors and actuators),
  • the control level (Industrial Control Systems, PLCs, DCS, etc.),
  • the supervisory level (SCADA (Supervisory Control and Data Acquisition) and HMI (Human Machine Interface)),
  • the planning level (raw material orders, orders, resource allocation),
  • and finally, at the top; there is the management level (purchasing, finances, strategies).

The problem

This version of the automation pyramid has a somewhat blurry line (the red area of the pyramid (Figure 2)) positioned at the strata of the supervisory level. It separates Information Technologies (IT) and Operation Technologies (OT).

This categorisation has its roots in the fact that the latter is the offspring of electronic/electrical engineering, while the former is the descendant of computer engineering. These two worlds have gone through very different historical paths. The industrial automation (or OT world) was living an undisturbed existence for decades until we realised that the technologies were becoming more mature and that connecting IT and OT would bring enormous benefits. Whereas, the IT world has developed at a phenomenal rate and continues to do so.

The accessibility of personal computers, smart devices, and initiatives like Industry 4.0; brought entirely new perspectives and expectations — there was an increasing demand for data acquisition, visualisation, and interconnectivity. In other words, the people in the industry expected to have the same experience with machine automation, as they have when using personal computers and smart devices.

text Figure 2

Have you ever had the opportunity to observe an IT guy and an automation engineer (let us call him OT guy) talking to each other, perhaps solving a task provided by the plant manager? It is both thrilling and rather chilling because they simply do not seem to speak the same language!

text

I have witnessed and been part of such encounters multiple times, and all though it is somewhat anecdotal, it is also worth considering. The OT guy has a lot to explain when it comes to the needs of the IT guy; very often, the OT guys’ competence is questioned. The truth is that it is not actually him, but the technology that lacks the competence needed. The red area in the above figure is real and it is more of a North/South Korean border than the porous US/Mexican one.

We have been talking about the fourth industrial revolution for about ten years now, and the results are disappointing.

Industrial automation does not develop at the same pace as we are used to in the other fields of technology, and the gap between industrial automation and other areas of information technologies is simply not closing.

Walk away from the Edge!

In recent times we have seen emerging technologies known as Edge or Industrial Edge. The idea is to bridge the IT — in particular cloud and OT assets — to gather, analyse, and visualise the data. Edge technologies aim to be the intermediate layer somewhere at the level of that aforementioned blurry (red) border line. Although one can recognise the benefits of the Edge concept, it does not answer the problems in industrial automation; it is a mere patch, on a nasty uncleaned scratch.

In this context, we can see a narrative run by the most prominent players in the industry that follows this line of reasoning:

Industrial automation is a conservative environment, and we need to respect it. Industrial engineers are not ready to adapt to existing information technologies. We need to feed the technologies to people at the right rate, and they shall eventually adapt to it. In other words, we cannot respond to the expectations of the industry at this time because the engineers are not ready to deliver on those expectations.

Had humanity used this approach, we would still be using clay tablets instead of digital ones!

Some engineers are ready, and some are probably not

There is one sure thing though; the tycoons providing technologies in the industry are technologically behind information technologies by decades.

Undoubtedly there are engineers that would love to immerse themselves into new technologies, but they do not have the opportunity. They are simply compelled to use morally antiquated technologies.

The concept of Edge falls into this context. The OTs are morally antiquated systems with Edge patches mostly covering, but not actually solving, the problem.

text

The question is why we need Edge, and why we are not able to extend our operational technologies using information technologies.

Impossible? Impractical? Well, let us see what is holding us back . . .

Industrial automation mythology

An individual cannot understand the complexity of the surrounding world, no matter how clever and how well s/he scores in an IQ test. Even left-brained Sheldon from The Big Bang Theory identified himself with the Vulcan race — idolising Mr Spock due to his intelligent, logical, and unemotional psyche.

text

To simplify our understanding, we rely on a set of beliefs. We believe our parents when they tell us things like; we should brush our teeth because our mouths will be full of little worms. It may help us to do the right thing when we are kids, even though the reason is not precisely accurate. However, later we eventually discover that the actual reason to brush our teeth is, of course, because it prevents tooth decay, and most importantly, the dentist costs money!

Do not get me wrong, I am a great fan of mythologies, and I believe they play an important role in personal and societal development. The problem with myths is that they sometimes solidify into immutable dogmas. Then, instead of serving their purpose, they hold us back from growing and developing further.

text

Solution? We need to outgrow the myths

Here is a selection of the top myths that are harmful to our industry:

Myth #1: Only hardware-based PLCs are suitable for a hard real-time environment — PC-based automation is not that reliable

This myth might have been true in the past, at the time of the first Personal Computers; running a version of an operating system that was written by a group of teenagers. Back then; a small, relatively cheap, dedicated system, was the best choice for industrial automation.

It is, however, not true today. PLC systems are way more expensive than their counterparts in the consumer market, and PC-based industrial systems are just as robust, durable, and reliable.

The electrostatic and mechanical properties of Industrial PC-based hardware is not an issue today. Indeed, virtualised hard real-time environments are actually a proven concept, and they open a wide range of possibilities that are unheard of in hardware-based platforms.

Myth #2: Ladder diagram and Function Block Diagram are easy to read, maintain, and diagnose

While this myth is true for simple problems, in the case of slightly more complex problems; one ends up with a convoluted and complicated solution that is hard to read, maintain, and diagnose. The complexity of tasks is relentlessly growing, and these tools are just not up to the challenge.

Moreover, relying on the diagnostics of machines from the actual program requires sufficiently skilled personnel with access to the programming, tooling, and the equipment. The control software should be smart enough to get the diagnostics in an accessible layman’s way.

In my experience, it is an illusion to have maintenance personnel that are skilled in solving programming problems. This illusion is created by the fact that some maintenance guys are sent on a one-week training programme, after which they feel they can solve various PLC programming issues, or at least act like that in front of their managers.

This feeling lasts until they have to solve a serious problem, after that, they simply call the supplier for support. In my experience, you can only find a maintenance man in about one or two plants from every hundred; who is deemed up to the challenge of solving actual programming problems in PLCs.

Myth #3: Developing and manufacturing control systems can only be done by the big players

Part of this myth is that you need a stable manufacturer/supplier to have the hardware accessible for aeons, and to protect the original investment. In other words, when your control system breaks; you must be able to replace it with precisely the same model (the only compatible one). This argumentation would actually be fair — if it were not obsolete.

In the case of PC-based control, you run in a virtualised environment. Ergo, you can run the same program on different hardware.

text

Myth #4: PLC programming hard real-time is difficult, and sometimes, it is rocket science

Most PLC programming is basic logic. The programmer does not have to constantly think about breaking hard real-time behaviour.

Nevertheless, things become trickier when we ask the PLC to carry out tasks it was not designed to do. Additionally, there is more than one component to this problem — such as, the aforementioned programming languages, as well as some of the limits of the PLC systems that are profoundly difficult to overcome.

It is important to mention here, that there are no other systems to date that would guarantee fast execution, low jitter, and rich connectivity to industrial hardware, in the same way that the PLC does, and that this property constrains the PLC systems in some aspects.

However, many of these limits can again be overcome in the PC-based industrial PLCs; they open the pathway to a myriad of possibilities for solving complex tasks, and at the same time, make it feasible to defer complex tasks to a non-real-time environment and higher-level frameworks.

PLC programming would not be so difficult if the tooling had actually evolved and adopted some of the proven principles of software engineering.

text

Conclusion: change of perspective

How long did it take Apple to develop the iPhone and revolutionise the industry? About thirty months. How long did it take SpaceX to send the first rocket into space? About six years. How long have we been dreaming about Industry 4.0 coming true? Nearly ten years.

Would anyone like to object by saying that this is a comparison of apples and pears? If so, I agree, but . . .

Have you ever noticed the pace at which technologies are moving these days, and have you ever noticed how industrial automation is reminiscent of continental drift?

A few words for the decision-makers

Would you like your plant or organisation to move forward and make Industry 4.0 come true? Here are some suggestions:

  1. Switch to PC-Based control systems. There are some excellent manufacturers that built on CoDeSys platforms (here is an article about the choice our company made).

  2. Let your programmers and suppliers move towards software engineering in order to work with the best tooling for programming and commissioning. Can you expect to hit a few bumps along the road? Yes. However, this decision will yield excellent results in a remarkably short length of time.

For my fellow colleagues working in the automation industry

Take inspiration from the world of information technology. We do not need to start from scratch. There is a wealth of ideas and proven paths out there. We can prevent the impasses and mistakes; we can learn from the experience of our IT counterparts. They are not our enemies. Here is what we should do — we need to start adopting: proven ways of programming that are inspired by information technologies, methods and workflows from software engineering, automated software testing, proper source control, design patterns, and proper and functional componentisation. In order to do this, we simply need to change the platforms we use. With a good retro-minded PLC like Siemens or AB, it is indubitably infeasible. Believe me, I have already traversed that path!

text

Short summary

In short, what I need to say is this: the blurry red line should be moved as low down the pyramid as possible. Information technologies should cover most of the automation pyramid; right down to the level of physical hardware. The edge of IT and OT should be physical hardware, and only, physical hardware.

text Figure 3

Tags: Deep Dive

Peter Kurhajec
CTO of Inxton

Peter Kurhajec

Hello! I'm the CTO of Inxton! More than a decade ago my pet project turned into a full-time job - INXTON. Now I'm on a mission to bring industrial automation and information technologies together.