Overcoming the challenges of continuously evolving Cyber-Physical Systems (#17)

“Design for easy repairability, especially in complex systems that must evolve over time.” - Ali Hajou

In this episode, the SPCs Unleashed team tackles the complex dimension of continually evolving live systems, with a particular focus on large, cyber-physical systems. Mark Richards, Ali Hajou, Stephan Neck, and Nikolaos Kaintantzis delve into the intricacies of designing and maintaining systems where hardware and software must seamlessly integrate over extended periods.

Ali opens the discussion with a story about a client who successfully implemented modular retrofitting to keep large machines up-to-date and operational even when certain parts were not yet available. Stephan and Niko share their experiences, highlighting the importance of a continuous delivery pipeline (CDP) that can accommodate the unique demands of cyber-physical systems, such as firmware updates and complex deployments.

The conversation shifts to the critical role of modular design and interface management in enabling continuous evolution without compromising system stability. The team underscores the need for stable, product-focused teams to manage both legacy systems and new innovations effectively.

As always, the episode concludes with a spirited debate on the most relevant Measure and Grow metric. While opinions differ, the consensus emphasizes the importance of modularity and the ability to deploy updates independently to ensure systems can evolve with minimal disruption.

Cast:

Mark Richards (Moderator)

Ali Hajou

Stephan Neck

Niko Kaintantzis

Transcript

Mark Richards: 0:01

You're listening to SPCs unleashed a shaping agility project that emerged from the 2023 Prague safe summit. The show is hosted by Swiss SPC, T Stephan Nick and Niko kaintances, Dutch, spct, ali hajou and Aussie safe fellow, Mark Richards. We're committed to helping SPCs grow their impact and move beyond the foundations. Taught during implementing safe each week, we explore a dimension from the frameworks competencies. We share stories about our journeys, the secrets we've found and the lessons we've learned the hard way. Welcome to SPC unleashed episode 17. We're here to talk about continually evolving live systems. This is the final part of the trilogy of the dimensions in enterprise solution, delivery of the three dimensions. It is probably the one which is most specifically focused on large, complex cyber physical systems. I think of both other dimensions. There are tools, there are ideas that you can apply to big complex software. You even apply it to not particularly big complex software, but getting into continual life, continually evolving live systems. It really narrows us into cyber physical and the challenges that come with continually evolving a live cyber physical system. So there's probably no surprise. The essence of the dimension is around build an end to end continuous delivery pipeline. Go figure. We've already done an entire episode talking about the continuous delivery pipeline. So today we'll just narrow ourselves in on what that looks like in a cyber physical world, evolving deployed systems. Again, we talked about that in the DevOps dimension, but we'll drill into probably what it looks like, specifically when it's deployed systems with hardware involved. And the third aspect of the dimension is actively managing AI and machine learning systems, which is something that we've all got a little bit of experience with, but to be honest, none of us felt deeply enough about our practical experience in that space to devote a lot of the episode to it. So we're going to focus much more on the CDP and evolving the deployed systems. But to kick us off, we're going to start with our favorite stories from clients? Because over the years, I think I speak for all of us when I say a lot of the best stuff I've learned. I've learned from watching a client do something clever, rather than from reading a book. So Ali, why don't you take us away? What was your favorite story from a client who did something really innovative, to continually evolve a live solution that perhaps looked impossible at the beginning. Yes,

Ali Hajou: 2:47

good, morning, evening, afternoon, nights, everybody. I this thing that really triggers me in this dimension is the, you know, the interchangeability of small sections of a solution. And you know, as a as a person who loves to tinker and sort of maintain my car, or at least be busy with my car, I really like and appreciate the ability to interchange parts so you have a live system, which is the car it's hopefully running. They try to maintain it, try to maintain it well. And if there's a if there's a module that either requires maintenance or requires an upgrade, because I simply want to it should be able to do so. And I've been working with a client who did that to very large machines, so creating a huge machine which is extremely expensive but has the ability to use retrofitted modules. And with that, and I'll talk a little bit more about that, having the ability to send the machine off to their clients, maybe with one of those modules being still an old version of it. So then at least it's out there at the client, at their clients. So it's it can be in use with all types of new modules, so you have the new machine, except for one piece, simply because that piece might not be done yet, or it's, you know, there's a parts shortage, or it's still in manufacturing, but at least the customer of my client would then have a running device that uses somewhere in the grand scheme of all of this newness, it still uses an old part, and that should be possible, so at least it's out there. So, you know, I'm going to talk a little bit more about that. So

Mark Richards: 4:57

I'm going to add a fresh story on our way across to. Another story in the world of not to continually evolve live systems. Today, we are running a test of using LinkedIn live as well as YouTube, and we were unable to figure out a way to test it anonymously before the show, and I was a little distracted while Ali was talking to go, did my live production test work? And yes, no, it doesn't appear to have so you know, if there's a lesson learned, it's make sure you test before you continuously evolve live systems. We are, however, live on YouTube at least, and we'll wait for the commentary and the laughter to happen at my expense on LinkedIn. Niko, why don't you take us away with your favorite story?

Nikolaos Kaintantzis: 5:50

I feel, I thought we experienced how this live testing yesterday, but I don't say much more. Maybe there are lawyers listening to I hope you can fly today and pay today. So in my case, it's a collection of favorite stories. It's all about the continuous, continuous delivery pipeline, and big companies saying we don't need this because our clients are also big. And the solution isn't that agile. That I need every week, every two weeks, every month. It's just once in a half a year or twice in a year. So I don't need this expense, because nobody wants that that frequency of often and often deployment. But what they didn't realize is that you also train a muscle memory. It's really about learning how to do it. So if you do it only once in a year or twice in a year, and then you have a bug in the live system, you end up with, oh, I have no idea how to fix it fast. So I give you a package where you have to install it manually, or it's just an emergency. So even if you think you don't need it, and even if your clients think or saying once in a month or every two months is too much for us. It's a memory as a muscle memory, a skill set you need. So please invest in continuous delivery pipeline, just that you're able to be fast in accidents or in faders. So

Mark Richards: 7:19

you had a great story from conversation at Nashville you're going to share. Niko, do you want to dive into that? Say it again? And a great, great story from a conversation you had in Nashville that you were going

Nikolaos Kaintantzis: 7:30

to share? Yeah, yeah. That was really cool. So it was the closing celebration at the state summit in Nashville, and there were people from a company who have life systems. And there was tanks, tanks in the sea. And what they were able, what they had to be able, were to deploy then when the ship or the tank is reachable, so it was not I deployed now, because I enjoy to deploy, because it's now, it's the window, so there really had to be able to deploy whenever it's possible and not whenever the time schedule says. So it was really cool to have this conversation and see what they are struggling what they are doing. So, yeah, it is also the same, like, the same story. You need the skill to because now it's a window, and now I have to deploy, and that's why you need to continuous delivery pipeline without it's you don't have the time, or you're not ready, or whatever. Beautiful

Mark Richards: 8:30

Stephan, it feels like your favorite story is a pretty recent one. Kind

Stephan Neck: 8:35

of, yes, I had the opportunity to enter the cyber physical world the last few years, and shifting from a more software environment to the cyber physical world is really exciting. I like it, and what I've learned is there's a lot of smart people out there. There's a lot of smart thinking. But how do you bring it together? How do you how do you make sure that you break those silos where there's lots of good people, and I've learned that the last year in the aircraft industries, hey, if it doesn't fly, you can't sell it. So make it fly. And how you make it fly is very crucial and important, because if you produce aircrafts, they're up there in the in the air, in the skies, like 20 to 30 years. And if you design it the wrong way, as we talked about last week or two weeks time. Hey, it's probably running out of life cycle time pretty fast. And, yeah, I like this cyber, physical world, and I'm still learning. So

Mark Richards: 9:46

I was reading the newspaper this morning about yesterday's events, and you know, there were a few comments coming through around at least this didn't happen. And you know, one of those comments was, no planes crashed. Uh. So when I saw you starting a story in the prep nights for the show around Pilates and aircraft, you know, continually deploying live systems. Is this like the planes in flight and you're continually deploying or, yeah, take us a little bit deeper into that.

Stephan Neck: 10:15

Yeah, okay, without, without inflicting my NDA, it's, it's more about the whole solution, life cycle, right? It's not about the this is a flying software box. It's still a hardware box with lots of software in it, lots of modules, lots of components. And the thinking goes, how do we design new model modules that fit into an existing space, an existing design, but it's upscaled. It's new technology. How can you change a tech stack over the time? How can you migrate over the time? That's the big discussion now in this, in this industries that I'm, I'm I'm supporting, and it's, it's really exciting. So it's more, how about having an economical life cycle that really pays off, right? And if you have a brand, or if you have a product that sells itself, even more you should think about the life cycle, because you might have competitors out there that will scratch on your door. They will scratch on your part of the market, and you never know what happens. So being being ingenious, being innovative is, is really crucial, and that's what kind of, yeah, is designing your success at the market. So

Mark Richards: 11:48

if we start to kind of transition and dig a little bit deeper into continuous delivery, I think there's a thing that's conjured in a lot of people's heads when you talk about continuous delivery, which is constant updates being inflicted on you, right? And spending hours a day in Miro, as I do, there are days where it feels like Miro updates 10 times. And you know what happens? We're in the middle of doing something, and all of a sudden there's a little bit of an hourglass and a screen flicker, and maybe a web page refresh, and Miro is updated. Likewise, if I'm playing a game, that game can be updated while I'm in the middle of a game, and, you know, I'm not having any choice in the matter. It's just stuff is getting pushed at me. But I think one of the biggest things to wrap your head around as you start to think about the CDP in a cyber physical space is that this is not continually inflicted on you. It's about saying, Well, this thing's gonna be out in the field. It's gonna be out there for a long time. It needs to be updatable, and you need to be able to safely and regularly update it. And Stephan has already kind of touched on, and Ali has touched on this idea of, you know, building into your design, the configurability to be able to let you start to change things in the field, but when you get into you know, what does that mean for continuous delivery? I know, for me from some of the cyber physical companies I've worked with, where a lot of their live updates in the field are firmware updates. One of the biggest bug bags they have is that people often don't take the firmware updates when they want them, right? So they push it, they push it, they push it. And, you know, it could be nine months, and all of a sudden somebody goes and applies, you know, the last 20 updates at once. So this is separation between I've pushed an update and somebody has chosen to take advantage of it, which makes life very interesting when you think about your continuous delivery pipeline. So Niko, why don't you take us in as a starter into the world of the continuous delivery pipeline, and some of your experiences there about how it becomes different in that cyber physical space. I

Nikolaos Kaintantzis: 13:57

would love to but let me add one thing more when I listen to Ali and to you and to Stephan, I realized when I did my education to become a computer scientist, the modelization was really an important part, and somehow, I think we forgot about it, because we just created one solution, and it was much more easier to write everything together and not making spaghetti gold, but but making it faster by not having proper interfaces. And when I listen to Ali saying, hey, I can run something with an old part in it, it's really an important skill that you have to regain again, because if you have this mobilization, you can just exchange models on the fly. You know how to interact. So it's a, it's a really cool thing for those who are as old as me, just get the old skills again, and you're, you're up in the modern time again. What you ask now about, about hardware and continuous integration. I remember the first time when I said to a client of me, uh. In their hardware space, in a medical device space, about continuously build every, every, every two weeks. He just said, You're crazy. You're a software guy. You have no idea how much it costs. So if we have to build this thing every two weeks it costs every new build costs 1000s, maybe million. And I told her, No, no. I didn't mean build every time a new hardware. I just told you, it's important to be able to integrate often. And what we ended up was they were building a platform, and you cannot build a platform every two two weeks, because it really costs 100 1000s. What they did is they built more generic platforms with clear interfaces. And then on top of this, you can put either software simulating the stuff you want to have in the hardware, or put bigger, bigger, cheaper hardware and experimenting with the hardware until it fits what you want. And then when this solution you build on top it's good enough. Then in the next time you create the platform, it just built in, in the hardware. So not build the hardware every two weeks, but having a generic hardware allowing you to simulating things in software or simulating with another colleague told me a story about did a lot of things with Raspberry Pi before they decided to build it in in the real hardware. So be able to experiment, be able to play, that's what we mean with continuous integration, continuous deployment in hardware. Because, yeah, if you build it every week in new hardware, then you are poor very you become poor very fast.

Ali Hajou: 16:40

Yeah, indeed, indeed, I, if I may just riff on top of that, the you know, when I was preparing for this dimension, honestly, I thought about a lot of activities that engineers already do or and already have done for many, many years, if we're talking about and I'll just come back to the car example, simply because you know, if you want to explain anything complex, you you can play, explain it as if it was a car. So in a car, you have different modules, different elements, and you can bolt them on. You can take them off again. You can replace them. You can adjust them. Those are interfaces, and interfaces are being developed forever. There's nothing new. This is nothing new. However, whenever we're trying to design a new system, and we are under extreme time pressure because, you know, the system needs to be done before november 2024 because we need to send that design over to all of our suppliers to produce all the different components so that we have the ability to integrate them somewhere in June 2025 you know that there's a lot of Time pressure there, and building interfaces, or building modules that are very repairable or interchangeable, or maybe interchangeable in a life system, which is, you know, a life ongoing system where you just, you know, take out a piece and then replace that without Bringing the entire system down. Think about a manufacturing plant. It has conveyor belts. It has all kind of machines. They call them stations, typically in in on the shop floor. So and these stations all have their own task. And you know, the entire factory is continuously producing, but then are you able to interchange one part out of that entire chain of machines without disrupting the flow of the of the factory. Because if, if that's the case, then the factory is standing still, and it's, as Niko mentioned, it's going to cost you a lot of money, and nobody wants that. So the, let's say, the difficulty there is not so much the idea of modular, modularization, so creating modules, or creating your system in modules, it's the ability to to, how should I call it? Be able to spend the time to make things more repairable in the future, sort of the, you know, getting a mortgage on your future. And in order to do so, you know, it just it costs more time. It costs more a brain power of more people, and people need to be more involved earlier on in the automotive industry and many other industries, qualification, homologation, which is, you know, getting, getting a design ready for mass manufacturing is a very costly thing. And if you have the time, and if you have the resources, you could, you could, you know, spend. There's a lot of time in there creating the most repairable machine ever. But typically it's not an engineering issue there. Typically it's much more of a marketing or a business issue. Stating, no, we don't want to spend that much time in making our systems extremely repairable or live updatable in the future. No, we just want to have that product out, out of our, let's say, out of our industries or our shop floors, as soon as possible. So the it's, it's always a very tricky thing to involve people very early on. Think about quality control, to involve them very early on into the designs that might not be complete yet, or might not be, let's say, the most repairable version of it yet. And I see this not really as an engineering issue, much more as it is a business issue that doesn't want to invest or maybe cannot invest, in making systems upgradable in a life environment.

Stephan Neck: 21:10

So let me pick up that thread, Ali, and also what Niko said when I prepared, I also probably fell into the trap thinking and writing about the CDP, and if I step back as a change agent, two or three steps, it's about the context, right? So some techniques, some practices, you have to contextualize that. So where are you in your solution, life cycle? Do you have to change some tech stack. Do you have to migrate? Is a different context with different challenges for the CDP. Then, if you are creating a new product, a product where you think about not only the total cost of ownership, as you mentioned, right? You have to, you have to engineer it, you have to come to the start of production. What could be an early MVP? What could be a prototype, and what could be advanced afterwards? For example, if you are in the aircraft industry, if you have a good brand, your early MVP, your first version, can't be quick and dirty, the customers the market have a high expectation, and it needs a certain time to engineer that product, that that new plane right, and advancing the solution or the system later is a totally different thing. But how do we bring together the different areas, the different dimensions, the different topics, when it comes to creating new stuff and producing the stuff and sustaining the life cycle. And what I liked, what I learned in the past is from from Tesla SpaceX, your product is not your product. It's a production line, right? So, how do you integrate new items? How do you integrate new modules along your production line? That means engineering, creating stuff, should be done at the same time, at the same place, where you look over your shoulder, you see what production line does you as an engineer, you in engineering, you are like a fork, and if it's better than the production line, then you can switch. Otherwise you don't reach a DOD, you don't reach the D or in those different sections along a production line. So for me, it's, it's contextualize where you are, what you try to achieve, is very important. And another aspect that I see more and more and mark you mentioned it. We wanted to skip it, but I have to bring it in again. It's data management and design for AI and machine learning right massive data sets, if it's about the value chain, if it's about the supply chain, you can't ignore it. How do you how do you bring in the digitalization and the transformation regarding the usage of those massive data sets? You can't ignore it. It's, it's huge, right?

Mark Richards: 24:20

So I don't dwell here for just a second, and I think it's been the ever interesting journey for us as we've worked through the dimensions of going which dimension are we talking about now, how do we make sure we talk about this dimension or this aspect of this dimension, rather than the thing that we naturally talk about? And it feels like we've all just very naturally talked about modular design, right? And, yeah, modular cyber physical design, yep. But if I started to think this modular design feeding through my continuous delivery pipeline, right? So take as a given, we need modular design so that we can, you know, ship out new components that upgrade pieces of the puzzle. Already out in the field, but if I just thought about what it means in my continuous delivery pipeline itself, so the modular design is what am I feeding into the pipeline. You know, is my production line, part of my CDP. Back to your last comments. Stephan, right, what are the implications when I think of the CDP itself as opposed to the modular concepts I'm fitting into the CDP, what hints or tips would we have for people

Stephan Neck: 25:27

who goes first floor is open? I vote for the hardware guy. Ali.

Nikolaos Kaintantzis: 25:37

You have now a label the hardware guy.

Ali Hajou: 25:41

Interesting. No, I'll take it. I'll take it. No. So you know, if just correct me if I'm wrong. So I'm going to test something with with you guys. Interface management is the the ability to basically stay in control when interfaces are changing. It's basically agreement management, the best way of interface management that I've seen, which is also one of the most low key or the most low tech, which is bring a person who has been busy with one module and another person busy with another module that is related to that is interfacing with a module, and just put them next to each other, let them talk with each other, because the one has been spending weeks, months designing one module together with his or her team, and the other exactly the same. But whenever you put them together, even without them having the actual hardware, but maybe even with just having ideas and maybe some initial designs, that is already interface management, because it's, you know, it's, in a way, an interface is a set of agreements between one section of a machine and another section of the machine. And you would like to be able that both modules, they both together, or whenever they're, you know, put together, that the electrical current in one module is the same as the electrical current in the other module, or at least, you know, on those connections amongst those two modules, so that somehow the best way of just reducing risk. And this is, again, in engineering. This has been out there forever, but it's typically done as a side track with FMEAs, which is, which is a failure mode in effect analysis, that's nothing new has it's part of every hardware engineering project, but the point is, how can we do that faster? And how can we, you know, rather than decoupling FMEA, which is, in a way, risk management and the design of all of those modules, how can we not decouple them and put them together? So, you know, just putting people together, in a way, forcing yourself to test interfaces very early on, is is just, it's risk management. It's the ability to to to figure out whether, in the nearby future, one module or another module might get in trouble if there are any updates, and it's very often, it's just ideas. You know, one module might be more prone to updates if anyone has any smart, adequate, smart components in their house, you know, smart light switches and so forth. You know that for the longest time, the ZigBee protocol has been the go to protocol for connecting all of these devices so you could buy, you know, some components, some products from this supplier using the ZigBee protocol. They're another amount of products from another supplier using the ZigBee protocol, and that works together. However, recently, more and more companies has have been switching over to matter which is a different protocol, which basically means that if you buy a new light switch that might not work with all of your lights that you have installed in the last couple of years, which, by the way, costs a lot of money. I just it's, you know, these things, very expensive lights, typically. But it's, it's, it's an interface. And I've been working with a client where these teams have been developing just that smart lights, smart switches for curtains and everything that you have in your house. And what they've done is the people who have been working on the ZigBee versions of these smart light switches, they have been working on the matter versions of those same problems. Product. While in the past, that was typically a completely new team that was assembled to work on this new generation, new generation product, you know, using new technology. So the whole notion of stable teams who outlive projects who are working on, let's say, newer versions of technology who are still communicating with each other very early on, one module with another module, communicating very early on with each other has just been extremely useful. Why? Because you keep the knowledge within the team. The interfaces are tested very early on, they figured out that, you know what, there's probably a huge market of people, of customers who are using ZigBee protocol components in their house. So why don't we create something that works with both ZigBee and with matter? If you purely look at it from a business or a marketing point of view, that makes total sense. But if you look at it from the, you know, from the angle of project management, you know, project manager might be like, wait a second, that's a lot of additional time that needs to be spent in order to create something that is compatible with everything. And I don't have that budget, or, you know, we need to spend way more time which will create the how to go to, which will make the eventual products that we're going to produce way more expensive, or we have way more smaller margins. And, you know, it might not be interesting, but typically that doesn't have to be that that much of an issue if you just put the engineers who have been working on the same products that are part of stable teams just together frequently.

Mark Richards: 31:49

So if I summed up your advice, and it's so good having the hard work eye on the team, because you actually have so many examples to draw from. Thank you. When I think of CDP in a cyber physical system, there's a mental image that comes to my head. You know, those people who take dominoes and they build these amazing like 1000s of dominoes in complex, interconnected pathways, and you knock over the first domino, and then you just watch this show happen, and pictures get drawn, and eventually the final domino falls, right? I have in my head this mental map, and I think a lot of people do that, says I'm going to have all these automated pipelines, and there's going to be a huge amount of complexity about one pipeline. That's the domino for three pipelines. And when they are all finished, they merge back to another domination. And really all you're saying Ali is you can think of the continuous delivery pipeline as the different way people work together to optimize flow through it, long before you ever worry about super sophisticated, complicated, automated pipelines.

Ali Hajou: 32:55

Yes, sure.

Stephan Neck: 33:00

Listening, listening to you guys. What? What just popped up in my mind is Conway's Law, right? We're not only talking about creating and maintaining the CDP for the cyber physical system. This the second system that creates the system, right? How do you bring together? And you mentioned it Ali, the collaboration of all those people, and I remember folks the last few months and years with customers, and you mentioned it Ali, it's out there. It's common sense. It's common practice, common methods. Modularization is there. We know how to deal with legacy systems and so on. But when you change an existing system, you're not only updating existing interfaces, you are creating new ones and for the social system, how do you bring the new ones in? How do you familiarize those people with the collaboration, the communication that is going on. And I've learned in informatics, information systems engineering, if you have an interface, and if the different components are moving in different with different speed, you need the adapters. But from time to time, you have to get rid of the adapters, and you have to kind of Yeah, neutralize those by updating the interface. And for me, that's the same in the social system. And I think the CDP is is yeah tied to conveys law. Again, this context has to be considered, otherwise you will have the wrong impact on your hardware design, on your mechatronic design. And what I've read in the preparation Niko brought up in the old days, hardware was sold, but nowadays, we are talking about not only cyber physical systems. We are talking about ecosystems, right? So. That has to be considered as well, if you think about the CDP, if you think about the people along the CDP, creating those solutions.

Mark Richards: 35:10

So let's jump on this. In the old days, hardware was sold, and it was actually mine, right? And you know, let's face it, in the old days, software was sold as well. I remember very distinctly my father when Adobe moved to the Creative Cloud software as a service, and he was outraged, because he'd always owned his copy of Photoshop, and he'd always had a choice to go when there's a new version. Is it worth upgrading this one and and all of a sudden, it was like, I don't get to own it anymore. But let's think about that, that the old days, because you take everything we've just talked about, it's easy when you're in greenfields to go today, I'm designing new products, and I'm going to design them so that I can continually evolve them. And one of the first lessons I was taught by my first cyber physical customer was, you know, we don't get to forget about the sins of our past, because the very first product we ever built as a company is still out there in the field, and it's 1000s. And my curiosity is, if I'm a company that's kind of heading down this enterprise solution delivery path, and they've got the new products that they can approach with this in mind, but then they've got the old products that are out there and inescapably out there in their 1000s or millions. What have you seen in terms of people being able to lessen the impact of that legacy, tackling an old system and going and saying, how do I start to even though I'd never thought about continually updating live systems 10 years ago when I released it, what could I do today? What have you seen?

Nikolaos Kaintantzis: 36:51

I think you have to be under control which versions are out and which versions you want to support. So if you're 1020, years on the market, you cannot support your very first baby, even if you love to at the end. It's difficult because it's everything, not in not incompatible. But, yeah, if it's very old, it's it's different to maintain it. So you have to make clear to your customers that the old things, yeah, if they work, the work, but you have no right for for a warranty, guarantee or whatever, that we help you solving your problems and with the new way that you don't own the thing, the hardware, it's not your own. You're precious. It's just rented. We can just minimize the legacy out. I I just thought about it. I have one software that's really, really, really old, but I don't expect any service. It just works. I use it once, once in once in a while. And yeah, if it doesn't work well, I have to switch to a new version. So I think you have, you need a clear communication which kind of products, versions you support and which no and nodes not, and have kind of a roadmap like Microsoft has add on. I'm not sure if Windows three, oh, still has a support, but it's out there.

Mark Richards: 38:13

There's still plenty of code bloat in the current Windows providing backwards compatibility to Windows three. Oh,

Nikolaos Kaintantzis: 38:21

yeah. I think it's, I think dos is still working. Yeah, it's still backwards compatible. Yeah, it's one thing. Think about backwards compatibility. But I think if I call my Microsoft and say I have a Windows three, one machine in my basement, can you help me with the upgrade, with installing the new Windows Edge browser, will define me as crazy, and don't help me. What

Mark Richards: 38:43

about you, Mr. Hardware? Any words of wisdom on this one? I think what I heard from Niko was, make a commercial decision and recognize that sometimes you can't change the past, and you've got to just be deliberate in your stance. Have you seen something clever that allows you to do more? No, it's

Ali Hajou: 39:01

exactly that. It's exactly that the you know, in the hardware hardware world, the legacy products that are still out there are typically called installed base. So the installed base is out there and might need to be maintained depending on the product. Some products are built to last 10 years. Some products are built to last 30 or 60 years. It has all, let's say different. How should I call it, a different mode of working? Behind it, the only thing that I that I that I see, you know, that I've read about and that I've that I've had endless amount of discussions about, is the difficulty of finding people that are able to maintain the legacy systems the install base, and that becomes exponentially more difficult whenever we are working in process. Project where we just assemble a team, you know, we there's a customer out there that still has very old systems that are that were designed for 10 years, but we're 30 years ahead, and that customer wants to pay for support. So what do we do? We assemble a small, little team who's going to maintain those old systems. But the thing is, because you're just assembling a new team, there's no knowledge out there of how the system has been built, so everyone has to dive into documentation. It's just it's very much of a hassle. And then again, it's very boring to work on old systems. It's way more exciting to work on NPI, new product initiations, new things. So it's very tough to work like that. So I think the only thing that has been working very well, of what I've seen, what I've seen, is really to work with stable teams. So stable teams that are organized in groups of people around a product family. In the world of safe we call that value streams. They outlive projects. You keep these people together. And you know, of course, there's people that are joining the company, there are people that are leaving the company, where there are people that are joining the value stream, they're leaving the value stream, sure, but you you're always having that same product focus, whether it is, you know, the old generation of products, because there is still either maintenance that needs to be done or every now and then, a small upgrade of an old running system. But you keep the people together, irrespective of the project, irrespective of the the, let's say the initiative that they're going to focus on. And that's that transition I've seen is a very different, difficult one. And in some companies, they call that the so called doorbell principle, that if a machine is broken, you know, you know exactly what part is broken or what element module is defective. You know you can press the single doorbell within within your production company, and that doorbell roots back to the team that has designed it or has has maintained it, because, again, it's a stable team, but it's, it's it's much more of a it's much more it's not, again, it's not an engineering decision. It's not an engineering difficulty. It's much more of a business difficulty of, are we going to keep a group together who has been developing and maintaining these legacy systems, just to make sure that the knowledge is kept together and that we can gracefully, eventually, over time, gracefully say Bye, bye to systems that are not maintainable anymore. Yeah, I

Mark Richards: 42:43

think that'd be the caveat I'd add to that would be thinking about a product family orientation rather than a specific product orientation. Because I think the trap of those old products, where there's a limited number of people with the memory is particularly if it was once upon a time, a startup, right at the time, that it was a new product that was the best and brightest in the company working on it. And if the best and brightest who got you going are always stuck in the past with no chance to work on the new and they feel like the only way they get to learn work on new things is to leave the company and find a new job, you're in a world of trouble as well. So I think orienting yourself towards product family to go, Look, we need to keep that memory together, but we also need to give it a chance to have a blend of work be the other one. But on that note, let's, let's move on to our little segment, which is just one sentence you've got. What? Just one sentence you want to share for people thinking about continually evolving live systems? I'm going to kick us off, and it probably triggers quite nicely to the last little conversation. Just because you couldn't continually update your old live systems doesn't mean you can't plan to do it for new ones. Don't get stuck in the past mindset when you're building new products. Niko, what's yours?

Nikolaos Kaintantzis: 44:09

I just changed it on the fly after the preparation. So mine will be modernization, and the skill to deploy continuously is even more important with live systems and big systems,

Mark Richards: 44:20

beautiful and Ali

Ali Hajou: 44:22

Nice. So Niko is continuously evolving his sentence. Live nice. I chose for the sentence design for easy repairability,

Nikolaos Kaintantzis: 44:37

plus there's no verb on your sentence.

Unknown: 44:40

I don't care. I have four words. It's extremely efficient. Is trading design. Yeah,

Nikolaos Kaintantzis: 44:46

it seems like, sorry. Stephan,

Stephan Neck: 44:49

okay, that's okay. So I went back to to the meaning of the CDP. The old agile mantra for me is verify and validate for fast feedback. Yeah. How fast we can look,

Mark Richards: 45:01

all right. So that takes us to measure and grow. And in the world of measure and grow, I think we're gonna have to start with Niko, because Niko in our warm up chat before we went live today, when measure and grow was boring this week. So Niko, why don't you tell us why measure and grow was boring for you?

Nikolaos Kaintantzis: 45:20

I didn't found anything special. It sound like had it already measured. It already was in the last in the last dimension. It felt like hoping the movie, unexpected trilogy. It's too long and not really too much meat on the bone. And then I ended up with the last one. So Okay, finally, I see something that it's worth to dig deeper. So the problem was really many, many things were just there already, and we measured them already. So what I've chosen then, and I see Ali has his sticker there too, is teams and arts can integrate, deploy and release value on demands, independently on demand. Sorry, independently. And this independently demoralization, I think that shows what is important in this dimension. So can they really deploy value independently? And that's the fat, fat underlined, marked hypothesized word that's important for measurement independently so. Ali, yep, as

Mark Richards: 46:25

you said it, well. Ali, or would you add something else to why you voted there?

Ali Hajou: 46:29

Well, the reason why I selected the same measurement was simply because the I'd like to see teams to be independent as much as possible from each other, which requires you to design a system that allows teams to be independent, which requires you to have a system idea which is, which is which, which allows, I don't know how to call it, independent release, independent ideas, independent technologies, independent, at least independence. And you know, it's just, you're reducing risks essentially by by splitting up a large system into smaller systems. And I thought, the last, yeah, the last, the last measurement was the ones that was closest to it, because the continuous delivery pipeline is a concept which has been is quite generalizable in the world of software, but in the world of machine building, you know, every type of product might have a completely different continuous delivery pipeline, and continuous is also has a completely different interpretation in different industries. So I was, I just had a, you know, this, you know, my brain split over there, like, Okay, I just, I can't, you know, generalize CDP for hardware, even though, conceptually, I'd be able to explain it, but in real life, it's just, you know, every industry has a slightly different approach. Hence, I just filtered out everything that has CDP. I'm not going to do. I'm not going to choose that. I'm going to choose something else.

Mark Richards: 48:16

Fair enough. So I must confess, after all of our chat today, I'm about to shift my vote. Nice, because I think if the discussion, if the discussion of today, was modularity, modularity, modularity, right, your design shall set you free.

Unknown: 48:34

Yes, exactly. I'm

Mark Richards: 48:36

going to shift my vote. But just before I shift my vote, the reason I originally settled on being able to separate deploy from release was, I think, in because so often there's a there's a gap between you making something available and somebody choosing to opt in. And that gap can be driven by quite significant reasoning, such as Niko waiting for text not to be buried under the sea. Or the reason that, you know, my car, which gets software updates delivered regularly, which I don't trust, to upgrade the operating system on my car on demand. So I never take the you know, Wi Fi delivered updates to my car, I'll wait for them to take it when it goes in for a scheduled service, because I'm just fearful. And so I think that that ability of separation, that was what was made me there. But I think the modularity I'm I'm with you boys, Stephan, you're the last holdout

Stephan Neck: 49:38

I'm getting a whole time now. Okay, as every week, I struggled, like Niko, what to choose, and my thoughts went back to, what is the ultimate what is the ultimate purpose of the CDP? It's delivering value and the most inclusive measure and grow. Here is the solution. Architecture accelerates exploring, integrating, deploying and releasing new value. Let me quickly explain why I chose this one solution. Architecture, for me is not only a technical viewpoint or perspective, it's also how do you architect your systems? And again, bringing in the solution, says the social system, bringing in the organization, right? And that's the interesting part that I had with all those discussions in the past few months with customers and clients and involved people, where they started thinking about, Hey guys, we probably have to rethink our system in terms of it belongs to the bigger system. What are we after really? How should we evolve over that time? And for me, I chose this one because it's the most inclusive one that includes exploration, integration, deployment and releasing towards new value. And how do you differentiate as a as a company, as an organization, if you're out there in the red ocean? Yeah,

Mark Richards: 51:09

I looked at that one, and I went, that was already in the measure and grow for DevOps and the continuous delivery

Nikolaos Kaintantzis: 51:17

pipeline. I thought the same. I told to myself, is this really important for, for this, this dimension, for, for, for this involved life system thing? And I thought, so, yeah, it's important. The same with your smart it's really important with your old one. It's really important to separate this idea of deploying doesn't mean the end user has it. It's really important thing. But was everything was on DevOps. That's why I've chosen the last one, which seems much more proper for for involving live systems. And with you

Mark Richards: 51:51

all right now, we do have an option always for somebody to go. I've got a measure I'd like to add. And we were all devoid of inspiration, except Mr. Hardware.

Ali Hajou: 52:06

I mean, I'm fully with Niko when, when he said, You know what? These, this list of measurements, yeah, it's pretty boring. There's a lot of repetition there, indeed. And I was, I was thinking. I just pushed myself and tried to think, Okay, if, in the context of this dimension, I would like to measure something which is very tough, probably, but still, if then I would like to measure, or take account of the amount of changes that are happening in interfaces, quarter on quarter, because you would like to have that amount to be zero, or as close to zero as possible, which requires you know to be an interface to be a well thought out from the get go. But I would, I would try to take a look at that

Mark Richards: 53:01

interesting, because I had my head back into versioned interfaces in software, and then I went, well, that's kind of like building shims in hardware, and engineers had shims. So it's exactly what it is. It's an interesting one, isn't it?

Nikolaos Kaintantzis: 53:16

I really love it, because usually, usually we try to be agile, agile and faster and faster and saying some things has to be stable. I love this

Mark Richards: 53:24

very interesting. All right, so. Jeanette, Sorry, go on. Stephan, you had some more words

Stephan Neck: 53:30

just to answer an Niko statement, agility only works if you move from one consistent state to another one, right?

Nikolaos Kaintantzis: 53:39

I know. Thank you. Yep.

Mark Richards: 53:42

All right, so we, I assume our extended gaps at the moment are happening because the Swiss love to have lots of holidays. But it's three weeks. We are back on the 10th of August, and we're done with enterprise solution delivery, and we're going to start talking about something that it feels like I spend almost all day, every day, talking about at the moment, and we're going to be in the world of Lean portfolio management, talking about strategy and investment funding.