Lean Agile Requirements in Large Scale Complex Systems

Building large-scale software and cyber-physical systems (CPS) are one of the most complex and challenging endeavors in the industry today.

Emerging practices from industries like Automotive, MedTec and Finance offer a different approach to handle this inherent complexity. This new approach focuses on a solution centric development, where the solution itself determines the kind of documentation produced, not necessarily the methodology employed. This leads into a separation of transient (development life cycle dependent objects) and solution objects.

Today, in this VUCA-World, most complex endeavors don’t start as a green field. A considerable amount of existing solution documentation (such as architecture/s, models, design decisions, and requirements) may have been accumulated over several years or even decades. Additionally, in some industries, e.g. automotive suppliers, the expected lifespan of newly built solutions is more than 20 years. This means that the future engineers who will work on these systems are currently attending Kindergarden… We must take this into consideration and ensure that current and evolving requirements, design and intent are clearly stated and documented.

To this end SAFe offers ‘Solution Intent’ which serves as a “repository for storing, managing, and communicating the knowledge of current and intended Solution behavior.” This elegant solution allows us to capture the knowledge and the basic understanding of current and evolving requirements,
design, and intent—that is, the larger purpose of the solution.

For this webinar, join Peter Pedros, CEO and Founder of PEDCO-Applied SAFe as we outline some of these emerging approaches and discuss the pros and cons (based on experience with Applied SAFe, the implementation of SAFe as a process model).

Watch the webinar replay below!

Myriam
So welcome everyone. We’re very glad to have you join us on this Wednesday for a Webinar on Lean Agile Requirements Engineering in Large Scale Complex Environments. My name is Myriam Sanie and I’m with the marketing team here at cPrime, and joining me today is Peter Pedross, founder and CEO at PEDCO, who will share with us some actionable items that we can take back to our organizations to help us with the complexity of a compliance in regulated environments. So without further ado, I’d like to let Peter take the reins. Peter, the floor is yours.

Peter
Thank you Myriam. Ladies and gentlemen. Good morning. And a warm welcome from Switzerland. My name is Peter Pedross and I’m founder and CEO of PEDCO. In today’s webinar, we discuss lean agile requirements engineering in large regulated environments, specifically an emerging practice which we call a solution. Many thanks to our partner cPrime for hosting this session today. Before I start with the content, I would like to introduce myself as I started at the age of 15 to program code for money, have worked in almost all possible positions in the last 35 years from Cobol Assembler, to small program or to tester to project lead software architect, quality manager. I studied also software engineering, finance and management psychology at the University of Zurich. Before I founded PEDCO, I was working at the leading Swiss financial institute where I was responsible worldwide for the entire set of life cycles, processes, methods and tools used for 9,500 developers. We also had to comply with regulations from the Swiss regulations from the Fed, London and Singapore. And we also had to reach CMMI Maturity Level 3 worldwide.

I have published several articles and given several lectures. My first one was about distributed small talk at the IBM conference in San Francisco in 95. And the last one was about scaled agility in regulated environments at the Agile in Automotive in Stuttgard some weeks ago. This is where Mercedes and Porsche come from. Of course, I’m also certified for the Scaled Agile Framework and Disciplined Agile Delivery as well as to European Foundation for Quality Assurance and CMMI. I’m also president of the board for computer science at the Swiss Association of Quality and also a member of the board of directors there. Scaled lean agility s being adopted extensively throughout the industry. Building large scale software and cyber-physical systems is one of the most complex and challenging endeavors in the industry today and for that, I will lay out the context of such an environment and aspect of requirements in SAFe.

There are some emerging practices from the industries like automotive, medtech and finance, which approach this complexity in different manners and it goes into a direction of where, not the methodology determines the kind of documentation, but the solution itself, and this leads into a separation of transient and solution options. We call this practice the solution centric development. I would like to outline some of these aspects and discuss its pros and cons. What I’m talking to you is mostly based on our experience with Applied SAFe, our implementation of SAFe as a process model. This webinar will conclude by some lessons learned and QA session.

The last 10 years have seen the disappearance of well-known products and the arrival of new competitors in the marketplace. Who would have thought in 2007 that Nokia would disappear as one of the great brands in mobile phone and that apple will take over the smartphone market almost overnight. With the rise of new competitors such as Uber, Tesla, Airbnb, Netflix, Google Facebook, and the others we witnessed. The rise of market pressures in all industries and new levels of innovation and efficiency are required as we combine such systems into cyberphysical systems. With the Internet of Things, Industry 4.0 (German), Lean startup and Agile, we move into an increasingly complex and interdependent realm. Creating agile teams has helped to get where we are today, but we also face limitations as small teams cannot build such complex systems in a timely manner. With this in mind, we also have to face regulatory and organizational environments which are becoming increasingly demanding. In the past it was some kind of a gentleman’s agreement, but these days have really changed. A study by Scott Ambler from Disciplined Agile Delivery shows that most agile team, delivery teams, more than 65 percent, are already facing compliance requirements, either regulatory and/or from their own organization and given this situation it is the best strategy to see this governance for such an endeavor.

This brand new study from Version One lists the most used frameworks at scale. The survey just came out two weeks ago and it shows that about 50 percent of all projects are either SAFe or Scrum of Scrums. And if you take into consideration that some sort of Scrum of Scrums is also included on the Program level of SAFe. This is the reason that we will show examples mostly based on experience with SAFe. Compared to the last year’s survey, the number of internally created methods and Scrum of Scrums has decreased in favor of SAFe, that is Disciplined Agile Delivery (DAD) and LESS have increased a place and I think that it’s fair to say that the more formal methods are picking up. These are some of the results from the same study and we see here at the top five tips for success with scaling agile. In turn, in 2017, Internal Agile coaches was rated as the most important, followed by executive sponsorship. In 2018 consistent practices and process has now moved to second place from third second. This is a strong indicator that SAFe is more and more really applied and practices and processes are getting more information. to you before.

In our experience with Applied Safe, we see the regulated domains exhibit varying levels of criticality from safety critical to security critical. One of the core characteristics of regulated environments is the necessity to comply with standards, regulations, directives, requirements, specifically requirements engineering and its management, as we will see the example of Automotive SPICE. There are a plethora of regulations and standards which apply across different regulated domains. As an example, the ISO 26262 and many more. These regulations are issued by a number of bodies or associations.

As software plays an increasingly important role in regulated environments and it is sad to see that a lot of people say that agile methods and regulated environments are fundamentally incompatible. And therefore just don’t apply agile in regulated environments. It is our strong opinion and also that the Swiss Association of Quality, that this is false and agile can indeed be applied in regulated environments, but it needs a defined logic. The granularity at which development process are expressed and adopted requires careful tailoring to the specific regulated environments. For example, some will require full traceability and some not. In agile, the talk is about the agile manifesto with the 4 value propositions, which you already know, individuals and interactions over processes and tools, customer collaboration over contract negotiation and responding to change or for new plan. If you take that into consideration, quality, agile processes, follow a logic in a plan, do, check, act cycle where some development is planned, done, results inspected and adoptions or made to improve the process to solve any problems that may arise and the principles of the agile manifesto do not include an overarching set of principles for regulated environments, but a number of core issues for software development in regulated environments may be inferred.

These issues include quality insurance, safety and security, efficiencies, traceability and verification and validation. In practice, weneed to have a quality. We need to have a managed process and an established quality management system. We need to talk about safety and security. We need to have transparency in execution and continuous compliance. We need to have a responsive planning and risk management to mitigate safety risks for users. All reference models, and this is really funny, talk about that you need to manage the process and the solution variance. You need to reduce waste in your process and do exactly what is needed. It is not OK to just define a huge process and say all projects or will comply with that. And what we also need is to have some kind of traceability. We need to ensure that process and product compliance is reached as well, and of course we are talking about verification and validation, that we have engineering based on principles and real practices so that we know how a product is specified, designed, built, and tested and tested in accordance with regulations. Based on the criteria is presented in the slide before we built our product, Applied SAFe.

It allows to implement SAFe in a fast and precise manner as it includes a comprehensive process, models with rules, activities, templates, guidelines, metrics, tailoring, milestones and as you can independently instantiate on each level, you can customize with the building, tailoring and their own multiple concurrent process variations and of course you can extend, adopt and integrate with your own processes. It includes the complete content of SAFe, and each word is elaborated in sync for future versions of SAFe and of course officially approved by Scaled Agile. It includes some compliance mechanisms and capabilities for regulated environments as well as a complete implementation of relentless improvement.

Now let’s move to the topic of solution development. In SAFe from the perspective of requirements engineering and what we see here is an overview to the solution development in SAFe. As a requirements engineer, I can see that the solution is delivered by the value stream which is realized by the people from the agile release trains. And most interesting for requirements engineers is that the solution intent captures the goal of the solution and allows us to explore and define the fixed and variable requirements, which are in part, derived from the solution context. The customer interacts with the solution builder to clarify the intent, validate assumptions and reviews progress. Solution management and the architects help to drive the development, makes priority decisions and manage the flow of features and capabilities and non-functional requirements. And this is the key of requirements engineering. If you go further in detail of the previous slide, we see here an element of SAFe, which is called a solution intent. The solution manages the product itself.

It includes the product lifecycle management as well, and when you build a system that has an unacceptable high cost of failure, one significant barrier to the agile adoption is the need for a more rigorous definition and validation of system behavior. While many practitioners resonate with the agile manifesto values statement of working solutions over comprehensive documentation, those can become conflicting priorities for enterprises which need both. Engineering of complex and highly reliable solutions requires a large amount of technical information. Much of this information reflects the intended behavior of a solution. Features, capabilities, stories, non functional requirements, system architecture, domain level models and designs, for example, for electrical and mechanical engineering, interfaces, customer specifications, test results, traceability, and more. In many cases, this information must become part of the official record, whether it’s out of necessity or regulation itself, and building systems for which behavior must be insured, including the life’s critical systems, mission critical systems and other systems governed by regulatory standards, for documents, traceability is a means to help ensure that the system behaves as intended. Traceability connects the elements of solution intent to each other and to the components of the system that realize the full system behavior. Basically, the solution intent contains what the current system does now on what changes are intended for a future state.

So this example here is a part of a domain model and it also shows the relevance for requirements engineering. For example, the solution. This is a word which is commonly used, but what means that in reality we see here that it’s made out of the current solution and the future solution. And how can a company ensure that several hundred people from have the same picture or the same relations in mind as we see right here and such things needs to be defined clearly. So if you’re talking about solution intent, we see here that the solution intent imposes the future solution and we see that we have requirements which are either a non-functional requirement or a backlog item and all backlog items in SAFe are either story, capability and feature. The solution intent itself is steered by the solution vision. The solution vision steers also the solution roadmap. And this is something that needs to be clear for all participants in scaled agility.

In support of bringing the benefits of lean and agile development for complex systems, SAFe provides also a scalable requirements model as a way to express system behavior with epics, capabilities, features, stories, and non-functional requirements. These artifacts largely replace traditional system requirement specifications with new paradigms based on lean agile development. These newer paradigms are intended to avoid focusing too early on a point solution, and this is based on not to pick specific requirements and designs far too early in the learning process. It rather leaves room for an emerging understanding based on intent and not specificity. For example, a feature is described by a phrased benefit hypothesis and an acceptance criteria. A story is elaborated by a user voice statement and an acceptance criteria. Also patterns under relationship for attributes, acceptance guidelines and testing are included to support the NFRs that are also imposed on the world’s most important systems.

Taken together, it’s a comprehensive mode, as we see here. But most practitioners, they only need to see a portion of these items. For example, agile teams primarily work with user stories, story acceptance test, and NFRs and each element is defined, provide the right amount of expression of behavior and testing at the various level of SAFe: Portfolio and Large Solution, Program and Team. You may say that this model appears complex, but that just reflects contemporary solution development at scale, which is indeed complicated, even with agile methods. Of course, if an element is not needed than it does need to be used. For example, capabilities are only needed if they have the large solution level. However, teams and programs that are building world-class enterprise solutions at the highest possible quality can probably apply most of these elements.

There is also help published in the form of various guidance papers, such as CMMI guide to Scrum and others. For example, we have the famous TF45 paper from the AMI, the Association for the advancement of medical instrumentation. We have here the link and we really recommend this paper as it describes how agile practices can be applied and developments still comply with medical device development regulations, such as the FDA Chapter 11 requirements. As we can see in this example, this guidance paper also talks about incremental development of stories and does not impose that a big upfront specifications needs to be built. We are very proud that one of the authors is also a member in our advisory board.

This is now a real down to earth example of how to deal with regulated requirements and how to link to a defined process element. We start here with the requirements of Automotive SPICE. If you look at the example of requirements specification, specifically 1703 stakeholder requirements, we see what is expected and needs to be defined. Up here is all the content from the reference model for that requirement. The lower part is if and how strong this requirement is sustained with the actual defined processes such as work products and more. For example, non-functional requirements as stated in 17.03 are reflected on the program level with program NFRs. Such a mapping is definitely something that an auditor likes to see and sometimes they even kiss your feet if they see something like that. We think that working with such a reference model is not a burden. They rather represent a huge book of knowledge and build on past things that really went wrong. So for us it’s a very good practice to take an applicable model to assess your own processes, even if it’s not required from the regulatory.

I wonder if you already know that emerging practice, which is solution centric development. This is something that we first encountered at one of the major Swiss financial institutes. There we had a problem with a system called WS80. This was a trading solution which was set into production in 1980 and was enhanced permanently by about hundred developers. So they were billions of dollars spent into that system. And of course a lot of documentation and requirements in various forms already written. So the question was how to deal with changing requirements and what do we do with the existing documents and this led us to the practice which we call now the solution-centric development and we have seen that’s now that similar practices are used in other industry like medtech and automotive as well, so here comes the problem statement.

If you think as a requirements engineer and you mentioned yourself, you have to start a new job in a large company, so what do you expect? Will you work in a green field approach is the rather lean startup and you don’t know what it will be in the future or do you really know that the solution needs to be maintained on 13 hands for the next 20 years to come? Or will you add or change requirements to a big existing documentations which probably are Nassi-Schneiderman diagrams or use case model and so forth. In the case of number two and three, you have to come up with a strategy on how to incorporate the fact that people working on this solution will change. They may even just be in the kindergarten right now, so you will document for them as well, and will the new feature, which is basically a new alternative flow in a use case, be described as a feature with some stories and existing use cases will not be touched. Will the new business rules be described as a simple story, but what happens to the large complex business rule document? You need to know how to handle that and what is acceptable not for you, not only for you, but also for the company in the long run. For these kinds of questions the solution centric approach comes into play.

Here is an overview as a skeleton to build up requirements with a natural evolution that we see here an anthology in architecture, and this is a possible solution. We started with a useful patchwork documentation based on a skeleton, which can be completed step by step.

For example, a first version would just be a small domain model and a use case model. In the next iteration you may add a floor which represents a use case or a flow with a use case, and if you document like that, then you need some guidelines to build up such an evolutionary and incremental documentation. Some companies, one in the Netherlands, they even decided to not specify acceptance criteria for their stories, but they directly trace from there to the right point into the skeleton. The skeleton must hold all attributes to scale up and support the needed flexibility and support for variations, and for that, a natural approach is best to comply with. And one discussion topic that we face sometimes as well is that applications are either already existing, include a lot of diagrams, structure diagrams, Nassi-Schneiderman diagrams, use-case models, they have a long expected lifetime, so depending on the attributes of a solution, it might be necessary to extend the definition of Dones and adopted to existing documentations. We work a lot with the DoDs and in such a special case you would consider features and stories as so-called transient objects or change requests and use it as a trigger to edit existing documentation, the permanent solution objects, so we have your stories and they’re just are triggering new additional alternative flows in a use case on the and with each new stories, you add new additional behavior and trace back to the triggering story.

Flow is based on agile with the features and stories still, but these are only used as a trigger to update requirements called tests and its documentation. Criteria for a solution centric approach is to make transparent what solution must sustain over the long run. The best practices to establish decisions, and here we have some typical criteria: the expected lifetime, existing solution and documentation, business criticality, intended use internally or externally, safety classification such as class A, B or C in ICC2304 or your own external security classification, HIPAA reference model, expected maintenance and complexity of solution size, previously used life cycles. Some best practices here is that you adopt and extend the DoDs (the defintion of done) sto the existing documentation. You’ll make decision transparent also for the auditors. You define a decision table in order to have condition decisions and to model the meta-model domain, and even for each type of solutions, tailored to the needs of the solution rather than the life cycles and on that, do not be dogmatic.

We see a real world example of a work product model which describes the domain and shows the main objects interrelationships. Different processes have different colors, for example, pink for requirements engineering. Such a model needs to be in sync with its producing processes and is a visualization of its results,. Tts communication media of any subject matter discussion within the domain and beyond. It helps to visualize key information, for example, we see that they’re transient objects. These are the blue ones or the solution options, which are the orange ones,. If there are mandatory here with them or if they’re tailorable or not. So this is all kinds of information which we had in such a model. Such different behavior can be driven with definitions of done and I guess that we are all aware of definitions of done for stories . and the definition of done is an agreed list of criteria that will be met for each story and without that, we don’t have out consistent meaning of done, and the estimating and velocity cannot be estimated. One very interesting point in scaled agility is that we not only talk about the story definition of done, but also on the label of team, systems, solution and releases, and here’s the representation of SAFe. We talk about specific definitions of done, for release, as required by the business, solution at least every PI, system increment at least every iteration and team increment at least every iteration and of course we might have even definitions of done for features, capabilities and epics.

In automotive, for example, The V model is the most commonly used approach for engineering and teams and especially requirements engineering are facing milestones. For example, like here, requirements freeze. Now it’s just a reality and it can’t be negotiated, you can’t negotiate with Wolkswagen. So new kinds of approaches need to be met. And this is something where the skeleton approach works really well. We use iterations and within them, we reach these milestones as well. So with such increments, we learn and fail fast and this is what agile is all about. So I would like to summarize our lessons learned from several customers and depending on the characteristics of a solution to be built, you may consider different flavors of documentation and approaches. For example, lifetime business criticality, existing documentation as a solution, expected people turn over, and time to market. A set of solution types help and the decision table helps to select the right agile requirements engineering approach. You base decisions on given facts and let it determine what kind of requirements documentation is needed and decision table. Design the solution for small batches and you should demonstrate compliance and correctness in each iteration, and in small iterations and you should avoid quality depth. You don’t build in compliance at the end of development. You do that with each iteration and just treat this as a part of normal system demo. Continue to teach. train, coach, and we think that agile quality management systems and reference models are tools, not a burden. So you define clearly which activities are core on which level. So there are some good practices, defined guidance on DoDs. A lot of different behaviors can be handled for each project. None of these approaches require a big upfront design. You shouldn’t document at the end and avoid any technical debt. And as I said, some companies who replaced the acceptance criteria with requirements specification directly. With each system demo, all related to any iterations should be completed and you should avoid policing behavior or replacing good judgement by processes. With that I’m finished with my presentation. Before I start with the questions and answer session, I just wanted to remind you that we are next week in the Detroit area. We are at the Agile in Automotive in Novi on May 15th and at the Agile & Beyond together with cPrime in Ypsilanti from May 16th. Hope to see you there. And now I would like to hand it over to Myriam. Myriam, do we have some questions?

Myriam
Yes we do. Peter, thank you so very much. If you have any questions, please enter them in your chat and we will try and get to as many of the questions as we can. So we have a first question from Francine. She asks from a quality perspective in regulated environments, is that all that’s needed for requirements engineering or do we need more?

Peter
This is a little bit what I said in that slide where I said that we need to have the quality and defined processes and if you have defined processes then we need to tailor them to the specific needs and we need to map that to the reference models. And then we need to show how we do the practice. This was at the very beginning where I showed the different reference models. And if you have that in place, you’re pretty fine with that and you know what you are doing, because what an auditor wants to see, they want to see that you know what you are doing, you do what you say and that you can prove that you are doing it. Myriam, do we have any other questions?

Myriam
Yes. And I think a relevant question with regard to technology that helps us to do these things. So Michael asks, how do the off-the-shelf products like Jira, for example, map to the epic and the stories on the data model that you showed a and then you find that some map better than others with regard to compliance?

Peter
In our opinion that was our experience with Credit Suisse. You know, when you have 9,500 developers, if you talk about Jira, we are talking about tools and in the model which I showed, we just say there are stories, or there are features and capabilities and they have a relationship, and this is the consistent part of the life cycle. Now we can say that some teams, they work with Jira, others with Team Foundation Service, but they have the same understanding of how that relates and what you can do now is you can say, hey, we have two, three or four different standard tools in a company and the teams can choose which one they want to use out of the pas existing code, for example, existing documentation, to use that tool or another tool, but they still have the same domain model in mind. So I would strongly recommend that you differentiate between what you are doing, the work products, the objects to solution objects and how you represent them in tools. It can be done with various different tools in place. I hope this answers to questions. I know that the reality is sometimes complex. We have four standard to requirements management tools in one leading Swiss financial institute and we still managed to have like five exceptions to audit tools. So we have always a lot of different tools for the same work to do.

Myriam
We have a more global question with regard to a SAFe. Pankaj asks, with SAFe advocating built-in quality, within the agile teams, where do quality departments fit in, especially with regard to compliance?

Peter
The quality department fits in an enterprise level, this is the small button when you look at it on the Scaled Agile Framework. You see here at the big picture the full SAFe configuration and up here is the enterprise. And of course here you have the whole organization for the quality management departments. They have the processes of how to develop the process for all the levels, how they can built the quality in the team, but you have to define those mappings to reference models, the activities, the work. But I’ve seen all those life cycles, the roles this is all done by the quality management department here on the enterprise level, and of course, they are doing also process assessment and training and skills management. So this is usually represented at the enterprise level. For example, in Applied SAFe, our highest level is the enterprise level and this is what is needed in regulated environments. This is a really good question. Thank you very much. Are there other questions?

Myriam
So we have another question related to kind of the higher level. Matthew asks, how you would come about for a single roadmap and vision and how you would share that architecture and delivery for a large enterprise, for example a bank. And let’s limit the question to compliance for a financial institution.

Peter
Vision. This is something that I haven’t seen so much on any specific defintions of how this is done. This is usually text and a document, but what I see strongly is software architecture. This is something that is quite defined and what we use several times is that we either refer to the ISPMA, the International Standard for Process Management or the description which is from the Software Engineering Institute in Pittsburgh. These models describe how you should document and define your architecture. And on the architectural level I think it is pretty much well defined. On the vision level, it’s more or less open space which you can concur. We just define it with pictures and text. This is my experience. Hope that helps with the question.

Myriam
OK, Peter now we have a specific question on one of the slides you presented, the mode that had the blue and orange objects that define the architecture. Are the product names always the same?

Peter
Yes, we strive for that when we define such a model, basically the first version we took was based on an iterative approach, this was based on the Rational Unified Process, and we had so called a translation process, which was underneath the model for the various names in the different lifecycles. For example, in waterfall, we ask for a so-called risk list, that was the defined artifact, and in agile, you don’t have risk lists. But of course you are dealing with risks. So this is something we took out of the model, and that every lifecycle deals with risk, and the risk list is just a specific report, so these reports are lifecycle specific and transient objects. But if possible, we always kept the same names. Hope that answer the question.

Myriam
OK, great. Thank you. We have a question specifically about the Applied SAFe product that support any time of compliance?

Peter
Yes, we have bindings to Automotive SPICE, to IC62304, to Chapter 11, to CMMI. When we built Applied SAFe, we started with reaching CMMI Maturity Level 5 out-of-the box, and that was our goal, and since then we have learned that most reference models, they’re basically asking for the same things, having a defined process That’s like the first question from the quality model, having a defined process, know what you’re doing, have defined tools, practice metrics in place to prove that you have done so.

Myriam
OK, great. Thank you so much Peter. So we’ve come to the end of our questions. I want to thank Peter very much for giving us a great presentation today and I’d like to thank you all very much for taking the time to join us today on this Wednesday and we hope you found this session of value. And as mentioned before, we will be sending out the recording of the presentation as well as the slides and we hope to see you soon in one of our other webinars. We appreciate your time and enjoy the rest of your day. Thank you.