Agile: from Drawing to MBSE/Enterprise Models with SPEM? (Last update 13/12/2018)

Introduction: about agile approaches

Time pressure, uncertain context and some negative experiences?for traditional Waterfall model approach are at the origin of the current success for adopting Agile (cf. the Agile Manifesto ) approaches and practices .

However it is important to point out that such approaches are not accurate for all kind of projects and systems, and that they came with?some?Strengh but also with some Weaknesses as well. It is also to be pointed out that?some other processes also addresse the issues associated to Waterfall, such as Open Up (a compromise between RUP and agile) or Spiral model (cf. next figure).

If considering the Scrum approach (which is one of the most popular today), it should be considered that it concerns small projects in terms of duration and team size.

It implies to create a multidisciplin team involving developers but also users with a high availiblity, if possible co-located and adopting an iterative way for developing and deploying and capturing users'?feedback.

With such approach, the Minimum Viable Product (cf. previous figure) is delivered very quick and then continuously enriched through capture and prioritization of requirements and associated tasks to perform.

But how to deal with complex products or organizations, with many projects and Scrum teams?

Such question is at the origin of emerging agile methodologies at scale, such as LeSS (Large Scale Scrum), DAD (Disciplined Agile Delivery) or?SAFe (Scaled?Agile Framework). A more complete landscape of Agile is provided in the next figure.

When having to defined their internal agile processes, based on one or several of those methodologies, the enterprises will have the choose between:

• Creating documents describing the process, including description of deliverables, associated templates and tooling for supporting the process (e.g. Backlog solution)
• modeling the process

Also, different approaches can be adopted concerning how to deal with documentation:

• within the code,
• as a set of shared and managed documents,
• using models of reference from which documentation (and sometimes code) are derived,

... knowing that many agile developers do prefer the first one, putting the priority on quick delivery. Does it mean that adopting agile in enterprises mean forgetting documentation and model driven approaches?

This article explores these questions. In particular, it explores if and how engineering processes, agile and not agile, can be modeled and aggregated, taking advantage of SPEM (Software & Systems Process Engineering Metamodel 2.0, OMG' specification) and including model based development. As model based development and agile approaches are not anymore restricted to software development, but also System Engineering, the realized exploration also includes Agile for MBSE. Finally, "Agile at scale" being to be considered at the scale of enterprises, Enterprise Modeling as a way to govern the evolution of the Information System in alignment with Enterprise's objectives and strategy is also considered.

The first part introduces SPEM and some tools allowing to use it, with underlying expected capabilities (Eclipse Process Framework - EPF - and Sparx' Enterprise Architect) and usage sample.

Then the second part explores how SPEM can be used - or not - for capturing SCRUM. As phases, iterations and milestones seem to be the engineering process items which are the most problematic, it is also studied in this part if and how other agile methodology at scale consider them.

The third part illustrates experimentation of capturing an agile methodology with SPEM, EPF and EA.

Then the conclusion presents some lessons learnt from the experimentation described in the third part, and then lists a set of questions which arose when making the survey about Agile:

• Q1- role of architects and usage of models?
• Q2- Closer link between SPEM and Model Based Approaches?
• Q3 - How can Agile be used with Product Life cycle Management and Digitalisation approaches?
• Q4 - Considering new life cycle models for circular economy?
• Q5- Is the Agile manifesto acceptable and sufficient?
• Q6- What is the impact of uncertainty and variability concerning success of Agile (and is it sufficient for addressing it)?
• Q7- What are the condition failure for Agile at scale, and can always the enterprise apply it?
• Q8 - How should be addressed Continuous Integration and how does it impact agile software process engineering?

For all these questions, some information are reflected concerning the state of the art and the state of the practice, with many external references.

Dedicated articles will be proposed very soon addressing each of these questions.

1 -Modeling a process with SPEM

Many process modeling languages exist, being standardized (BPMN , XPDL , SPEM ...) or not. Why some many languages, including standardized ones? In fact, those languages were created for many different usages and needs.

SPEM was created in order to support the following use cases:

1. manage libraries of reusable Method Content
2. develop and manage Processes for performing projects
3. configure a cohesive process Framework customized for each project needs
4. create project plan templates for enactment of process in each project context

It comes with an implementation of reference, the Eclipse Process Framework (EPF ), with illustrates usage of SPEM and demonstrates the ability to customized and enact processes using different configurations.

With the specifications comes a SPEM UML profile, for usage with UML modeling environment, but also a MOF metamodel for usage for Domain Specific Language supporting environments.

SPEM provides structuring elements:

• a method library
• method plugins and configurations
• method content packages and processes packages
• delivery processes and capability (or process) patterns
• process planning templates

Considering classification of modeling constructs used with ArchiMate (Active, Passive, Behavioural), we can consider that we have:

• Behavioural modeling elements: phase, iteration,?milestone, activity, task definition, task definition usage and step.
• Active modeling elements: role, role set and tools
• Passive modeling elements: work product (with artifact as a specialization)?and guidance (with many specializations proposed by SPEM2 and implemented in EPF: guideline, roadmap, tool mentor, white paper, etc.)

If considering all the concepts defined in SPEM, about xxx concepts are defined. I derived an OWL ontology from SPEM (as I made it for ArchiMate ) in order to make it easier to have an overview of all what is defined in SPEM. It can also be a way for linking SPEM models in modeling environments with other data hosted by other applications.

It was also needed to clarify SPEM constructs which are structuring physically the model (packages, containments) in order to ensure it is appropriately used for collaborative modeling by distributed teams. If the question doesn't arise when using EPF, which already made some choices with predefined and frozen structure, it is less obvious when using a UML modeling environment, which gives more freedom. However, it is important to agree on a commonly agreed structure in order to facilitate collaborative modeling. My choice was to be as closed as possible to the one proposed by EPF, with some adaptation (when needed) resulting from the way SPEM is implemented/supported by the used solution. The following figure illustrates what could be an agreed structure for a team having to deal with a shared library within an organisation. Let's notice that, as standard SPEM Guidance categories are not part of the SPEM profile used by Sparx's Enterprise Architect. So my choice was to create a dedicated package for each and to create guidance instances.

In order to ensure the good usage of SPEM with the selected modeling environment, it was needed to have a look at the specifications (in order to ensure the whole alignment with the specification - or eventually capturing if something missing or different) and at EPF, in order to have a good idea of the service to be provided for supporting modularisation, tailoring and publishing of selected configurations. Such study allowed to point out potential restrictions in terms of expected functionnalities for supporting SPEM use cases when using UML modelling environment with a simple SPEM profile. Also, it allowed to find opportunities for additional functionnalities or alternative ways for publishing processes and manage them in configuration.

Finally, I also studied the different published processes, being by the SPEM community (SCRUM, XP, UP) but also at the outside. In particular, I identified an online model of the TOGAF 9.1 at the APG (A Process Group) website. ATPL demonstrates that SPEM can be used for a whole and complete methodology, and for tailoring enterprise processes combining several processes.

I was then ready to attempt modelling Agile processes with SPEM.

2- Modelling Agile process with SPEM

I first considered how SCRUM is usually described in the documents of reference for the methodology. One of the ressource of reference is the SCRUM Guides web site .?Looking at the description of SCRUM,?I?identified that the terms "phase", "milestone", "activity" or "task" are not used.?Used terms are "scrum", "backlog", "sprint", "iteration"?and "work". How then to model SCRUM?with EPF (EPF being a metamodel, so a language for modeling a methodology)??

I had then a look at SCRUM model proposed on the EPF web site, in order to see how they addressed the problem. In this model,??only tasks are defined. No?phases, no milestones, no activities. Also, the role of architect is not considered.

It is difficult, considering enteprise processes for project management, to imagine that no phases and no milestones will be defined. It is also difficult to envisage what could be the role of an architect (solution, system, enterprise) when using SCRUM.?So?I researched on the web the term "SCRUM" combined with "lifecycle", "milestone", "phase" or "architect". The main links I obtained are pointing on?resources related to?LeSS, Spotify , DAD?or SAFe. For those, SCRUM approach is one of the building block, completed with complementary processes, arfacts to be produced and tools. In fact, most of the time, we have kind of hybrid approaches combining SCRUM projects with more traditional (or not) approaches related to the project management practices.

SAFe introduces a set of original concepts: "Train", "Epic" or "Value Stream", and requires definition of a vision plus the identification of the artifacts to produce. Several architect roles exist, being for solutions, systems or enterprise. SAFe?doesn't use phases, but makes a specific?usage of milestones which are progress milestones and Program Increments:

"Milestones are used to track progress toward a specific goal or event. There are three types of SAFe milestones: Program Increment (PI), fixed-date, and learning milestones".

"Milestones mark specific progress points on the development timeline, and they can be invaluable in measuring and monitoring the evolution and risk of a program. In the past, many progress milestones were based on phase-gate activities. In SAFe, however, progress milestones are better indicated by the fixed cadence of Iterations and Program Increments (PIs)".

In LeSS, different scrums run in parallel. No definition of phases or milestones.

With the Spotify approach , the focus is put on the autonomy.motivation and trust. Multiple and frequent decouples releases of functionalities are to be produced by tribes, organised in squads, and relying on guilds (a matrix organisation)/. The product owner should play the role of architect, without being in an ivory tower. Let's note that some Spotify can be combined with SAFe, as demonstrated by AXA .

DAD proposes some principles which complement those of the Agile Manifesto. In particular, the Enterprise Awareness is considered as important.

Enterprise Awareness is considered as important

In DAD, phases are proposed, with different lifecycle models (cf. previous figure) to be used according to the kind of project. DAD is one of the less disruptive agile approach concerning traditionnal projects, as the proposed lifecycle proposed are closely related to Project Management best practices.

SCRUM is not adapted to all the projects, and sometimes other alternative approaches could be more suited: Unified Process, Spiral, etc. Also, approaches defined for System Engineering and MBSE will have to be considered. Experimentation times are coming, for which different approaches will be combined and tailored in order to suite the needs of each organization. It could be sometimes complicated. It is the reason why I think Computer Aided Process design, plus easy and systematic enactment, are quite important. I think it should not rely on documents (too difficult to maintain and to manage) but on models. SPEM looks accurate for modelling processes, but also for combining legacy processes from different disciplines, as illustrated by what is proposed by APG: combination of TOGAF with other processes, such as the UP (cf. figure below).

After building such an overview of agile processes at scale and of SPEM capabiliites, it is then possible to start modelling and making some proposals for modelling agile approaches at scale with SPEM.

3 -Some proposed models (section to be extended)

Some models have been produced, using both Enterprise Architect and EPF (next figure), in order to gain some experience and in order to identify potential pain points or difficulties.

Some phases and milestones were defined.?It was preferred to use phases, milestones and iterations at the upper level of the process definition. If willing to define milestones at a lower level, for iterations, it is something that can be envisaged. Unfortunately, scrum iterations should cover several phases in one iteration, including deployment, usage and feedback, starting from the minimum viable product. So do we have preliminary phases before reaching the minimum viable products? Also, how do end a project? With decommissioning?

It should be considered that a SCRUM project may have a duration corresponding to the life duration of an application. So it means it could be very long, but it is notaligned with the SCRUM philosophy (which requires sort projects with whole availibitity of the members of the agile team), or that complementary process are to be considered (for maintenance and evolution). So the proposed model is not necessarily the perfect one, it is just an illustration of what can be produced relying on SPEM, knowing that each organisation (and project during the tayloring exercice) will have to define their own and make it evolve, with?the continuous improvement effort associated to total quality.

Conclusion: lessons learnt and next steps

Lessons learnt

Studying the different agile processes at scale and producing some agile process models in SPEM allowed?learning some lessons:

1. The future processes in the enterprises will probably have to be hybrid processes, combining agile processes such as SCRUM, and other more traditional (or not) processes completing what SCRUM is not covering, in particular consideration of phases and important milestones when considering a SCRUM project as part of a global porfolio in the whole enterprise.
2. For this, it could be of use to assemble and tailor some legacy processes on the shelves provided by different community. Switching from document based process description to model based description can bring a lot a added value.
3. SPEM is a good candidate language for producing such model, and is closely related to ability to tailor, to continuously improve and to publish configured processes.
4. Some issues exist concerning the way to consider phases and milestones, and to combine some principles which are sometimes contradictory. Pragmatism and efficiency should drive the choices and tradeoff between them. Using SPEM related solutions could be a way to track results and to gain experience and knowledge about what works or not.

Q1- Role of architects and usage of models?

The first question which arise is how to introduce the roles of architects in such processes and how to introduce practices related to usage of modeling tools they can rely on, such as ArchiMate/UML/SysML?modeling environments for enterprise architects defining the Enterprise continuum, solution continuum and architecture continuum (as defined by TOGAF)?

The question of the role of Architects for Agile was also centric at the Enterprise Architecture Day. For many enterprises which initiate the journey for becoming agile, such as AXA, the approaches are mainly document centric, and the question of using modelling is just arising, with ArchiMate being a candidate. Modelling approaches relying on UML are considered as too complex. However, for complex Information Systems and considering the complexity of applications which rely on several technological layers and are implemented using several languages, an architectural referential should be highly valuable for global consistency. The overall architecture can't be reflected only in the code. So introduction of Model based approaches will probably be something that will be addressed in the future, after discovering risks induced by a weakly formalised and governed architecture.

It should also be pointed out that thedivergence that exists concerning usage of models or not in agile approache is not new, and that many past and present approaches are considering the role of architecture, architects and modeling, in particular in order to support extension of the Agile Manifesto in order to be less development centric (cf. The Disciplined agile manifesto and Scaling Agile series )

• Enteprise architect and agility
• Agile Architecture: strategies for scaling agile development

• Principles of agile architecture
• Agile architecture in the digital age , an Open Group white paper proposing an Agile Framework on top of Micro Services platforms
• Discussion concerning emergent design and intentional architecture, as in SAFe . It seems in many cases more related to sofware/code architecture, applying systematic refactoring (factorisation of code) for simplification of code with at the end an emerging structure which was not planned from the beginning (so not specified in a model that came before). In some other case, it is all about being able to restructure an application at the last time, knowing that it implies easility reconfigurable solutions. But as pointed out here , it can be very effective to evolve towards a solution and architecture rather than crystal ball gaze by creating a (probably wrong) intentional architecture.

Very few of these articles are presenting concrete cases, with usage of modeling languages, and the way modeling should be effectively practiced is, even with SPEM modeling, remains very often under-described. How to deal with that is the purpose of the next question.

Q2- Closer link between SPEM and Model Based Approaches?

The second question which arise is about the link with model based approaches (relying on ArchiMate, OWL, UML, SysML, etc.) and SPEM. Most of the time, processes created with SPEM are document centric, and some templates are referenced which are document template. How to proceed if adopting a model based approach??How to qualify the different model artifacts to be produced and validate them? Is it also possible to create model templates, which will guide the architect teams in order producing easily consistent sets of models?

Not so much is available on the Web, I found:

• The thesis "SPEM4MDE : un métamodèle et un environnement pour la modélisation et la mise en ?uvre assistée de processus IDM" (SPEM4MDE: a metamodel and an environment for modelling and the aided usage of MDE) by Samba Diaw .

• Tim Weilkiens (thanks for information sharing, Tim) who created the SYSMOD methodology initially tried to use EPF for publishing his book. For book publishing, it proved to be too complicated. For sure, the EPF was not designed for book editing. Unfortunately, it means that no proposal is available on how to link MBSE and EPF an efficient way :(
• Tamara Hambrick (thanks for information sharing, Tamara) indicated that there will be a presentation of the usage of SPEM for Agile and MBSE methodologies at NDIA SE this year and it's pitfalls as well as successes, and that it will be publicly available. To be followed...
• I had the chance to participate to the OpenDevFactory project (within the scope of "Usine Logicielle " some years ago, where ability to derive Project Planning and Enterprise workflow from SPEM models produced with EPF was studied in order to support PLM and SLM collaboration with an Open Development Factory. But the considered models for the adopted Model Driven approach were process models, not the engineering artefacts produced by the involved engineers.

• I also experimented u AndroMDA, a model driven platform allowing to deploy business logic code generated from Business concepts on top of Enterprise Application Servers, using a standardized component model (Enterprise Jave Beans, derived from the CORBA Component Model). It was very effective for fast prototyping at scale during Europeran Research Projects. From traditional approach with 80% of the time for producing demonstrator and 20% for asssessing it, it allowed to reverse the trend with 20% for developing the demonstrator, and 80% to assess it, plus the abilty to easily extend the prototypes per use cases. It was made possible in particular by applying the pattern of separation of concern, separating business services from technical services. Also, there were a clear separation of front end models, with contexutalized use cases and back end models (serialized data and business objects consummed by use cases for building views on these objects). A whole factory was available, relying on maven, for model analysis, code generation, compilation and distribution both on testing and operational platforms, everything being also combined with testing. So a whole design/devops platform relying on continuous integration technologies. I think it can be properly combined with SPEM, with clear identification of structured artefacts, driven by the efficiency of the development process, with a modular and easily reconfigurable execution platform.

It should also be considered that not only Software Engineers, but also System Engineers are working on proposing agile MBSE approaches, as illustrated by the presentation "Introduction to the Agile System Engineering Life Cycle MBSE pattern " by Bill Schindele and Rick Dove at INCOSE in July 2016. Will some processes more model based oriented will emerge from the MBSE community? (also let's have a look at the Agile SE Working Group activities at INCOSE ). An important principle pointed out by this communities the following:

Tools are useful, but can be a hindrance as well. The tool needs to support and encourage the right behaviour.?They can never be a substitute for good processes, people or culture

However it doesn't prevent to be Computer Aided. Should we come back to the paper area?

Geoff Shuebrook pointed out the PHD work of Bruce Powel Douglass, about Agile MBSE practices (aMBSE) and related to IBM Harmony. It illustrates a very recent usage of SPEM, which is used for describing aMBSE microcycles and nanocycle, by means of SPEM activities and iterations. It includes usage of executable UML for simulation. It illustrates potential usage of SPEM for Agile combined with MBSE. It also comes with a deep analysis of agile principles and on when and how it can fit with MBSE.

In all cases, I think that an important value of SPEM is to point out that what are not the used languages and tools, but the clear definition of model artefacts and the consequently, as an artefact is a something concrete that can be delivered, the ability to split a model physically. It is the purpose of some of my articles and papers related to complex systems of systems, with the need to distinguished composite models of a system, and model of a composite system. For agile at scale, with several teams working on a part of the information system, the composability of models is very important if willing to take advantage of Model Based approach and cross team collaboration. Let's note it can be achieved only with a distributed graph (so some limitation of ontology and linked data ). UML/MDA modeling platforms are more and more mature concerning the ability to master complexity and work/responsability distribution.

Q3 - How can Agile be used with Product Life cycle Management and Digitalisation approaches?

A fist approach could be to make development of PLM open standards more agile, as well as their assessment (cf. SIP) and the dynamic aggregation of partners processes and SI (cf. IMAGINE). The two projects allowed to provide a Proof of Concept and a Proof of Value for combining virtualisation and modelling technologies. The approach extends devops approaches, considering not only development and operations, but also design, all this being made iteratively and taking advantage of CAD/PLM approaches with libraries of components which is regularly updated and extended. It also consider the scale of multiple enterprises working on a same product (Dynamic Manufacturing Network), with a target the establishment of digital continuity over the partners and over the time at an acceptable price (no disruption but mastered continuous evolution).

Such approach can be very effective even for deployment on several applications supporting the different System Engineering processes and Business Processes,

An interesting presentation was also made by Paul Empringham (PI): "Can Agile Methodology Deliver Successful PLM Projects?"?In the Paul's presentation, it appears that adopting agile for some PLM artefacts (e.g. new workflow) seems to be of a particular interest. Some are still subject to Waterfall.?The proposed approach is also an hybrid one.

Many questions also arise concerning the usage of Digital Thread, being a simple SysML model as experimented by Whirlpool, or digital twins which are impacting the traditional V cycle as proposed by Boeing. If not directly sold as "Agile" approaches, it should not be forgotten that Computer Aided approaches relying on models always aimed at reducing the cycles of development of a product.

So it questions agile approaches focussing only on the produced code.

Q4 - Considering new life cycle models for circular economy?

Also, how to adapt the enterprise to the emerging challenges such as circular economy, with products having several life, and consequently more interactions and needs for sharing data about the products within the digital business ecosystem? The enterprise continuum will not be sufficient anymore. End to end connectivity, as the future target according to CIMDATA (next figure), will have to consider digital manufacturing networks. Engineering processes will have to be considered within the context of a complex system of systems, and with a set of circular processes which will hate to be made consistent.

So probably new life cycle models to consider as the context of Agile processes such as SCRUM or XP. Is it suited?

Q5- Is the Agile manifesto acceptable and sufficient?

There were many reactions concerning this articles. Some with strong reactions concerning incompatibility of usage of Engineering Process Modelling, or Modelling, with the Agile principles as stated in the manifesto. Some other reactions were about considering that what is the most important is a required change of mindset, with a better focus on users (I don't think that most of professional people had ever lost this focus but I agree with the principle consisting in considering all the concerns users, actors and stakeholders).

In the reverse, some already try to define hybrid processes and even are rejecting the Agile Manifesto and proposing new more accurate Manifesto.

• the Architectural Thinking association put the emphasis on the need to adopt a design thinking approach and to rely on modelling. They are requiring a more holistic Manifesto.

• "Agile without Magic " reacts because of the following: "Agile projects fail the way they are not supposed to. The management and stakeholders get increasingly frustrated when they are kept in the dark about projects because they are "Agile". And even when an Agile project is delivered, the result does not necessarily makes the enterprise happy." So they are proposing a new set of principles in order to prevent such results.

Within the agile community, I identified interesting resources in french "Agile : Controverses et reflexions " (Agile: controversies and thinking) and in english "The Decline and Fall of Agile ", pointing out some controversy related to how and when Agile is applied (or misapplied), and potential incomplete coverage of needs by SCRUM or XP. An interesting response is given in "Agile: de A à Z ". On other controversy is related to the fact that some agile practitioners consider that the ultimate - and the only model to consider - is the code itself. The code is the model . Some people disagree. You can papers here .

I think being aware of such controversies and reactions is important in order to efficiently deal with strengths and weaknesses of agile, considering both opportunities and risks which are associated.

Let's consider both strengths and weaknesses of agile, identifying both opportunities and risks which are associated

Q6- What is the impact of uncertainty and variability concerning success of Agile (and is it sufficient for addressing it)?

The growing pace of change for Information and Communication Technologies and Internet Landscape, as well as approaches related to VUCA (Volatility, Uncertainty, Complexity and Ambiguity) could also explain the motivation of enteprises for Agility. More reactivity is needed in order to face quick obsolescence of adopted solutions, and for being able to have some Return on Investments on what is developped. So the duration of cycles have to be reduced but ROI is to be ensured. But does it reduce complexity? Agile approach always recommand to make the things simple, but complexity is not related to the system a project want to deliver, but from the multiplicity of systems which are produced and constitute the Information System. Not sure agile allows reducing such a complexity, as illustrated by the following diagram illustrating how efficiency can decrease over the time, when the debt is growing.

So can Agile address such issue, without relying on architects and being model driven?

Q7- What are the condition failure for Agile at scale, and can always the enterprise apply it?

Adopting Agile approaches comes with some constraint, in terms of colocation and resources availibility and engagement. For big companies, with geographically distributed workers which are involved in several projects at the same time, is it really possible to work only on SCRUM mode? And also, how to manage allocation of budget and control it appropriately? After an important investment on virtualisation of enterprises, it is not so sure that Agile is always possible, without some adaptations. What should it be?

Q8 - How should be addressed Continuous Integration and how does it impact agile software process engineering?

In huge enterprises with many SCRUM team, the question of Continuous Integration arises, with as a constraint the need for configuration management, testing and integration platforms for the support of the teams. But how and when is the choice made? Is it included within the scope of each SCRUM team, or should it be done at an upper level, creating some pre-requisites for the SCRUM projects being able to run? So may be we have to come back to more traditional approaches related to system engineering in order to address it, with the need to consider different life-cycle phases.

Addressing these questions will be the subject of future articles of the series. Let's follow, and don't hesitate to share experience, ideas, opinions and ideas.

Some references