The Chargeback Model is Broken and We Found a Way to Fix It!
With the growth in Software-as-a-Service (Saas) and Platform-as-a-Service (PaaS), organizations have a much easier time deploying software and infrastructure. The ease in which complete environments can be "spun up" has created new efficiencies but has also introduced a massive amount of change. The pace of this change has consequences and one of them is the impact it has on the IT Chargeback Model.
Before we begin, this blog provides an excellent description of how a chargeback model works. To summarize:
The IT chargeback is a method of charging internal consumers (e.g., departments, functional units) for the IT services they used. Instead of bundling all IT costs under the IT department.
For a chargeback program to work there needs to be a fair degree of transparency involved.
Each participant in the program needs to be able to see, track and understand the usage or consumption of the services in question. Life is good when things are simple but as applications and infrastructures evolve, the picture grows more opaque. Constant change requires constant effort and accelerating this change requires an acceleration of effort. Unfortunately this hasn't occurred in the age of the "spin up" infrastructure and the overtaxed IT department.
The Chargeback Conundrum
Over the last decade the success rate of chargeback programs measured by their ability to claw back funds has gone from bad to worse. Chargeback programs require procedural controls for them to function. These controls include steps such as formal requests, the implementation of naming conventions and the sharing of change control documents. The controls are there to help the IT department collect and maintain a database of who owns what so that they know who and how to charge.
In the past, a lengthy procurement process meant there was no excuse for not having a properly documented request. Today, in the "spin up" world, users are blessed with near-instant feedback and near instant results. The argument then becomes: why spend the money on self-service capabilities if there is a much larger bureaucratic delay required to leverage these capabilities?
The Chargeback Conundrum is the challenge faced by IT when trying to keep chargeback programs viable and yet still foster an environment of rapid innovation.
In addition, Agile development methods have also added to the Chargeback Conundrum. The legacy Waterfall approach placed a great deal of emphasis on documentation whereas the current Agile methods emphasize incremental, near-term value. It promotes the attitude of "Get it to work now, we will worry about documentation later." Under constant deadline pressure, guess what never happens?
The movement from Waterfall to Agile, coupled with the push for self-service software, has meant the pace of change occurs rapidly and the implementation of controls has suffered. This has consequences which IT departments should not ignore. It means a loss of control and a loss of transparency which not only makes a chargeback program collapse it has other longer term consequences. Without some form of transparency IT environments will only continue to become more complex, less transparent and more difficult to change.
The rigors of proper documentation were put in place to protect the business (from itself) but recent innovations and rapid deployment capabilities have fostered a "wild west" environment. A lack of documentation or procedural controls equates to a lack of law or order. We lose sight of who is doing what and for why.
Technical Challenges
IT departments have to assume that procedural controls will continue to grow increasingly difficult to maintain. Therefore they will need to turn to alternative methods to capture the same type of information. Fortunately there are options. As users access and consume applications or services they leave evidence behind in the form of application logs, help desk tickets and audit logs. These "digital footprints" can be a rich source of information but in order to leverage this data several technical challenges have to be overcome:
1) Data Centrality. Data that can potentially solve this problem is found in multiple, disconnected platforms. IT departments will need to collect and store this data from these platforms in a central data warehouse that is flexible enough to handle complex and multi-dimensional data. Traditional table-based approaches require a huge amount of engineering upfront which is often a non-starter. A graph database with its robust support for complex, multi-dimensional data, is a much better choice.
2) Data Mapping. Assuming naming-conventions will never work and the "wild-west" is the new normal, IT departments will need an approach to map different terms and labels together and do this dynamically. Doing this by hand is not an option!
As an example: an application can be called "Application B" in one system but called "Application 123" in another. The IT organization will need to some mechanism to auto-map these terms together.
3) Reporting. The end deliverable will be a set of reports that provides the justification for the chargeback. IT will need to show the connections that exist in the data in order to prove their case. Without this type of evidence, the business may not agree to pay the bill.
Loading data into a common platform, applying a common structure to it and allowing users to access this data via a simple interface is how a chargeback model is brought back to life!
Process Tempo is a platform perfectly designed to meet these requirements. We will now discuss how this platform can be implemented to solve this problem.
The Data Warehouse Design
The first step is to design the conceptual model of how the data will be stored in Process Tempo. This is achieved using the Process Tempo Visual Modeler. This model will drive the design of the Process Tempo data warehouse:
At the center of the model is the ApplicationID which is the core bit of information that glues all of this data together. We need some method of connecting activity and users to this application for the purpose of creating a Chargeback Record (the lavender node to the bottom-right). The Application ID will be the hardest part of the model to define which is why we discuss it in detail later in this blog.
The Aggregate Usage Metric node (the grey node) is how we will store the aggregate usage by user, application and time period. We will rely on this data to produce chargeback reports.
The white nodes represent individual users (the UserID) and the Department they work for. The traditional system of record used to produce this data is often LDAP but this data could also be found in other systems which maintain this relationship.
The green nodes represent the data sources we will need to put this model together. To solve our chargeback problem it is likely we need three sources: LDAP data, log activity data and help desk ticket data. When combined and merged into a usable shape, this data will tell us what we need to create the appropriate chargeback report.
Note: Some organizations may have a treasure trove of support or help desk tickets to work with which neatly tie a user to an application. Some organization may not. Some sleuthing is required to find the best data to use to solve this problem. Help desk data is often a good place to start.
领英推荐
Creating Aliases to Map Terms
With the model in place, we can now load the data into Process Tempo. For this we use our built-in Import Modeler which makes this effort very easy. We will save the details on how this is done for another blog post.
A Screenshot of the Process Tempo Import Modeler:
Once the log records are loaded the next step is to create the aliases required to match application identifiers together. As we discussed already, relaying on a single, unique identifier for each application or service falls apart in the new "spin up" world. The consistent use of the same identifier or label is a rare luxury.
A real-world example:
A user creates a new record for the application "Process Tempo" in their procurement system. The system then assigns a unique identifier of "12345" to this record. This data is thought to be the cleanest list of application data since all application purchases must first be approved in the procurement system
However, when deployed in the client's operating environment the administrator spins up several instances of Process Tempo and the following new labels (or identifiers) are created: process-tempo-dev-1, process-tempo-dev-2, process-tempo-prod-1.
Later, another user with a different role decides to store log information from this application and so creates a new index in their log system called: pt-platform.
Here is what we end up with:
Multiply this simple example for each combination of applications, department and roles across an entire organization and you can see how big of a problem this can become!
With strong procedural controls in place, the user that manages the environment and the user that manages the log files would know to reference the same label and the same identifier in their respective systems. Without these controls they leverage their own naming conventions and suddenly the picture gets very murky. What is needed is a method to map these different terms together using a common Alias.
The Process Tempo Alias Engine
The Process Tempo Alias Engine is a feature purpose-built to create relationships between two or more terms. When a relationship does not exist the Alias engine attempts to infer a relationship based on what the engine knows about these terms.
The engine incorporates two approaches: text similarity and relationship similarity.
Text similarity uses string comparisons to check for typographical errors or slight differences in spelling. In our example, text similarity (also known as 'fuzzy matching') would find a strong match between the labels process-tempo-dev1 and process-tempo-dev2 as they are only different by a single character. However, it would struggle to find a comparison between these labels and the label: pt-platform. Matching on text only works when there is enough information in the labels to provide a good enough comparison.
The relationship similarity approaches is different in that it looks at the space around the two objects to see what they have in common. If user "A" created process-tempo-dev1 and user "A" also created the record process tempo record in the procurement system, there exists a commonality between these records. The more two records have in common, the stronger the bond between them. Using the strength of this bond the engine can infer for us a relationship.
A Node Similarity Example
Since the procurement database is deemed as the "golden record" for application data we will use this as our target dataset. Our source dataset will attempt to match to this one. The reverse also works so source and target are interchangeable.
In the below diagram we are attempting to create an Application Alias. To make this work, the Alias Engine will look at the graph of existing applications, owners and departments. If applications share similar names, owners and departments then the engine will create a direct relationship between the applications using a "Also Known As" relationship, aka an Alias. (Pun intended).
Tying it All Together
With the data loaded into Process Tempo and the relationships in place, we can now start to analyze the results. Is pt-platform in the log system the same as Process Tempo in the procurement system? We can now test to see how well the Alias engine performed. We may need to tweak the inputs and rerun the tests. The good news is that the Alias Engine can be run multiple times. The same is true for the text-matching capability. Combining the output of the two should give users a high-degree of confidence in the results.
After our testing is complete we can now use the Process Tempo Report Builder to pull all of this data together for our final report back to the business. The report is to provide tangible evidence back to the business that indicates the consumption of a given service by a given department. Evidence for a chargeback!
If questioned, the IT department can walk the user through the connections in the data that produced this result. Process Tempo makes this very easy to do using the integrated search and explore features that come with the product.
Summary
The chargeback model is broken and can only be solved with data. The challenge is that this data tends to be very messy, disconnected and distributed. Process Tempo has solved this problem and created a platform to help IT departments bring this data together to create the appropriate chargeback reports.
For more information, visit us at www.processtempo.com and use our Contact Us form or our Calendar form to get in touch with us.
Chief Marketing Officer | Product MVP Expert | Cyber Security Enthusiast | @ GITEX DUBAI in October
2 年Phil, thanks for sharing!