Boilerplate Stories for Analytic Solutions
Photo by FORTYTWO on Unsplash

Boilerplate Stories for Analytic Solutions


Core Stories

  • Data Source Exploration
  • Data Model
  • Mapping
  • Pipeline Development


?

Story templates

Data Source Exploration

Title:

As a Data Engineer, I want to be able to connect to and extract valued data, so that I can build data pipelines.


Description:

Research data source type, connection details, and schema. Use the before mentioned research items to establish a connection and extract a data sample. After the successful extraction, perform data quality checks on the sample to determine if the data holds needed value.

?
Quality checks to perform:?

Attributes meet an acceptable level of null values.?
Attributes meet an acceptable level of corrupted values.

(The threshold for null and corrupted values is per use case)


Acceptance criteria:

- Document with research items: Source type, connection details, and schema.
- Document uploaded to a sharable location.
- Data source passing quality checks.


Role(s) for story:

Preferred

- Data Analyst?

Secondary

- Data Engineern        


Data Model



Title:

As a Data Engineer, I want to have a target data model, so I can map source attributes to target attributes.


Description:

Explorer data model options (e. g., snowflake schema, star schema, single table, etc) that best fits the solution, and create an ERD representation of it.?

(The data source exploration will provide essential information to determine which model to choose, or if any is needed. Also, this story should be encompass the target(s) for all pipelines of the product)

?
Acceptance criteria:

- Visio file containing ERD of data model.
- File loaded to a sharable location.?

?
Dependencies:

- Data Source Exploration has been completed and acceptance criteria met.

?	
Role(s) for story:

Preferred

- Data Modeler

Secondary

- Data Engineerl
        


Mapping



Title:

As a Data Engineer, I want to have a mapping document, so I can design and implement a data pipeline.


Description:

Create a mapping document where source attributes are mapped to target attributes. Additionally, the document needs to contain any transformations that the attribute(s) will undergo while moving from source to target.

?
Acceptance criteria:

- Document containing mapping of source to target, and transformation per attribute
- Document uploaded to sharable location


Dependencies:

- Data source exploration has been completed and acceptance criteria met.
- Data model ERD has been completed and acceptance criteria met.


Role(s) for story:

Data Modeler, Data Engineer, Data Analyst, Business Analystg
        


Pipeline Development


Title:

As a Business Intelligence Engineer, I want to have a consumption data set, so I can design and implement a presentation layer.


Description:

Extract data from specified source(s), and transform based on business requirements. Finally load to designated target(s).


(The tools used will be specific to the architecture, and should be included in the description when known. Additionally, the environment should also be specified e.g., DEV, QA, PROD)?

?
Technical info

Source(s):

<Insert source 1 name and location, or description >
<Insert source 1 name and location, or description >
??…

Transformation(s):?

<Insert transformation 1>
<Insert transformation 2>
…

Target(s):

<Insert target 1 name and location, or description>
<Insert target 2 name and location, or description >
…

?
Acceptance criteria:

- Pipeline uses specified source(s)
- Data is loaded to specified target(s) in corresponding environment
- Data passes data quality units tests such as:
  - Attributes meet an acceptable level of null values.?
  - Attributes meet an acceptable level of corrupted values.
  - Target row counts meet expectations.
  
  ?
  
Dependencies:

- Mapping has been completed and acceptance criteria met.
  
  
Role(s) for story:
  
Preferred
  
- Data Engineer
  
Secondary
  
- Data Analyst
- Business Intelligence Engineert
        


?

?



?

David Vo

The CFO for Your Personal Finances | Helping Business Owners & Professionals Make Smarter Money Decisions

6 个月

Awesome Allan, thanks for sharing!

回复

要查看或添加评论,请登录

Allan Hawkins的更多文章

社区洞察

其他会员也浏览了