Boilerplate Stories for Analytic Solutions
Core Stories
?
Story templates
Data Source Exploration
Title:
As a Data Engineer, I want to be able to connect to and extract valued data, so that I can build data pipelines.
Description:
Research data source type, connection details, and schema. Use the before mentioned research items to establish a connection and extract a data sample. After the successful extraction, perform data quality checks on the sample to determine if the data holds needed value.
?
Quality checks to perform:?
Attributes meet an acceptable level of null values.?
Attributes meet an acceptable level of corrupted values.
(The threshold for null and corrupted values is per use case)
Acceptance criteria:
- Document with research items: Source type, connection details, and schema.
- Document uploaded to a sharable location.
- Data source passing quality checks.
Role(s) for story:
Preferred
- Data Analyst?
Secondary
- Data Engineern
Data Model
Title:
As a Data Engineer, I want to have a target data model, so I can map source attributes to target attributes.
Description:
Explorer data model options (e. g., snowflake schema, star schema, single table, etc) that best fits the solution, and create an ERD representation of it.?
(The data source exploration will provide essential information to determine which model to choose, or if any is needed. Also, this story should be encompass the target(s) for all pipelines of the product)
?
Acceptance criteria:
- Visio file containing ERD of data model.
- File loaded to a sharable location.?
?
Dependencies:
- Data Source Exploration has been completed and acceptance criteria met.
?
Role(s) for story:
Preferred
- Data Modeler
Secondary
- Data Engineerl
领英推荐
Mapping
Title:
As a Data Engineer, I want to have a mapping document, so I can design and implement a data pipeline.
Description:
Create a mapping document where source attributes are mapped to target attributes. Additionally, the document needs to contain any transformations that the attribute(s) will undergo while moving from source to target.
?
Acceptance criteria:
- Document containing mapping of source to target, and transformation per attribute
- Document uploaded to sharable location
Dependencies:
- Data source exploration has been completed and acceptance criteria met.
- Data model ERD has been completed and acceptance criteria met.
Role(s) for story:
Data Modeler, Data Engineer, Data Analyst, Business Analystg
Pipeline Development
Title:
As a Business Intelligence Engineer, I want to have a consumption data set, so I can design and implement a presentation layer.
Description:
Extract data from specified source(s), and transform based on business requirements. Finally load to designated target(s).
(The tools used will be specific to the architecture, and should be included in the description when known. Additionally, the environment should also be specified e.g., DEV, QA, PROD)?
?
Technical info
Source(s):
<Insert source 1 name and location, or description >
<Insert source 1 name and location, or description >
??…
Transformation(s):?
<Insert transformation 1>
<Insert transformation 2>
…
Target(s):
<Insert target 1 name and location, or description>
<Insert target 2 name and location, or description >
…
?
Acceptance criteria:
- Pipeline uses specified source(s)
- Data is loaded to specified target(s) in corresponding environment
- Data passes data quality units tests such as:
- Attributes meet an acceptable level of null values.?
- Attributes meet an acceptable level of corrupted values.
- Target row counts meet expectations.
?
Dependencies:
- Mapping has been completed and acceptance criteria met.
Role(s) for story:
Preferred
- Data Engineer
Secondary
- Data Analyst
- Business Intelligence Engineert
?
?
?
The CFO for Your Personal Finances | Helping Business Owners & Professionals Make Smarter Money Decisions
6 个月Awesome Allan, thanks for sharing!