免费学习该课程!
今天就开通帐号,24,600 门业界名师课程任您挑!
Define Databricks workflow components
- [Instructor] Before we go into modifying the Databricks YAML file, let's figure out what we want to deploy first. Let's say we want to make our model serving with a feature lookup use case production ready. Let's assume we have new data coming in every week, and the model must be retrained every week. That means that every week we have to update the feature on the online table, retrain the model, update the endpoint. In this video, we'll move away from the notebooks we had and create three Python files for these three tasks, which must be executed one after another. Before we walk through them step by step, let's focus on the pre-processing first. We need to mimic the data refresh part. Let's create a table called extra train set. It has a timestamp column, update timestamp UTC. Every week, we want to insert the data into the train set table from the extra train set table, but only if the data came in after the last update timestamp from the train set. If such data exists and is…