Music Recommender Part 7: Clean Up

Overview

### Clean up : Delete all Resources Created in the past 8 notebooks (nb 00-06) In the past notebooks we have created many Amazon Resources; represented by their ARNs : Amazon Resource Names. In order not to incur any cost in keeping those resources running, such as endpoints etc. We will use this notebook as a reminder to clean up and delete all the resources you have created in this music recommendation example.

First we will read in all parameters saved in the ‘music-rec’ namespace as we went from one notebook to the next, second we will use a little utility under the ./code/demo_helpers.py script file to actually delete all resources passed

Contents

• Overview

• Part 1: Data Prep using Data Wrangler

• Part 2a: Feature Store Creation - Tracks

• Part 2b: Feature Store Creation - User Preferences

• Part 2c: Feature Store Creation - Ratings

• Part 3: Train Model with Debugger Hooks. Set Artifacts and Register Model.

• Part 4: Deploy Model & Inference using Online Feature Store

• Part 5: Model Monitor

• Part 6: SageMaker Pipelines

• Part 7: Resource Cleanup

import json
import boto3
import pathlib
import sagemaker
import numpy as np
import pandas as pd
import awswrangler as wr

from sagemaker.estimator import Estimator
from sagemaker.workflow.pipeline import Pipeline
from sagemaker.workflow.steps import CreateModelStep
from sagemaker.sklearn.processing import SKLearnProcessor
from sagemaker.workflow.step_collections import RegisterModel
from sagemaker.workflow.steps import ProcessingStep, TrainingStep
from sagemaker.processing import ProcessingInput, ProcessingOutput
from sagemaker.workflow.parameters import ParameterInteger, ParameterFloat, ParameterString
from sagemaker.feature_store.feature_group import FeatureGroup

import sys
import pprint
sys.path.insert(1, './code')
from parameter_store import ParameterStore

ps = ParameterStore(verbose=False)

parameters = ps.read('music-rec')

bucket = parameters['bucket']
prefix = parameters['prefix']
ratings_data_source = parameters['ratings_data_source']
tracks_data_source = parameters['tracks_data_source']
val_data_uri = f"s3://{bucket}/{prefix}/data/val/val_data.csv"

pipeline_endpoint_name = parameters['pipeline_endpoint_name']
pipeline_name = parameters['pipeline_name']

fg_name_tracks = parameters['fg_name_tracks']
fg_name_ratings = parameters['fg_name_ratings']
fg_name_user_preferences = parameters['fg_name_user_preferences']

dw_ecrlist = parameters['dw_ecrlist']

pipeline_name = parameters['pipeline_name']
dataprep_pipeline_name = parameters['dataprep_pipeline_name']
train_deploy_pipeline_name = parameters['train_deploy_pipeline_name']

endpoint_name = parameters['endpoint_name']
pipeline_endpoint_name = parameters['pipeline_endpoint_name']

mpg_name = parameters['mpg_name']

region = boto3.Session().region_name
boto3.setup_default_session(region_name=region)
boto_session = boto3.Session(region_name=region)

s3_client = boto3.client('s3', region_name=region)

sagemaker_boto_client = boto_session.client('sagemaker')
sagemaker_session = sagemaker.session.Session(
    boto_session=boto_session,
    sagemaker_client=sagemaker_boto_client)
sagemaker_role = sagemaker.get_execution_role()

account_id = boto3.client('sts').get_caller_identity()["Account"]

# when demo_helpers.delete_project_resources() is ran it will delete all the resources created by this demo
sys.path.insert(1, './code')
import demo_helpers  # our custom set of functions


def remove_all_resources():
    demo_helpers.delete_project_resources(
        sagemaker_boto_client=sagemaker_boto_client,
        sagemaker_session=sagemaker_session,
        endpoint_names=[pipeline_endpoint_name, endpoint_name],
        pipeline_names=[pipeline_name, dataprep_pipeline_name, train_deploy_pipeline_name],
        mpg_name=mpg_name,
        feature_groups=[fg_name_ratings, fg_name_tracks, fg_name_user_preferences],
        prefix=prefix,
        delete_s3_objects=True,
        bucket_name=bucket
    )

# Uncomment the next line and run to delete all resources
# remove_all_resources()