If this helped you, please share!

SageMaker lifecycle configuration time constraints

Published May 17, 2019 in data - 0 Comments

Creating AWS SageMaker Lifecycle configuration scripts to customize notebook instances beats installing packages and making other environment changes in notebook instances. One advantage is that the customization code doesn’t need to be copied from notebook to notebook. Another is that the lifecycle configurations are managed outside of and separately from notebook instances, and can be applied to multiple notebook instances.

One thing to be aware of however, is that lifecycle configuration scripts have a 5 minute execution time limit. I ran into this attempting to clone a conda environment with conda create. The goal was to create a fresh conda environment to install custom packages instead of the standard environments available in SageMaker. Trying to copy any of the conda environments during notebook creation looks like it will take longer than the 5 minute limit.

lifecycle rule timeout

Testing with conda env export and conda env create with an environment file also takes too long. I’ll have to see if another strategy works better.

Tags: aws, python

No comments yet

Leave a Reply: