Ayla Khan – MLOps, DevOps, occasional ooops

TIL: Python recursion limits

Published September 27, 2022 in today I learned

Today I learned that you can get (sys.getrecursionlimit()) and set (sys.setrecursionlimit(limit)) the Python interpreter’s maximum stack depth. I can see that on my system, the maximum stack depth or recursion limit is 3000. Exceeding the limit results in a RecursionError. This came up in a Prefect flow using a LocalDaskExecutor that attempted to serialize an […]

Tags: python

TIL: Python logger extra keyword

Published April 21, 2022 in today I learned

Today I learned that extra arguments can be passed to any Python stdlib logger method that logs a message with the extra keyword argument as a dict. The extra arguments may be used in a formatter or with other logging tools like structlog. Output is: 2022-04-21 00:47:31,691 Critical error message! | Linux #29~20.04.1-Ubuntu SMP Fri […]

Tags: python

Python Dependency and Environment Management Deep-Dive

Published April 11, 2022 in devops

Struggling to make sense of the Python package and virtual environment landscape? Not sure how what tools have features that make building CI/CD for your Python project easier? Frustrated by slow dependency resolution? To improve Recursion‘s Python developer experience, my colleague Dan Maljovec and I did a deep dive on those very topics and wrote […]

Tags: python

TIL: How to pass extra gunicorn arguments to MLflow server

Published March 22, 2022 in today I learned

The MLflow server exposes the –workers flag to change the number of gunicorn workers, but if you want to pass other arguments to gunicorn then you need the –gunicorn-opts flag: It looks like –waitress-opts serves the same purpose if running MLflow on Windows. If passing the –gunicorn-opts flag to a container running the server on […]

Tags: kubernetes , mlops

Use CloudFront to serve S3 static site over HTTPS

Published March 31, 2021 in devops

Say you created a simple, static website but you don’t want to manage or pay for a lot of hosting solution. One option is to use S3 to host your static website; a cheap and easy to manage solution if you stay within lower usage limits. Unfortunately S3 only serves sites over HTTP, and browsers […]

Tags: aws

Book Review: The Pragmatic Programmer: your journey to mastery, 20th Anniversary Edition, 2nd edition

Published January 31, 2021 in review

Disclosure: I was not compensated for writing this post nor was my review solicited. All opinions are 100% my own. Quick Take: mostly worth your time and can be a good review for professional programmers. This book may offer the most value for people entering the field. The Pragmatic Programmer book was recommended by a […]

Tags: review

Course Review: Effective Pandas – Standard

Published November 2, 2020 in data , review

Disclosure: I was given access to Effective Pandas – Standard to review. I was not compensated for writing this post and the links to the course are not affiliate links. All opinions are 100% my own. Quick take: Effective Pandas is a high quality course with well thought out content. I felt like it was […]

Tags: pandas , python , review

Python text analysis tools: FuzzyWuzzy’s basic string matching

Published March 29, 2020 in data

The Python FuzzyWuzzy module uses Levenshtein edit distance to implement fuzzy string matching. FuzzyWuzzy’s matching tools return results on a scale from 0 to 100. The simplest matching tool FuzzyWuzzy offers is the ratio(..) function: The basic ratio function works well for simple string matching. However if you’re trying to fuzzy match a single word […]

Tags: data wrangling , nlp , python

Ijson coroutines and generators

Published February 27, 2020 in data

Reader comments on an old post about the ijson parser prompted me to check out the project’s more recent releases. The latest pre-release (v3.0rc1) added a coroutine interface, which allow users to supply their own file readers and have more control over when the parser is called. It looked like a fun feature to explore, […]

Tags: data wrangling , python

Python text analysis tools: Levenshtein Distance

Published January 31, 2020 in data

Figuring out how similar two strings are and then making that similarity a quantitative measurement is a basic problem in text analysis, text mining and natural language processing. There are a number of efficient methods to solve this problem. This survey looks at Python implementations of a simple but widely used method: Levenshtein distance as […]

Tags: data wrangling , nlp , python

1 2 3 4 Next »