What I've learned building MLOps systems for four years

submited by

Style Pass

2024-09-04 10:00:06

As title says, I’ve worked on building MLOps systems for nearly four years. Things run amazingly fast, and as a person that is also four years experienced guy, I kinda feel like I’ve tried to not drown by the new techs in deep learning(LLMs), trying to adapt software engineering, try to catch good positions in good companies remotely(that sucks), and more.

Check my LinkedIn profile, this blog post will make much more sense, and it’s fun to stalk as always. I had some experience on two stealth mode startups on LLMs, check this post.

This post is kinda like an honest overview of my experiences, and what I’ve thought on engineering, machine learning (operations), through years. I think those questions were in your mind as well, and I’m not answering any of that, just sharing my opinions.

In 2021, I started working on energy consumption models, and it was my first real dive into operational applications. The problem was straightforward at first: we had to predict the daily electricity consumption for eight cities, for every 24 hours, but 24 hours ahead. Day-ahead electricity forecasting. Since I was working on the problem, the number of users kept increasing, the average consumer started using more energy, and new factories were being built. The demand for energy also shifted with changes in the economy. All means model drift and data drift.