I ran "rm -rf ~" in production
Earlier today I was telling my dad how I am not paying for cloud database services that handle all the infrastructure and backup for you because I can just do those things myself for free. Then I proceeded to set up a script to do backups for a NoSQL database I have been maintaining for a while now. rclone is a great tool for this, so I quickly wrote up the following script with my lagging SSH connection (comments added for folks without UNIX experience).
Now, mind me, I forgot about an important fact — "~/backups", while intended to stand for "/home/username/backups", it just stands for a relative "~/backups", which means a folder named "~" will be created in the current directory. Indeed, as soon as I ran this script to test, the following was the output for ls:
Even my dad sitting across the table noticed the look of surprise in my face. Now, luckily, I just backed up my database, so even if it was deleted, I would have the data. But MongoDB doesn't store the data in the home directory anyway. Shortly, I realized another problem: all my SSH keys (located in "~/.ssh") are gone, and password-login is disabled for this server. I quickly set up another set of keys using the existing connection. Fortunately, it was not too much of a disaster otherwise:
