I think I thought of that in like, 2003, when developing my first web site. My lazy ass was just thinking of using PHP's ability to execute terminal commands to do all the heavy lifting on the backend for everything for me, because I sucked as a programmer. That would have been a terrible, terrible idea in hindsight.
This was way before I learned about form input sanitization too. I was working off of "For Dummies" books.
Send this to my last DevOps team. Jesus. They got a Rube Goldberg machine in place of sane infra. Shit’s so complicated, I know only one guy with his finger on the pulse, and that’s because he built a ton of it over 18 years!
This is great until your job outgrows a single computer or you want to have redundancy. Also, chains of bash tools don’t have the best error management when something chokes in one of the middle steps in the pipe. You can still leverage simple bash tools for a lot of the under the hood stuff, but you start needing something more to act as the glue petty quickly when you scale. KISS should still apply.
I think that’s the point. Don’t jump to the complex right away. Keep it simple and compose the capabilities you have readily available until you need to become more complex. When the task requires it, yeah, do the complex thing, but keep the simplicity mandate in mind and only add the new complexity that you need. You can get pretty far with the simple, and what about all of the situations where that future pivot or growth never happens?
The philosophy strikes a cord with me - I’m often contending with teams that are building for the future complexities that they think might come up, and we realize later that we did get complexity in the problem later, but not the kind we had planned for, so all of that infrastructure and planning was wasted on an imaginary problem that no only didn’t help us but often actually make our task harder. The trick is to keep the solution set composable and flexible so that if complexity shows up later, we can reconfigure and build the new capabilities that we need rather than having to maneuver a large complicated system that we built on a white board before we really knew what the problem looked like.
This works until you scale the team beyond 1 person and someone else needs to decipher the 30 line awk | sed | xargs monstrosity you created. Give me a real programming language any day.
I’ve hacked plenty of bash aliases, functions, and scripts using coreutils myself; but sometimes you need something a bit more robust when it comes to error handling, retrying, maintainability, and an actually distributed solution instead.
xargs on its own might be more resilient than a distributed crawler, as one would expect, but if I’m tasked with building a distributed data processing pipeline I want more guarantees from the system as a whole, not only from its individual building blocks.
The time and effort put into embedding these guarantees in hacked shell scripts running on a dozen machines might be better invested into building a more solid foundation instead.
Love ol’ Jim. My stepdad and I caught him for his 2014 tour and it’s been one of the highlights of both our live entertainment experiences lol that and seeing Rush in 2012.
Edit: “live” entertainment lol… Gotta love mobile.
widgetsandshit.com
Newest