As a self-employed software engineer, I use AI daily to help with my work. But I use it in a very targeted way and with a heavy skepticism I call “Don’t Trust and Heavily Verify”. This is the opposite of “vibe coding” where the developer is simply typing sentences into the AI and getting it to do everything behind the scenes with little to no verification of the resulting code.
So, while I do appreciate what modern AI is capable of doing, I also realize that it’s a magic trick of sorts; a very impressive magic trick to be sure, but one must understand the limits of AI and realize that it’s not actually thinking. In fact, it doesn’t actually know ahead of time what the next word in its response will be displayed. That is determined on the fly by a huge probability tree.
There are two new videos on this subject that are well worth watching but, for some background, please read about the recent failures of AI at Amazon.
Here’s the first video and here’s the second.
The company I’m working for has decided that trying out AI is worth the risk. It works pretty well as a glorified autocomplete where the developer is reading through the code as AI suggests it. Lately, I’ve been using it to generate prototypes as a tool for discussion. The code has been pretty clean, but then, I asked it to generate clean code.
Half the time ai generated code doesn’t even compile nevermind work. I’ve asked it to refactor code so many times and it’s utter garbage. It does give you a decent outline of how to split stuff up but I do that all manually.
For what it’s worth, I’ve tried X’s Grok and Google’s Gemini. As you undoubtedly know, there are different levels of “thinking” (I’m using that word loosely) from something akin to “Fast” up to “Pro”. I have found that there’s a big difference in the quality of what it does the higher the tier one uses. It’s often not perfect the first time, but after a few iterations, it tends to get it correct.
Apparently (Amazon) learning at scale has costs of similar scale.
I’m updating and revising some 10 year old php code. Googles AI is just barely good enough to write 8 or 10 lines of coherent code and the links to the php manual are very good.
AI is merely a somewhat better search engine, maybe 20% better. The gigantic cost of that 20% improvement is insane.
I am a retired software engineer. (Stroke in 2014 has made me less effective at debugging.) I have found ChatGPT and Copilot only marginally useful for writing C and gCode, although they often get ne close. SuoerGrok is pretty useful. Even better, it is generally useful. I had a problem with reassembling a Walther PPK/S recently and it made a very good suggestion that solved the problem.
Grok has replaced Google search for me. Great for proofreading emails and memos. I’ve used it for short scripts (Python, powershell, bash) to good effect. No substitute for testing. No substitute for thought, but a great assistant.
Think of it as a gifted but eccentric intern. Sometimes over-eager. Sometimes impaired. Sometimes retarded.