«AI» is being propagated by big tech as a workforce multiplier, but my personal experience with public chatbots such as Gemini, ChatGPT, Claude models makes me skeptical of its value to IT professionals.

Problems I have discovered, and why those won’t be solved.

Ambiguous, verbose answers. LLMs are statistical models. They tend to surround the answer with «error correcting code», so that the meaning can be understood, as “AI” tends to misuse terms. This could be also due to the algorithm requiring to copyright the answer, instead of citing it verbatim.

Expensive. Promoters talk about constant model improvement, but omit the associated cost increase. This question is especially relevant to countries with lower labor cost. Companies already choose PostgreSQL because it’s free, when Oracle DB is technically superior. So why would they pay for a chatbot, if labor is cheaper?
What about the lost opportunity of growing senior staff, and being locked-in to a LLM technology? In the short-term, companies expect to save 4 salaries when employee is doing 5 times the work, however, this is a strategic loss in the R&D environment, as the company will lag behind a competitor, who now has 4 more employees with 5 years of deep expertise, compared to one LLM-dependent all-rounder, using the model, knowledge of which hasn't advanced by 5 years of non-public “know-how”.

Lower job quality. It is satisfying to count resolved tickets, but having less time for each problem leads to more errors, creating work when it is least expected. Having little own knowledge, the LLM agent lead won’t resolve those issues, even with heavy-duty prompting, as the solution is not in the training data.

Long-term availability. There is no guarantee that services will be available tomorrow at today’s price. Particularly for models with non-disclosed sources of training data, where license risks exist, and when companies sell “AI” below cost.

Serving the interest of “big tech”. As the model can’t distinguish the weight of arguments, for some questions it tends to answer with marketing materials, widely available on the internet, thus more probable. There is possibility of conflicts of interest within the model, but you won’t know it, as you can’t ask the chatbot to disclose it.

Companies will return to IT professionals over time to janitor the LLM slop. Big tech just can’t rehire employees explicitly, for now. There is still a strong need to demonstrate efficiency gains from the products they sell.
As soon as customers start choosing non-LLM-focused competitors, I expect a strategic senior IT employee HR crisis will happen. The question is when will it become evident, and whether there are alternatives for the customer.