When AI Falls Short: A Developer's Guide to Knowing Better

Jorge Cortez
January 14, 2025
Read time
Tech Talk

AI vs. A Developer with Good Documentation

Imagine this: you’re at work, trying to implement a new functionality in a project. It’s a seemingly simple task—nothing too complicated, or so you think.

As a programmer in 2025, you have a powerful arsenal of tools at your disposal: Mr. G, Copilot, Cursor, or any of the numerous LLMs designed to enhance productivity. Naturally, you turn to one of them for help.

You type out a straightforward prompt: "Act as an expert in {your domain}. Generate a template for the integration I’m building, explain how it works, and show how to achieve XYZ." Confident, you hit enter. Boom—the AI generates everything you need. You copy-paste the code, fill in your API keys, and run the app.

And… it fails.

Does This Happen?

Unfortunately, yes—and it happened to me just a few days ago.

I was setting up a CI/CD pipeline between Azure and GitHub using GitHub Actions. On paper, this integration shouldn’t be that complex. It took me about 15 minutes to set up the basics—adding permissions, configuring the GitHub Action, approving the Azure connection request. You know the drill.

But then came the hard part: troubleshooting why the pipeline wouldn’t work as expected.

I spent over an hour prompting AI tools like Copilot (on Azure), ChatGPT, and GitHub Copilot, trying to pinpoint the issue. Each AI-generated solution was close but not quite right. After numerous failed attempts, I decided to ditch AI for this task.

I turned to something many might consider antiquated in 2025: reading the documentation.

Sure enough, the answer was there, hidden in Azure’s CI/CD setup guide. A small but critical detail in their example script resolved the problem immediately.

‍

Photo by Emiliano Vittoriosi on Unsplash

The Role of AI in Development

AI has come a long way, but situations like these highlight an important reality: AI is not infallible. It’s incredibly powerful in the right hands but limited by its context and the data it’s trained on.

This isn’t to say AI is bad. Far from it. The issue often lies in its inability to fill in gaps that only a human—with experience, intuition, and access to up-to-date documentation—can address.

When AI fails, it’s usually because it lacks:

  • Context: AI can’t fully understand your project, codebase, or specific problem.
  • Adaptability: AI might not have access to the most recent documentation or nuanced examples.
  • Judgment: AI can’t differentiate between a correct solution and one that only seems correct.

This is why I believe AI, for now, remains a tool—a powerful one, but not a replacement for a knowledgeable developer.

The Rise of AI Over-Reliance

There’s an emerging pattern in the developer community that concerns me. Some users are becoming overly reliant on AI, expecting it to solve every problem perfectly.

Let’s break down the types of AI users I’ve observed:

1. The Conservatives
‍
These are the skeptics. They believe AI is coming for their jobs—or worse, will bring about the rise of Skynet and doom humanity. They avoid AI entirely, preferring to solve problems manually, even if it takes longer. In my experience, many in this group are older professionals nearing retirement.

2. The Hobbyists
‍
Hobbyists see AI as a novelty. They play around with it for fun—generating DnD character names, sorting data in Excel, or drafting creative writing prompts. They don’t use AI for critical tasks but enjoy exploring its capabilities.

3. The Progressives
‍
This group (which includes me) sees AI as a tool with immense potential but clear limitations. We view AI as a complement to human expertise, not a replacement. Progressives understand that while AI can assist with tasks like code generation or research, it still requires human oversight.

4. The Radicals
‍
Radicals want AI to do everything for them. Need an essay? Ask Mr. G. Need a new app? Let an LLM generate it. While this group often pushes AI to its limits, their blind trust in AI is problematic. They tend to accept AI-generated outputs without question, leading to errors and inefficiencies.

‍

Photo by Fabian Grohs on Unsplash

Why Good Documentation Still Matters

The key takeaway from my CI/CD pipeline experience is this: good documentation is irreplaceable.

While AI can provide templates, explanations, and even full implementations, it often lacks the specificity and reliability of well-maintained documentation. When faced with a tricky issue, there’s no substitute for going straight to the source.

In a world increasingly dominated by AI, developers who understand and utilize documentation effectively will have a significant edge. They’ll know when to rely on AI—and when to fall back on tried-and-true methods.

The Future of Development

As AI continues to evolve, so will its role in our workflows. I believe the future will look something like this:

  • AI as a Co-Pilot: Developers will act as orchestrators, using AI to automate repetitive tasks while providing oversight and direction.
  • Emphasis on Prompt Engineering: Crafting effective prompts will become a valuable skill, akin to debugging or optimization today.
  • Enhanced AI-Documentation Integration: Tools like Copilot will eventually integrate seamlessly with official documentation, bridging the gap between AI-generated solutions and real-world requirements.

But until then, we need to approach AI with caution. It’s a tool, not a crutch. Developers must continue honing their skills, staying curious, and—most importantly—reading the docs.

Final Thoughts

AI is transformative, no doubt about it. But it’s not a silver bullet. The best developers will always be those who combine the power of AI with a deep understanding of their craft, guided by experience, intuition, and—yes—good documentation.

Let’s embrace AI, but let’s also remember what makes us human: our ability to learn, adapt, and solve problems creatively.

‍

Read Next

Explore Blog
A look back at 2024: rediscovering hobbies, embracing challenges, and growing as a software engineer. From launching projects like WebTricks to setting ambitious goals like achieving a GitHub Green Wall in 2025, join me as I reflect on the past year and share my plans for the future.
2024 in Review: Growth, Challenges, and Big Plans for 2025
Discover Webtricks, a free, open-source JavaScript library to enhance no-code platforms. Built for developers, by developers, with scripts to overcome limitations and improve web functionality.
Webtricks is here! - Merry Christmas for the no-code community!
From learning to code in the early 2000 to building a tech career, my journey's been all about learning, experimenting, and growing with every line of code. Here’s where it’s led me.
A decade as a profesional software developer