By Michael Ramos —

Lessons from modernizing 10 services using Generative AI

I joined a new team at IBM a few months back, with the goal of creating MCP servers for many of the internal APIs that support employees. As I began getting up to speed on the project, I realized the APIs could use some love. These APIs are heavily used production services, but they lack much testing, documentation, and coding standards. I was worried what would happen if I needed to make changes to the API while I created the MCP servers. I decided I needed to first take a pass at improving the APIs, and I thought this would be a great opportunity to use generative AI.

For this project, I used IBM Bob. Bob is an AI-powered development partner that helps write and manage code. This is my first experience using AI powered coding tools, and I'd like to share what I've learned. If you're new to using AI at work, you might find these tips useful in your projects too.

Start with learning about the project

The most important thing to tackle early is understanding the project you're working on. Since I was new to the team, I didn't have any context around the history of the projects, and I didn't even know what most of these services did. To make matters worse, most of the projects were lacking any documentation. Before changing any code, I asked Bob to document the project so I could begin to understand what the service did and how it fit in the overall stack.

Here's the real prompt I used during this stage:

Task 1: Documentation

Requirements

The repository must have a concise, newcomer-friendly README.md that includes:
 • What this repository does (1–2 paragraphs)
 • Technologies used
 • High-level architecture (use Mermaid diagrams)
 • How to run locally (prerequisites + commands)
 • How tests are run
 • How this service is deployed (if applicable)

If the README is missing or incomplete:
 • Create or rewrite it
 • Keep it concise and skimmable
 • Assume the reader has never seen this repo before

This resulted in documentation that I could easily consume and get an overall understanding of the project. This understanding helps guide the follow-up steps.

Plan mode is your friend

Plan mode is available on many AI tools now, and I found it to be incredibly useful for this project. By using plan mode, you give the AI opportunity to think through all changes before just jumping into coding. This also helped me think through exactly what I wanted to accomplish. Practically, using plan mode means you're less likely to find the AI gets confused or misinterpreted your prompt.

During this project, I followed a workflow:

  1. Craft what I thought was a great prompt and ask the AI to create a phased plan for implementation. Write the plan to markdown.
  2. In a loop, as the AI to implement the phases and update the plan with status. For each implementation phase, I started a new task to clear context.
  3. Commit the changes in between tasks to save state.

If the plan needed adjusting, there was opportunity to do so between tasks. However, if things got too off track, I found it easier to scrap the implementation in progress and go back to planning from the beginning.

Don't overwhelm your reviewers

The same best practices we used before AI generated code are still important. For this project, that meant splitting my changes into multiple pull requests so I could keep the size of the changes small. I decided to organize the changes by level of risk. First, documentation only changes, testing improvements next, followed by code changes. This makes for two pull requests that should be very easy to review, and a third that is now well documented and tested to aid reviewers.

At IBM, we don't have any AI tools specifically created for reviewing pull requests yet, but these would help. I've experimented with using the same tool that writes code to review it, but there is high risk with this approach: using the same model could mean making the same mistakes. If available, a tool for reviewing AI generated code could improve the bottleneck of pull requests.

Don't boil the ocean

I had goals at the beginning of the project to improve documentation and testing, and I had to restrain myself from rewriting entire services. AI tools can make it so easy to quickly make large changes, which feels so powerful in the moment. However, in my case, these are production services and any changes require careful reviews. Even though it is easy to make sweeping changes, I'm not an expert in the project and reviews would be lengthy. Instead, set targeted goals and stick to them.

In the end, using generative AI didn’t change the fundamentals of good engineering. Clear goals, thoughtful planning, small and reviewable changes, and a solid understanding of the codebase still mattered just as much as before. Tools like Bob helped me move faster and build confidence as I improved unfamiliar services, but the real value came from using AI deliberately and not letting it run unchecked. If you’re new to AI-powered coding tools, start small, stay disciplined, and treat them as a partner.

  1. Start with learning about the project
  2. Set standards early
  3. Plan mode is your friend
  4. A consistent prompt makes for maintainable projects
  5. Don't overwhelm your reviewers
  6. Before changing any code, add tests
  7. Use AI to look for inconsistencies
  8. Don't boil the ocean