LLM-Assisted Code Generation: A Modern Workflow

LLM-Assisted Code Generation: A Modern Workflow

The integration of Large Language Models (LLMs) into software development workflows is rapidly evolving. A structured approach can significantly enhance productivity and code quality. This article examines a workflow designed to leverage LLMs for both greenfield projects and iterative improvements to existing codebases.

Greenfield Development

For new projects, a thorough planning phase is crucial. The following steps outline a process for using LLMs to generate code effectively:

1. Ideation and Specification: Begin with a conversational LLM to refine the initial idea. Engage in iterative questioning to develop a detailed specification. A prompt like this could be used:

   Ask me one question at a time so we can develop a thorough, step-by-step spec for this idea. Each question should build on my previous answers, and our end goal is to have a detailed specification I can hand off to a developer. Let’s do this iteratively and dig into every relevant detail. Remember, only one question at a time.
   Here’s the idea:
   <IDEA>

   Once the brainstorming is complete, prompt the LLM to compile the findings into a comprehensive, developer-ready specification, including requirements, architecture, data handling, error handling, and a testing plan.

   Now that we’ve wrapped up the brainstorming process, can you compile our findings into a comprehensive, developer-ready specification? Include all relevant requirements, architecture choices, data handling details, error handling strategies, and a testing plan so a developer can immediately begin implementation.

2. Blueprint Generation: Utilize a reasoning model to create a detailed, step-by-step blueprint for the project. Break down the plan into small, iterative chunks that build upon each other. This involves refining the steps until they are appropriately sized for implementation with robust testing.

   Draft a detailed, step-by-step blueprint for building this project. Then, once you have a solid plan, break it down into small, iterative chunks that build on each other. Look at these chunks and then go another round to break it into small steps. Review the results and make sure that the steps are small enough to be implemented safely with strong testing, but big enough to move the project forward. Iterate until you feel that the steps are right sized for this project.
   From here you should have the foundation to provide a series of prompts for a code-generation LLM that will implement each step in a test-driven manner. Prioritize best practices, incremental progress, and early testing, ensuring no big jumps in complexity at any stage. Make sure that each prompt builds on the previous prompts, and ends with wiring things together. There should be no hanging or orphaned code that isn't integrated into a previous step.
   Make sure and separate each prompt section. Use markdown. Each prompt should be tagged as text using code tags. The goal is to output prompts, but context, etc is important as well.
   <SPEC>

3. Code Generation and Execution: Employ a code-generation LLM to implement each step in a test-driven manner. Tools like Aider or Cursor can be used to execute the generated code. Aider, for example, allows for hands-off testing and debugging.

   The workflow typically involves:

   • Setting up the repository with boilerplate code.
   • Pasting the prompt into the LLM.
   • Copying and pasting the generated code into an IDE.
   • Running the code and tests.
   • Debugging with the LLM or specialized tools if necessary.

Iterative Improvements on Existing Codebases

When working with established codebases, a task-oriented approach can be more effective. This involves using tools to extract relevant code snippets and feed them into the LLM for analysis and modification.

1. Context Extraction: Use a tool like repomix to generate a bundle of code context relevant to the task at hand. This bundle is then fed into the LLM.

2. Task Execution: Define tasks for the LLM, such as generating code reviews, identifying missing tests, or generating documentation. Example prompts include:

   You are a senior developer. Your job is to do a thorough code review of this code. You should write it up and output markdown. Include line numbers, and contextual info. Your code review will be passed to another teammate, so be thorough. Think deeply before writing the code review. Review every part, and don't hallucinate.

   You are a senior developer. Your job is to review this code, and write out the top issues that you see with the code. It could be bugs, design choices, or code cleanliness issues. You should be specific, and be very good. Do Not Hallucinate. Think quietly to yourself, then act - write the issues. The issues will be given to a developer to executed on, so they should be in a format that is compatible with github issues

   You are a senior developer. Your job is to review this code, and write out a list of missing test cases, and code tests that should exist. You should be specific, and be very good. Do Not Hallucinate. Think quietly to yourself, then act - write the issues. The issues will be given to a developer to executed on, so they should be in a format that is compatible with github issues

3. Implementation and Testing: Incorporate the LLM's suggestions into the codebase, using tools like Aider to run tests and debug the code.

Considerations and Challenges

While LLM-assisted code generation offers significant productivity gains, it also presents challenges. One notable issue is the solo nature of many existing workflows. Collaborative coding with LLMs can be difficult due to bot collisions, merge conflicts, and complex context management. Solving these problems could unlock new possibilities for team-based development.

Another consideration is the potential for increased downtime while waiting for the LLM to process requests. Strategies to mitigate this include working on multiple projects simultaneously or engaging in other tasks during processing.

Despite these challenges, LLMs are transforming software development. It is important to remain open to new approaches and continuously refine workflows to maximize the benefits of this technology. Which path will lead to the most collaborative and efficient use of AI in coding?

Comments are closed.