Wednesday, 7 May 2025

From Good to Great: Enhancing LLM Responses with Self-Refinement

In the rapidly evolving world of generative AI, tools like Large Language Models (LLMs) offer unprecedented possibilities for automation, content generation, and problem-solving. However, these models generate responses based on probabilities, not inherent correctness. This can lead to inaccuracies or incomplete answers.

 

To address this, the self-refinement technique has emerged as a powerful strategy. By iteratively evaluating and improving an AI's output, you can achieve better accuracy, clarity, and depth in responses.

 

What is Self-Refinement?

Self-refinement is a structured process where the output of an LLM is evaluated critically and refined iteratively to produce better responses. The key idea is to leverage the model’s ability to respond to feedback, either provided by you or generated by the model itself.

 

The technique can be broken down into these steps:

1.   Initial Prompt: Pose a question or problem to the LLM.

2.   Initial Response: Allow the model to generate a response.

3.   Critique: Provide feedback on the response, identifying areas for improvement.

4.   Refinement: Ask the model to refine its response based on the critique.

5.   (Optional) Repeat: Continue iterating until the desired quality is achieved.

 

 

Example: To generate better program to find prime number

 

Prompt: Generate a Java program to check whether a number is prime or not

 

AI Answer: Here’s a simple Java program to check whether a number is prime:

import java.util.Scanner;

public class PrimeChecker {
    public static void main(String[] args) {
        Scanner scanner = new Scanner(System.in);

        // Prompt user for input
        System.out.print("Enter a number to check if it's prime: ");
        int number = scanner.nextInt();

        // Check if the number is prime
        if (isPrime(number)) {
            System.out.println(number + " is a prime number.");
        } else {
            System.out.println(number + " is not a prime number.");
        }

        scanner.close();
    }

    /**
     * Checks if a number is prime.
     * @param number the number to check
     * @return true if the number is prime, false otherwise
     */
    public static boolean isPrime(int number) {
        if (number <= 1) {
            return false; // Numbers less than or equal to 1 are not prime
        }
        for (int i = 2; i <= Math.sqrt(number); i++) {
            if (number % i == 0) {
                return false; // Found a divisor, not prime
            }
        }
        return true; // No divisors found, number is prime
    }
}

Prompt:

 

I notice that you're explicitly closing the Scanner, but if an exception occurs in the isPrime method, the Scanner will not be properly closed.

 

AI Answer

You're absolutely right! Explicitly closing the Scanner without considering exceptions could leave it open if an exception is thrown before reaching the close statement. To ensure the Scanner is properly closed even when an exception occurs, you can use the try-with-resources construct or a finally block.

Option 1: Use Try-With-Resources
The try-with-resources construct ensures that resources implementing AutoCloseable (like Scanner) are automatically closed when the block is exited, whether normally or due to an exception.

import java.util.Scanner;

public class PrimeCheck {
    public static void main(String[] args) {
        try (Scanner scanner = new Scanner(System.in)) {
            System.out.print("Enter a number: ");
            int number = scanner.nextInt();
            
            if (isPrime(number)) {
                System.out.println(number + " is a prime number.");
            } else {
                System.out.println(number + " is not a prime number.");
            }
        } catch (Exception e) {
            System.out.println("An error occurred: " + e.getMessage());
        }
    }

    public static boolean isPrime(int number) throws IllegalArgumentException {
        if (number <= 1) throw new IllegalArgumentException("Number must be greater than 1.");
        for (int i = 2; i <= Math.sqrt(number); i++) {
            if (number % i == 0) {
                return false;
            }
        }
        return true;
    }
}


Option 2: Use a finally Block
If you cannot use try-with-resources (e.g., older Java versions), a finally block ensures the Scanner is closed.

import java.util.Scanner;

public class PrimeCheck {
    public static void main(String[] args) {
        Scanner scanner = null;
        try {
            scanner = new Scanner(System.in);
            System.out.print("Enter a number: ");
            int number = scanner.nextInt();

            if (isPrime(number)) {
                System.out.println(number + " is a prime number.");
            } else {
                System.out.println(number + " is not a prime number.");
            }
        } catch (Exception e) {
            System.out.println("An error occurred: " + e.getMessage());
        } finally {
            if (scanner != null) {
                scanner.close();
            }
        }
    }

    public static boolean isPrime(int number) throws IllegalArgumentException {
        if (number <= 1) throw new IllegalArgumentException("Number must be greater than 1.");
        for (int i = 2; i <= Math.sqrt(number); i++) {
            if (number % i == 0) {
                return false;
            }
        }
        return true;
    }
}

Prompt

 

Great, let's use the try-with-resources statement, move the logic into a utility class, and remove the main method.

AI Answer:

Certainly! Here's how you can refactor the code by moving the logic into a utility class, using the try-with-resources statement, and removing the main method:

import java.util.Scanner;

public class PrimeNumberUtility {

    public static boolean isPrime(int number) throws IllegalArgumentException {
        if (number <= 1) throw new IllegalArgumentException("Number must be greater than 1.");
        for (int i = 2; i <= Math.sqrt(number); i++) {
            if (number % i == 0) {
                return false;
            }
        }
        return true;
    }

    public static void checkPrime() {
        try (Scanner scanner = new Scanner(System.in)) {
            System.out.print("Enter a number: ");
            int number = scanner.nextInt();

            if (isPrime(number)) {
                System.out.println(number + " is a prime number.");
            } else {
                System.out.println(number + " is not a prime number.");
            }
        } catch (Exception e) {
            System.out.println("An error occurred: " + e.getMessage());
        }
    }
}

Benefits of Self-Refinement:

·       Improved Accuracy: Encourages the model to consider alternative perspectives and fix potential errors.

·       Deeper Exploration: Allows for iterative deepening of answers by exploring nuances.

·       Collaboration Simulation: Mimics a human collaboration process where feedback refines solutions.


 

Previous                                                    Next                                                    Home

No comments:

Post a Comment