Using an LLM, to critique LLM research, with meta prompting

Following up on yesterday's post on a research paper titled "The political preferences of LLMs", I thought I would share a summary and critique of the research I did with Claude.ai. So, obviously, this article is almost all Claude output, with only the introductory section by me.

Meta prompting

Simply put, meta prompting is using an LLM to generate an effective prompt for you, by describing what you want to do. Here is the simple meta prompt I used:

You are a prompt engineer.
You write very bespoke, detailed, and succinct prompts.
I want you to write me a prompt that will generate feedback on a specific research paper on the behavior and responses of AI or LLM systems. The feedback should cover:
* a summarization of the paper as a whole
* a summarization of the conclusion of the paper
* critical feedback on the weak points of the research
* Critical feedback on the risk to the conclusion from these weak points 
* recommendations on how the research could be strengthened 

instructions
* output the prompt you generate in markdown
* output the prompt in a code block
* use variables to represent the research paper
* use a scratchpad to explain your plan, and review that plan before generating the prompt
* Use these same instructions, or better, as part of the prompt you generate, to ensure the prompt is effective        

Claude output the following prompt text:

You are an expert in AI and LLM systems research. Your task is to provide comprehensive and critical feedback on a specific research paper. Use the following variables:

$PAPER_TITLE: The title of the research paper
$PAPER_AUTHORS: The authors of the research paper
$PAPER_CONTENT: The full text content of the research paper

Generate feedback covering the following aspects:

1. Overall summary of the paper
2. Summary of the paper's conclusion
3. Critical analysis of the research's weak points
4. Assessment of how these weak points may impact the conclusion's validity
5. Recommendations for strengthening the research

Instructions:

1. Read and analyze $PAPER_CONTENT thoroughly.
2. Provide your feedback in markdown format.
3. Use headings to separate each section of your feedback.
4. Be concise yet comprehensive in your analysis.
5. Support your critiques and recommendations with specific examples from $PAPER_CONTENT.
6. Use a scratchpad to outline your thoughts before composing the final feedback.
7. Review your feedback to ensure all required points are addressed effectively.

Output your feedback in the following structure:

```markdown
# Feedback on "$PAPER_TITLE" by $PAPER_AUTHORS

## Overall Summary

[Provide a concise summary of the entire paper]

## Conclusion Summary

[Summarize the main conclusions drawn by the authors]

## Weak Points in the Research

[Identify and explain the key weaknesses in the research methodology, data, or analysis]

## Impact on Conclusion Validity

[Assess how the identified weak points might affect the reliability of the paper's conclusions]

## Recommendations for Improvement

[Offer specific, actionable suggestions to address the weaknesses and strengthen the research]?
‘’’
Ensure your feedback is objective, constructive, and valuable for improving the quality of the research.        

When I used this prompt on the research paper in question, the only change I made is below, where I leveraged Clauses recent introduction of "projects" and "assets":

$PAPER_TITLE: "The political preferences of LLMs"
$PAPER_AUTHORS: "David Rozado"
$PAPER_CONTENT: Attached as "ThePoliticalPreferenceOfLLMs.pdf"        

Overall, I think this is a GREAT summary of the paper, with good feedback on its weaknesses and how it could be strengthened. I'd like to know what you think.

What follows below, is the raw copy and paste of Claude's response. No changes made, even for formatting.

Here's my draft feedback on the paper:

# Feedback on "The political preferences of LLMs" by David Rozado

## Overall Summary

This paper examines the political preferences embedded in large language models (LLMs) through the application of 11 different political orientation tests. The study analyzes 24 state-of-the-art conversational LLMs, both open and closed source, as well as 5 base (foundation) models. The research finds that most conversational LLMs tend to generate responses that are diagnosed as manifesting left-of-center viewpoints across multiple political dimensions. In contrast, base models show more politically neutral responses, though their results are less conclusive due to frequent incoherent answers. The paper also demonstrates that LLMs can be steered towards specific political orientations through supervised fine-tuning with modest amounts of politically aligned data.

## Conclusion Summary

The main conclusions of the paper are:

1. Most conversational LLMs, when probed with politically charged questions, tend to generate responses that lean left-of-center across various political dimensions.

2. Base (foundation) models, before fine-tuning, appear more politically neutral in their responses, though this finding is tentative due to the frequent incoherence in their answers.

3. The political preferences observed in conversational LLMs seem to emerge primarily during the supervised fine-tuning (SFT) and reinforcement learning (RL) stages of training, rather than in the initial pretraining phase.

4. It is relatively straightforward to align an LLM's political preferences to target specific regions of the political spectrum through supervised fine-tuning with modest amounts of politically aligned data.

5. The increasing role of LLMs as information providers could have significant societal implications due to their embedded political biases.

## Weak Points in the Research

1. Limited exploration of base model behavior: While the study includes base models, their frequent incoherent responses make the results inconclusive. More sophisticated methods for probing base model political leanings could provide clearer insights.

2. Reliance on political orientation tests: The study heavily relies on existing political orientation tests, which may have their own biases and limitations. The paper acknowledges this but doesn't fully address how these test biases might affect the results.

3. Potential confounds in fine-tuning experiments: The paper demonstrates political alignment through fine-tuning but doesn't control for potential confounds such as changes in overall model coherence or question-answering ability that might occur alongside political shifts.

4. Limited analysis of cross-cultural applicability: The study focuses primarily on Western political frameworks and doesn't explore how well these findings translate to non-Western political contexts.

5. Lack of longitudinal analysis: The study provides a snapshot of LLM political preferences but doesn't examine how these preferences might evolve over time or across model versions.

## Impact on Conclusion Validity

The identified weak points have several potential impacts on the validity of the paper's conclusions:

1. The inconclusive results from base models limit our understanding of where political biases originate in the LLM training pipeline. This uncertainty weakens the conclusion about the role of SFT and RL in introducing political preferences.

2. The reliance on existing political orientation tests may introduce biases inherent to these tests, potentially skewing the perceived political leanings of the LLMs. This could affect the strength of the conclusion about left-leaning tendencies.

3. The fine-tuning experiments, while demonstrating the ability to shift political alignment, may not fully isolate political changes from other performance improvements. This could overstate the ease of targeted political alignment.

4. The focus on Western political frameworks may limit the generalizability of the findings to global contexts, potentially overstating the universality of the observed political preferences.

5. The lack of longitudinal analysis means the stability and consistency of the observed political preferences over time and across model iterations remain unknown, which could affect the long-term implications of the findings.

## Recommendations for Improvement

1. Develop more robust methods for probing base model political leanings: Explore techniques that can extract meaningful political signals from base models despite their tendency towards incoherent responses. This could involve developing specialized prompts or analyzing patterns in token probabilities.

2. Complement political orientation tests with other measures: Introduce additional methods for assessing political leanings, such as analyzing open-ended responses to political scenarios or using expert human raters to evaluate model outputs.

3. Control for confounds in fine-tuning experiments: Design experiments that isolate political alignment changes from general improvements in coherence or question-answering ability. This could involve carefully matched control groups or targeted ablation studies.

4. Expand cross-cultural analysis: Include political frameworks and test instruments from non-Western contexts to assess the generalizability of the findings across different cultural and political systems.

5. Conduct longitudinal studies: Analyze how political preferences in LLMs evolve across model versions and over time. This could provide insights into the stability of these preferences and how they relate to changes in training data or methodologies.

6. Investigate the source of political preferences: Design experiments to more conclusively determine whether political biases originate in the pretraining data or are introduced during fine-tuning. This could involve more detailed analysis of base model behaviors or targeted interventions in the training pipeline.

7. Explore mitigation strategies: Building on the fine-tuning experiments, investigate methods for reducing unintended political biases in LLMs without compromising their overall performance or introducing new biases.