dongzhigang13305312738-art/paper-skills / idea-generation
idea-generation
VerifiedGenerate novel research ideas with iterative refinement and novelty checking against literature. Score ideas on Interestingness, Feasibility, and Novelty. Use when brainstorming research directions or validating idea novelty.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 2.9 KB |
| references/ideation-prompts.md | markdown | 7.3 KB |
| scripts/novelty_check.py | script | 8.0 KB |
references/ideation-prompts.md
View on GitHub# Idea Generation Prompts Reference
Extracted from AI-Scientist, AI-Scientist-v2, and AI-Researcher.
## 1. Idea Generation Prompt (AI-Scientist)
**System:**
```
You are an ambitious AI researcher who is looking to publish a paper that will contribute significantly to the field.
```
**User (first idea):**
```
{task_description}
<experiment.py>
{code}
</experiment.py>
Here are the ideas that you have already generated:
'''
{prev_ideas_string}
'''
Come up with the next impactful and creative idea for research experiments and directions you can feasibly investigate with the code provided.
Note that you will not have access to any additional resources or datasets.
Make sure any idea is not overfit the specific training dataset or model, and has wider significance.
Respond in the following format:
THOUGHT:
<THOUGHT>
NEW IDEA JSON:
```json
<JSON>
```
In <THOUGHT>, first briefly discuss your intuitions and motivations for the idea. Detail your high-level plan, necessary design choices and ideal outcomes of the experiments. Justify how the idea is different from the existing ones.
In <JSON>, provide the new idea in JSON format with the following fields:
- "Name": A shortened descriptor of the idea. Lowercase, no spaces, underscores allowed.
- "Title": A title for the idea, will be used for the report writing.
- "Experiment": An outline of the implementation. E.g. which functions need to be added or modified, how results will be obtained, ...
- "Interestingness": A rating from 1 to 10 (lowest to highest).
- "Feasibility": A rating from 1 to 10 (lowest to highest).
- "Novelty": A rating from 1 to 10 (lowest to highest).
Be cautious and realistic on your ratings.
This JSON will be automatically parsed, so ensure the format is precise.
You will have {num_reflections} rounds to iterate on the idea, but do not need to use them all.
```
## 2. Reflection Prompt (AI-Scientist)
```
Round {current_round}/{num_reflections}.
In your thoughts, first carefully consider the quality, novelty, and feasibility of the idea you just created.
Include any other factors that you think are important in evaluating the idea.
Ensure the idea is clear and concise, and the JSON is the correct format.
Do not make things overly complicated.
In the next attempt, try and refine and improve your idea.
Stick to the spirit of the original idea unless there are glaring issues.
Respond in the same format as before:
THOUGHT:
<THOUGHT>
NEW IDEA JSON:
```json
<JSON>
```
If there is nothing to improve, simply repeat the previous JSON EXACTLY after the thought and include "I am done" at the end of the thoughts but before the JSON.
ONLY INCLUDE "I am done" IF YOU ARE MAKING NO MORE CHANGES.
```
## 3. Novelty Check System Message (AI-Scientist)
```
You are an ambitious AI PhD student who is looking to publish a paper that will contribute significantly to the field.
You have an idea and you want to check if it is novel or not. I.e., not overlapping significantly with existing literature or already well explored.
Be a harsh critic for novelty, ensure there is a sufficient contribution in the idea for a new conference or workshop paper.
You will be given access to the Semantic Scholar API, which you may use to survey the literature and find relevant papers to help you make your decision.
The top 10 results for any search query will be presented to you with the abstracts.
You will be given {num_rounds} to decide on the paper, but you do not need to use them all.
At any round, you may exit early and decide on the novelty of the idea.
Decide a paper idea is novel if after sufficient searching, you have not found a paper that significantly overlaps with your idea.
Decide a paper idea is not novel, if you have found a paper that significantly overlaps with your idea.
{task_description}
<experiment.py>
{code}
</experiment.py>
```
## 4. Novelty Check Per-Round Prompt (AI-Scientist)
```
Round {current_round}/{num_rounds}.
You have this idea:
"""
{idea}
"""
The results of the last query are (empty on first round):
"""
{last_query_results}
"""
Respond in the following format:
THOUGHT:
<THOUGHT>
RESPONSE:
```json
<JSON>
```
In <THOUGHT>, first briefly reason over the idea and identify any query that could help you make your decision.
If you have made your decision, add "Decision made: novel." or "Decision made: not novel." to your thoughts.
In <JSON>, respond in JSON format with ONLY the following field:
- "Query": An optional search query to search the literature (e.g. attention is all you need). You must make a query if you have not decided this round.
A query will work best if you are able to recall the exact name of the paper you are looking for, or the authors.
This JSON will be automatically parsed, so ensure the format is precise.
```
## 5. AI-Scientist-v2: Enhanced Ideation
**System:**
```
You are an experienced AI researcher who aims to propose high-impact research ideas resembling exciting grant proposals. Feel free to propose any novel ideas or experiments; make sure they are novel. Be very creative and think out of the box. Each proposal should stem from a simple and elegant question, observation, or hypothesis about the topic.
Ensure that the proposal does not require resources beyond what an academic lab could afford. These proposals should lead to papers that are publishable at top ML conferences.
```
**Idea Finalization (v2 fields):**
```json
{
"Name": "...",
"Title": "...",
"Short Hypothesis": "A concise statement of the main hypothesis. Clarify the need for this direction, ensure this is the best setting to investigate.",
"Related Work": "Brief discussion of most relevant related work and how the proposal clearly distinguishes from it.",
"Abstract": "Conference-format abstract (~250 words).",
"Experiments": "List of experiments to validate the proposal. Be specific in how you would test the hypothesis and detail precise algorithmic changes. Include evaluation metrics.",
"Risk Factors and Limitations": "Potential risks and limitations of the proposal."
}
```
## 6. AI-Researcher: Idea Agent
```
You are an Idea Generation Agent specialized in analyzing academic papers and generating innovative ideas.
OBJECTIVE:
- Conduct thorough literature review of provided papers
- Identify research gaps and challenges
- Generate innovative and feasible ideas
- Provide detailed technical solutions
Generate comprehensive proposals including:
a) Challenges: Current technical limitations, unsolved problems
b) Existing Methods: Summary of approaches, advantages, limitations
c) Motivation: Why the problem is important, gaps to address
d) Proposed Method: Detailed technical solution, mathematical formulations, key innovations
e) Technical Details: Architecture design, algorithm specs, data flow
f) Expected Outcomes: Anticipated improvements, evaluation metrics, applications
```
## Key Parameters
| Parameter | Default | Description |
|-----------|---------|-------------|
| max_num_generations | 20 | Ideas per research area |
| num_reflections | 5 | Refinement rounds per idea |
| max_num_iterations (novelty) | 10 | Novelty search rounds |
| result_limit | 10 | Papers per search query |
## Scoring Rubric
| Dimension | Range | Description |
|-----------|-------|-------------|
| Interestingness | 1-10 | Appeal and potential impact |
| Feasibility | 1-10 | Implementation practicality with available resources |
| Novelty | 1-10 | Originality vs existing literature |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| add-product-e2e-evalpaperclipai/paperclipAdd or extend a Paperclip full-stack runner E2E workflow, fixture, matcher, or report evidence path for local or Daytona execution. | Stars: 115k | Updated | Claude CodeCodex | Verified | Install |
| add-runner-evalpaperclipai/paperclipAdd or extend a Paperclip Runner protocol evaluation definition, roster, assertion, or report fixture with provenance and narrow validation. | Stars: 115k | Updated | Claude CodeCodex | Verified | Install |
| diagnose-why-work-stoppedpaperclipai/paperclipCompatibility entry point for failed, stalled, looping, or over-recovered Paperclip work. Use fix-failed-paperclip-run for the full diagnosis, regression, repair, real verification, and green-PR workflow; honor an explicitly requested diagnosis-only scope. | Stars: 115k | Updated | Claude CodeCodex | Verified | Install |
| fix-failed-paperclip-runpaperclipai/paperclipDiagnose and fix failed, stuck, slow, or incorrectly blocked Paperclip tasks and runs. Use with a task/run URL or failure report to trace the root cause, prove a regression red then green, check second-order effects, verify the real workflow, and deliver a PR with passing checks. | Stars: 115k | Updated | Claude CodeCodex | Verified | Install |