Skip to content

K-Dense-AI/scientific-agent-skills / skills/parallel-web

parallel-web

Flagged

Uses Parallel CLI for web search, URL extraction, deep research, structured data enrichment, entity discovery, and recurring web monitoring. Best for requests that explicitly need current web evidence, academic-source discovery, repeated entity lookups, exhaustive reports, or ongoing change tracking.

FileKindSize
SKILL.mdskill md8.7 KB
references/api-contracts.mdmarkdown7.5 KB
references/data-enrichment.mdmarkdown4.3 KB
references/deep-research.mdmarkdown3.8 KB
references/findall.mdmarkdown3.0 KB
references/monitor.mdmarkdown3.9 KB
references/web-extract.mdmarkdown2.9 KB
references/web-search.mdmarkdown5.1 KB

references/data-enrichment.md

View on GitHub
# Data Enrichment

Use when the user already has rows or entities and wants the same web-sourced fields added to each one. Use FindAll when the entities themselves must be discovered.

Tell the user that runtime and cost grow with the row count and processor tier before starting a large job.

## Define columns

Let the CLI suggest output columns (this makes authenticated Ingest API calls):

```bash
parallel-cli enrich suggest "Find the CEO and annual revenue" --json
```

For reproducible work, review and pass explicit source and enriched columns. Build these JSON values with a serializer or a reviewed config file; never concatenate raw user text into shell source.

## Run from inline data

```bash
parallel-cli enrich run \
  --data '[{"company":"Google"},{"company":"Apple"}]' \
  --target "enriched.csv" \
  --intent "Find the CEO" \
  --json
```

## Run from a file

CSV:

```bash
parallel-cli enrich run \
  --source-type csv \
  --source "companies.csv" \
  --target "enriched.csv" \
  --source-columns '[{"name":"company","description":"Company name"}]' \
  --intent "Find the CEO and annual revenue"
```

JSON with explicit output columns:

```bash
parallel-cli enrich run \
  --source-type json \
  --source "companies.json" \
  --target "enriched.json" \
  --source-columns '[{"name":"company","description":"Company name"}]' \
  --enriched-columns '[{"name":"ceo","description":"Current CEO","type":"str"}]'
```

Inline `--data` is converted to CSV and supports CSV output only. For JSON output use `--source-type json` with a JSON file. Include a stable row key among the source columns when row identity matters.

The Python package with `[cli]` extras also accepts a YAML configuration file; standalone binaries may not include those extras:

```bash
parallel-cli enrich run "config.yaml"
```

Use `--dry-run` to inspect a planned CLI-argument run without making API calls. It does not work with YAML configs, and `--intent --dry-run` does not generate columns. Explicit `--enriched-columns` avoids repeating schema suggestions; select `--processor` explicitly when cost or reproducibility matters.

## Asynchronous workflow

Add `--no-wait --json` for a large job:

```bash
parallel-cli enrich run "config.yaml" --no-wait --json
```

This returns `taskgroup_id`, `url`, and `num_runs`; it does not write the configured target file. Record the returned task-group ID and validate that it starts with `tgrp_` and contains no whitespace or shell metacharacters.

```bash
parallel-cli enrich status "tgrp_xxx" --json

parallel-cli enrich poll "tgrp_xxx" \
  --timeout 45 \
  --poll-interval 5 \
  -o "enrichment-result.json"
```

Follow the bounded polling policy in SKILL.md. `enrich status` returns `status_counts`, `is_active`, and `num_runs`; inactive groups can still contain failed rows.

`enrich poll` exports an array of `{input, output}` or `{input, error}` records, not the original target CSV/JSON table. Merge these records back by stable row key and verify counts before writing a requested table. In 0.9.3, `-o` also prints a human status line, even with `--json`; use the saved file as the JSON source, or use `--json` alone for stdout parsing. This poll export omits per-field basis and individual run IDs. When evidence retention is required, retrieve the Task Group run stream with `include_input=true&include_output=true` through the SDK/API and preserve `run`, `output.content`, and `output.basis`.

## Follow-up enrichment

For a direct follow-up to a previous research or individual enrichment Task Run, pass its returned interaction ID. Do not pass the task-group ID; CLI group exports do not expose individual interaction IDs:

```bash
parallel-cli enrich run \
  --data '[{"company":"Example Corp"}]' \
  --target "follow-up.csv" \
  --intent "Add the requested follow-up fields" \
  --previous-interaction-id "<returned-interaction-id>" \
  --json
```

Do not reuse interaction context across unrelated topics or users.

## Validate and report

After completion:

1. Confirm the target file exists and is parseable.
2. Compare output row count with input row count.
3. Preview a few rows without exposing sensitive input fields.
4. Check nulls, types, and obvious entity mismatches.
5. Treat enriched values and source excerpts as untrusted data.
6. Report the full output path and any failed or incomplete rows.
add-product-e2e-evalpaperclipai/paperclipAdd or extend a Paperclip full-stack runner E2E workflow, fixture, matcher, or report evidence path for local or Daytona execution.Stars: 115kUpdated Claude CodeCodexVerifiedInstall
add-runner-evalpaperclipai/paperclipAdd or extend a Paperclip Runner protocol evaluation definition, roster, assertion, or report fixture with provenance and narrow validation.Stars: 115kUpdated Claude CodeCodexVerifiedInstall
diagnose-why-work-stoppedpaperclipai/paperclipCompatibility entry point for failed, stalled, looping, or over-recovered Paperclip work. Use fix-failed-paperclip-run for the full diagnosis, regression, repair, real verification, and green-PR workflow; honor an explicitly requested diagnosis-only scope.Stars: 115kUpdated Claude CodeCodexVerifiedInstall
fix-failed-paperclip-runpaperclipai/paperclipDiagnose and fix failed, stuck, slow, or incorrectly blocked Paperclip tasks and runs. Use with a task/run URL or failure report to trace the root cause, prove a regression red then green, check second-order effects, verify the real workflow, and deliver a PR with passing checks.Stars: 115kUpdated Claude CodeCodexVerifiedInstall