Skip to content

K-Dense-AI/scientific-agent-skills / skills/matchms

matchms

Verified

Processes, cleans, compares, and searches tandem mass spectra with matchms. Use for MS/MS file I/O, metadata harmonization, peak filtering, spectral similarity, library matching, score matrices, and molecular-similarity networks. Use pyopenms instead for LC-MS feature detection or proteomics pipelines.

FileKindSize
SKILL.mdskill md12.5 KB
references/filtering.mdmarkdown11.9 KB
references/importing_exporting.mdmarkdown10.6 KB
references/migration.mdmarkdown8.4 KB
references/similarity.mdmarkdown13.8 KB
references/sources.mdmarkdown9.6 KB
references/workflows.mdmarkdown13.9 KB
scripts/library_search.pyscript20.6 KB

references/workflows.md

View on GitHub
# Practical Workflows (matchms 0.33.1)

These patterns use the current 0.33.1 API. Adapt tolerances and quality filters
to the instrument, acquisition method, and scientific question.

## 1. Standard Query/Reference Preprocessing

Apply the same peak processing to both collections. Metadata enrichment can
differ when a reference library contains known structures and queries do not.

```python
from matchms import SpectrumProcessor
from matchms.filtering import (
    default_filters,
    normalize_intensities,
    require_minimum_number_of_peaks,
    require_precursor_mz,
    select_by_mz,
    select_by_relative_intensity,
)
from matchms.importing import load_spectra


def preprocess(path):
    raw = [default_filters(spectrum) for spectrum in load_spectra(path)]
    processor = SpectrumProcessor(
        [
            (require_precursor_mz, {"minimum_accepted_mz": 50.0}),
            normalize_intensities,
            (select_by_mz, {"mz_from": 20.0, "mz_to": 1500.0}),
            (select_by_relative_intensity, {"intensity_from": 0.01}),
            (require_minimum_number_of_peaks, {"n_required": 5}),
        ]
    )
    cleaned, _ = processor.process_spectra(
        raw,
        progress_bar=False,
        create_report=False,
    )
    return cleaned, ["default_filters", *processor.processing_steps]


queries, query_steps = preprocess("queries.mgf")
references, reference_steps = preprocess("library.msp")
assert query_steps == reference_steps
```

`SpectrumProcessor` may reorder built-in filters. Persist the final
`processing_steps`, not only the requested list. Run the aggregate
`default_filters` before the processor; if passed as one callable, it is treated
as a custom filter and placed after registered filters.

## 2. Run the Bundled Library Search

The bundled CLI is the fastest route to a reproducible CSV:

```bash
uv run python scripts/library_search.py \
  queries.mgf library.msp hits.csv \
  --metric modified \
  --tolerance 0.02 \
  --top-k 10 \
  --min-score 0.6 \
  --min-matches 5
```

The script:

- applies the same configurable peak filters to both inputs;
- supports cosine, exact/greedy modified cosine, neutral-loss, BLINK, and Flash
  modes;
- handles both scalar and structured matchms score records;
- rejects non-finite numeric settings and invalid/nonpositive precursors when
  the selected operation needs them;
- applies the requested relative-intensity cutoff inside Flash as well;
- records identifiers, precursor m/z, rank, score, and matched peaks;
- refuses pickle inputs;
- refuses to overwrite output unless `--force`; and
- stops before an unexpectedly large pairwise matrix unless `--max-pairs` is
  explicitly raised.

Use `--help` before batch use.

## 3. Programmatic Spectral-Library Search

```python
from matchms import calculate_scores
from matchms.similarity import ModifiedCosineGreedy

metric = ModifiedCosineGreedy(tolerance=0.02)
scores = calculate_scores(
    references=references,
    queries=queries,
    similarity_function=metric,
)

score_name = "ModifiedCosineGreedy_score"
matches_name = "ModifiedCosineGreedy_matches"
rows = []

for query_index, query in enumerate(queries):
    ranked = scores.scores_by_query(query, name=score_name, sort=True)
    for rank, (reference, value) in enumerate(ranked[:10], start=1):
        rows.append(
            {
                "query_index": query_index,
                "query_id": query.get("spectrum_id", query.get("id")),
                "reference_id": reference.get(
                    "spectrum_id",
                    reference.get("compound_name"),
                ),
                "rank": rank,
                "score": float(value[score_name]),
                "matches": int(value[matches_name]),
            }
        )
```

Filter on both score and matched peaks only after examining their distributions.
Do not label arbitrary cutoffs as universal high/medium/low confidence.

## 4. Efficient Precursor-Gated Search

For identity-oriented search, a precursor gate can reduce expensive spectral
calculations:

```python
from matchms import calculate_scores
from matchms.similarity import ModifiedCosineGreedy, PrecursorMzMatch

scores = calculate_scores(
    references,
    queries,
    PrecursorMzMatch(tolerance=10, tolerance_type="ppm"),
    array_type="sparse",
)
scores.filter_by_range(
    name="PrecursorMzMatch",
    low=0.5,
    above_operator=">=",
)
if len(scores.scores.row) == 0:
    raise ValueError("No candidates passed the precursor gate")
scores.calculate(
    ModifiedCosineGreedy(tolerance=0.02),
    array_type="sparse",
    join_type="left",
)
```

This restricts computation to retained coordinates only when fewer than half
of all pairs remain and the join is left/inner. Otherwise matchms 0.33.1
computes the full matrix before joining; a 1-by-1 input always uses `pair()`.
Stop when no candidates remain. This gate is inappropriate for broad analog
discovery across precursor shifts or adducts.

## 5. Reproducible `Pipeline` Workflow

`Pipeline` combines import, ordered filtering, sparse score gating, and score
calculation. A workflow can also be written to YAML.

```python
from matchms import Pipeline
from matchms.Pipeline import create_workflow

common_filters = [
    "make_charge_int",
    "add_compound_name",
    "derive_adduct_from_name",
    "derive_formula_from_name",
    "clean_compound_name",
    "interpret_pepmass",
    "add_precursor_mz",
    "derive_ionmode",
    "correct_charge",
    ["require_precursor_mz", {"minimum_accepted_mz": 50.0}],
    "normalize_intensities",
    ["select_by_relative_intensity", {"intensity_from": 0.01}],
    ["require_minimum_number_of_peaks", {"n_required": 5}],
]

workflow = create_workflow(
    yaml_file_name=None,
    query_filters=common_filters,
    reference_filters=common_filters,
    score_computations=[
        ["precursormzmatch", {"tolerance": 10, "tolerance_type": "ppm"}],
        ["filter_by_range", {"name": "PrecursorMzMatch", "low": 0.5}],
        ["modifiedcosinegreedy", {"tolerance": 0.02}],
        [
            "filter_by_range",
            {"name": "ModifiedCosineGreedy_score", "low": 0.6},
        ],
    ],
)

pipeline = Pipeline(
    workflow,
    progress_bar=False,
    logging_level="WARNING",
    logging_file="matchms-pipeline.log",
)
pipeline.run(
    query_files="queries.mgf",
    reference_files="library.msp",
    cleaned_query_file="queries-cleaned.mgf",
    cleaned_reference_file="library-cleaned.msp",
    create_report=False,
)
scores = pipeline.scores
```

Notes:

- `create_workflow()` expects matchms filter callables/names and score class
  names lowercased in `score_computations`.
- The first score should establish the coordinate set when later metrics should
  be candidate-gated.
- `filter_by_range` mutates the retained coordinates.
- `Pipeline.run()` stores final scores on `pipeline.scores`.
- Output and YAML paths must not already exist.
- For all-vs-all scoring, omit `reference_files`; Pipeline then sets symmetric
  mode.

To persist and reload a workflow:

```python
from matchms import Pipeline
from matchms.Pipeline import create_workflow
from matchms.yaml_file_functions import load_workflow_from_yaml_file

create_workflow(
    yaml_file_name="workflow.yaml",
    query_filters=common_filters,
    reference_filters=common_filters,
    score_computations=[["cosinegreedy", {"tolerance": 0.02}]],
)

workflow = load_workflow_from_yaml_file("workflow.yaml")
Pipeline(workflow).run("queries.mgf", "library.msp")
```

Treat YAML as configuration, not as a substitute for recording software and
input versions.

## 6. All-vs-All Comparison

```python
from matchms import calculate_scores
from matchms.similarity import CosineGreedy

scores = calculate_scores(
    references=spectra,
    queries=spectra,
    similarity_function=CosineGreedy(tolerance=0.02),
    array_type="sparse",
    is_symmetric=True,
)
```

`is_symmetric=True` permits implementations to avoid redundant calculation;
Flash 0.33.1 does not exploit this flag for runtime savings. The
logical pair count still grows quadratically. Estimate size before launching:

```python
n = len(spectra)
unique_pairs_with_diagonal = n * (n + 1) // 2
```

For large collections, consider:

- precursor/metadata candidate masks;
- `BlinkCosine` or matrix-oriented `FlashSimilarity`;
- batched query subsets;
- sparse score thresholds; and
- approximate-neighbor indexing with validated recall.

Do not densify a large result with `to_array()` merely to find top hits.

## 7. Fast Matrix Scoring

### Flash entropy

```python
from matchms import calculate_scores
from matchms.similarity import FlashSimilarity

scores = calculate_scores(
    references,
    queries,
    FlashSimilarity(
        score_type="spectral_entropy",
        matching_mode="fragment",
        tolerance=0.02,
        noise_cutoff=0.01,
    ),
)
```

### Fast modified-cosine-like mode

```python
scores = calculate_scores(
    references,
    queries,
    FlashSimilarity(
        score_type="cosine",
        matching_mode="hybrid",
        tolerance=0.02,
    ),
)
```

Flash outputs a scalar `FlashSimilarity` field without a matched-peak count.
Validate ranking agreement against a transparent baseline on representative
data. Flash allocates a dense matrix even when sparse output is requested.

## 8. Mirror-Plot Validation

```python
from pathlib import Path

output = Path("top-hit-mirror.png")
figure, axis = query.plot_against(reference, figsize=(10, 6), dpi=200)
figure.savefig(output, bbox_inches="tight")
```

Inspect:

- unshifted and shifted matched peaks;
- precursor and adduct agreement;
- whether the score is driven by one dominant fragment;
- unexplained high-intensity peaks; and
- collision-energy/acquisition differences.

## 9. Spectral Similarity Network

Networks require an all-vs-all `Scores` object and a unique identifier on each
spectrum:

```python
from matchms import calculate_scores
from matchms.networking import SimilarityNetwork
from matchms.similarity import ModifiedCosineGreedy

scores = calculate_scores(
    spectra,
    spectra,
    ModifiedCosineGreedy(tolerance=0.02),
    array_type="sparse",
    is_symmetric=True,
)

ids = [s.get("spectrum_id") for s in spectra]
assert all(isinstance(value, str) and value for value in ids)
assert len(ids) == len(set(ids))

network = SimilarityNetwork(
    identifier_key="spectrum_id",
    top_n=20,
    max_links=10,
    score_cutoff=0.7,
    link_method="mutual",
    keep_unconnected_nodes=True,
)
network.create_network(
    scores,
    score_name="ModifiedCosineGreedy_score",
)
network.export_to_file("spectral-network.graphml", graph_format="graphml")
```

Supported export formats include GraphML, GEXF, GML, Cytoscape JSON, and JSON.
`top_n` must be at least `max_links`. `max_links` limits outgoing additions per
iteration, not total node degree (incoming links can increase it). Use unique
nonempty string IDs: numeric IDs can become mixed integer/string nodes.
Equal scores near the strict link limit
can make tie selection order-dependent; use deterministic identifiers and
document parameters.

Network components are hypotheses about spectral relatedness, not proof of
shared structure or biosynthetic origin.

## 10. Structure Versus Spectrum Similarity

Use separate labels and outputs:

- spectral similarity compares measured peak patterns;
- fingerprint similarity compares known molecular structures;
- correlation between the two can be analyzed only where reliable structures
  are available.

Do not include a query's true structure in candidate ranking when evaluating a
spectral identification method; that leaks the answer.

## 11. USI-Based Reproducible Pair

```python
from matchms.importing import load_from_usi
from matchms.filtering import default_filters, normalize_intensities
from matchms.similarity import CosineGreedy

reference = load_from_usi(
    "mzspec:GNPS:GNPS-LIBRARY:accession:CCMSLIB00000424840"
)
query = load_from_usi(
    "mzspec:MSV000086109:BD5_dil2x_BD5_01_57213:scan:760"
)
if reference is None or query is None:
    raise ValueError("The USI resolver returned no usable spectrum")
reference = normalize_intensities(default_filters(reference))
query = normalize_intensities(default_filters(query))
result = CosineGreedy(tolerance=0.02).pair(reference, query)
```

Record resolver, retrieval date, and returned metadata. A USI is stable
provenance, but resolver availability and returned annotations can change.

## 12. Provenance Record

At minimum, save:

```python
import json
import platform

import matchms

provenance = {
    "matchms_version": matchms.__version__,
    "python_version": platform.python_version(),
    "query_files": ["queries.mgf"],
    "reference_files": ["library.msp"],
    "metadata_harmonization": True,
    "processing_steps": query_steps,
    "similarity": {
        "class": "ModifiedCosineGreedy",
        "tolerance_da": 0.02,
    },
    "selection": {
        "top_k": 10,
        "minimum_score": 0.6,
        "minimum_matches": 5,
    },
}

with open("provenance.json", "w", encoding="utf-8") as handle:
    json.dump(provenance, handle, indent=2, default=str)
```

Also retain input checksums, library release/date, acquisition information,
code/configuration, and any manual curation decisions.

## Failure Modes

- **No modified-cosine results:** check precursor m/z after harmonization and
  filtering.
- **Unexpected `TypeError` from `SpectrumProcessor`:** call
  `process_spectrum()`/`process_spectra()`; the processor is not callable.
- **Cannot format a score as float:** extract a named field from the structured
  score record.
- **Top-hit value treated as an index:** `scores_by_query()` returns a Spectrum.
- **Huge memory use:** avoid dense conversion; gate candidates and estimate pair
  count.
- **No network edges:** verify identifier fields, score layer name, threshold,
  and that the scores are all-vs-all.
- **Fingerprint warnings:** migrate from `add_fingerprint()` to `Fingerprints`
  and bridge to the current `FingerprintSimilarity` only when required.
- **Output already exists:** matchms intentionally refuses overwrite in generic
  writers and pipeline outputs; choose a new path or replace the intended output
  when the user has authorized it.
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall