Skip to content

K-Dense-AI/scientific-agent-skills / skills/matchms

matchms

Verified

Processes, cleans, compares, and searches tandem mass spectra with matchms. Use for MS/MS file I/O, metadata harmonization, peak filtering, spectral similarity, library matching, score matrices, and molecular-similarity networks. Use pyopenms instead for LC-MS feature detection or proteomics pipelines.

FileKindSize
SKILL.mdskill md12.5 KB
references/filtering.mdmarkdown11.9 KB
references/importing_exporting.mdmarkdown10.6 KB
references/migration.mdmarkdown8.4 KB
references/similarity.mdmarkdown13.8 KB
references/sources.mdmarkdown9.6 KB
references/workflows.mdmarkdown13.9 KB
scripts/library_search.pyscript20.6 KB

references/filtering.md

View on GitHub
# Filtering and Spectrum Processing (matchms 0.33.1)

Use this reference when building or debugging metadata-cleaning, peak-processing,
or quality-control pipelines. The authoritative API is the matchms
[`filtering` package](https://matchms.readthedocs.io/en/latest/api/matchms.filtering.html).

## Filter Contract

Most matchms filters:

- accept one `Spectrum` as the first argument;
- return a `Spectrum` or `None`;
- default to `clone=True`, so direct calls usually return a modified copy;
- use `None` to indicate that a `require_*` condition failed; and
- should be assigned back when called directly.

```python
spectrum = normalize_intensities(spectrum)
spectrum = require_minimum_number_of_peaks(spectrum, n_required=5)
if spectrum is None:
    # The spectrum failed a quality requirement.
    ...
```

Do not continue passing `None` through ordinary filters. `SpectrumProcessor`
stops the chain and discards rejected spectra for you.

## Prefer `SpectrumProcessor`

`SpectrumProcessor` accepts:

- a built-in filter name such as `"normalize_intensities"`;
- a callable such as `normalize_intensities`; or
- `(filter, parameter_dict)`, where `filter` can be a name or callable.

Built-in filters are automatically sorted into matchms's required filter order.
Custom filters are appended unless an explicit position is supplied with
`parse_and_add_filter()`.

```python
from matchms import SpectrumProcessor
from matchms.filtering import (
    default_filters,
    normalize_intensities,
    remove_peaks_relative_to_precursor_mz,
    require_minimum_number_of_peaks,
    require_precursor_mz,
    select_by_mz,
    select_by_relative_intensity,
)

# Run the aggregate metadata harmonizer before SpectrumProcessor. If included as
# one callable inside the processor, it is treated as custom and placed last.
spectra = [default_filters(spectrum) for spectrum in spectra]

processor = SpectrumProcessor(
    [
        (require_precursor_mz, {"minimum_accepted_mz": 50.0, "maximum_mz": 1500.0}),
        normalize_intensities,
        (remove_peaks_relative_to_precursor_mz, {"offset_to_precursor": -1.6}),
        (select_by_mz, {"mz_from": 20.0, "mz_to": 1500.0}),
        (select_by_relative_intensity, {"intensity_from": 0.01}),
        (require_minimum_number_of_peaks, {"n_required": 5}),
    ]
)

cleaned, report = processor.process_spectra(
    spectra,
    progress_bar=False,
    create_report=False,
)
print(processor.processing_steps)
```

Important current behavior:

- `SpectrumProcessor` is **not callable**.
- Use `processor.process_spectrum(spectrum.clone())` to preserve one input.
  Without a processing report, `process_spectrum` passes `clone=False` to
  filters and can mutate its argument; it does not clone on entry.
- Use `processor.process_spectra(spectra)` for a list.
- `process_spectra()` returns `(processed_spectra, processing_report)`.
- With `create_report=False`, the processor clones each input once and disables
  per-filter cloning where supported.
- With `create_report=True`, filters are cloned step by step to measure changes.
  The aggregate `default_filters` function is not in the processor's built-in
  order registry and does not expose a `clone` parameter. If passed directly,
  it is placed after registered filters and detailed reporting logs a warning.
  Run it before the processor or expand it into the nine filters below.
- `process_spectrums()` is a deprecated spelling.

## What `default_filters()` Does

In 0.33.1, `default_filters()` applies exactly:

1. `make_charge_int`
2. `add_compound_name`
3. `derive_adduct_from_name`
4. `derive_formula_from_name`
5. `clean_compound_name`
6. `interpret_pepmass`
7. `add_precursor_mz`
8. `derive_ionmode`
9. `correct_charge`

It does **not** normalize peaks, add retention fields, harmonize structural
identifiers, require precursor metadata, or enforce peak-count quality. Add
those steps explicitly.

## Peak Processing

### Normalize and select peaks

- `normalize_intensities(spectrum_in, clone=True, scaling=None)` — normalize to
  maximum intensity 1 by default; pass `scaling=(low, high)` for min-max scaling.
- `select_by_intensity(spectrum_in, intensity_from=10.0, intensity_to=200.0,
  clone=True)` — keep an absolute intensity interval.
- `select_by_relative_intensity(spectrum_in, intensity_from=0.0,
  intensity_to=1.0, clone=True)` — keep a fraction-of-maximum interval.
- `select_by_mz(spectrum_in, mz_from=0.0, mz_to=1000.0, clone=True)` — crop the
  fragment m/z interval.

`select_by_relative_intensity` divides by the current maximum internally; prior
normalization is optional, and retained intensities keep their original scale.
Cropping away a base peak before this filter can change the retained peaks.

### Reduce and clean peaks

- `reduce_to_number_of_peaks(spectrum_in, n_required=0, n_max=inf,
  ratio_desired=None, clone=True)` — keep the most intense peaks within count
  constraints.
- `remove_noise_below_frequent_intensities(spectrum_in,
  min_count_of_frequent_intensities=5, noise_level_multiplier=2.0, clone=True)`
  — estimate and remove repeated low-intensity noise.
- `remove_peaks_around_precursor_mz(spectrum_in, mz_tolerance=17, clone=True)`
  — remove peaks within a precursor-centered exclusion window.
- `remove_peaks_relative_to_precursor_mz(spectrum_in,
  offset_to_precursor=-1.6, clone=True)` — remove peaks above a cutoff relative
  to precursor m/z.
- `remove_peaks_outside_top_k(spectrum_in, k=6, mz_window=50, clone=True)` —
  retain peaks within `mz_window` Da of one of the globally `k` most intense
  peaks; this is not a per-window top-k filter.
- `remove_profiled_spectra(spectrum_in, mz_window=0.5, clone=True)` — reject
  spectra likely to contain profile-mode rather than centroided data.

These defaults are starting points, not instrument-independent truth. Record
chosen windows and thresholds.

### Peak quality requirements

- `require_minimum_number_of_peaks(spectrum_in, n_required=10,
  ratio_required=None, clone=True)`
- `require_maximum_number_of_peaks(spectrum_in,
  maximum_number_of_fragments=1000, clone=True)`
- `require_minimum_number_of_high_peaks(spectrum_in, no_peaks=5,
  intensity_percent=2.0, clone=True)`

Older examples commonly use invalid
`require_minimum_number_of_high_peaks(n_required=..., intensity_threshold=...)`
arguments. In 0.33.1 the names are `no_peaks` and `intensity_percent`, where the
latter is a percentage rather than a 0-1 fraction.

## Precursor, Adduct, Charge, and Ion Mode

- `interpret_pepmass(spectrum_in, clone=True)`
- `add_precursor_mz(spectrum_in, clone=True)`
- `add_parent_mass(spectrum_in, estimate_from_adduct=True,
  overwrite_existing_entry=False, estimate_from_charge=True, clone=True)`
- `add_precursor_formula(spectrum_in, clone=True)`
- `make_charge_int(spectrum_in, clone=True)`
- `correct_charge(spectrum_in, clone=True)`
- `clean_adduct(spectrum_in, clone=True)`
- `derive_adduct_from_name(spectrum_in, remove_adduct_from_name=True,
  clone=True)`
- `derive_ionmode(spectrum_in, clone=True)`
- `require_correct_ionmode(spectrum_in, ion_mode_to_keep)`
- `require_matching_adduct_and_ionmode(spectrum)`
- `require_matching_adduct_precursor_mz_parent_mass(spectrum, tolerance=0.1)`
- `require_precursor_mz(spectrum_in, minimum_accepted_mz=10.0,
  maximum_mz=None, clone=True)`

`require_precursor_below_mz()` is deprecated. Use
`require_precursor_mz(maximum_mz=...)`.

Modified-cosine and neutral-loss scoring require a valid `precursor_mz`.
Parent-mass matching additionally requires a valid `parent_mass`. Check that
masses are finite and positive: `require_precursor_mz` uses inclusive bounds
and does not itself reject NaN or infinity. The bundled CLI adds that check.

## Compound Names, Formulae, and Structures

### Name and formula processing

- `add_compound_name(spectrum_in, clone=True)`
- `clean_compound_name(spectrum_in, clone=True)`
- `derive_formula_from_name(spectrum_in, remove_formula_from_name=True,
  clone=True)`
- `derive_formula_from_smiles(spectrum_in, overwrite=True, clone=True)`
- `require_compound_name(spectrum)`
- `require_formula(spectrum)`

### SMILES, InChI, and InChIKey

- `derive_inchi_from_smiles(spectrum_in, clone=True)`
- `derive_inchikey_from_inchi(spectrum_in, clone=True)`
- `derive_smiles_from_inchi(spectrum_in, clone=True)`
- `harmonize_undefined_inchi(...)`
- `harmonize_undefined_inchikey(...)`
- `harmonize_undefined_smiles(...)`
- `repair_inchi_inchikey_smiles(spectrum_in, clone=True)`
- `repair_not_matching_annotation(spectrum_in, clone=True)`
- `require_valid_annotation(spectrum)`

`derive_annotation_from_compound_name(spectrum_in,
annotated_compound_names_file=None, mass_tolerance=0.1, clone=True)` uses
PubChemPy name search when no valid SMILES/InChI exists. It requires a plausible
compound name and `parent_mass`, selecting an annotation by neutral mass
agreement. The tolerance is an absolute mass difference, not ppm. It can read
and append its optional CSV cache and can retry network failures for a long
time. Run deliberately, retain original annotations, and preserve the cache
with provenance. Name/mass agreement does not resolve isomers or validate the
query identity. No credential is required for the public PubChem service.

### Structure/mass repair

Current repair helpers include:

- `repair_adduct_and_parent_mass_based_on_smiles`
- `repair_adduct_based_on_parent_mass`
- `repair_parent_mass_from_smiles`
- `repair_parent_mass_is_molar_mass`
- `repair_parent_mass_match_smiles_wrapper`
- `repair_smiles_of_salts`
- `require_parent_mass_match_smiles`

Several require an explicit `mass_tolerance`. Do not silently "repair" library
annotations without preserving original fields and logging the rule used.

## Retention and MS-Level Metadata

- `add_retention_time(spectrum_in, clone=True)`
- `add_retention_index(spectrum_in, clone=True)`
- `require_retention_time(spectrum_in, minimum_rt=None, maximum_rt=None,
  clone=True)`
- `require_retention_index(spectrum_in, clone=True)`
- `require_correct_ms_level(spectrum, required_ms_level=2)`

Retention time and retention index are not interchangeable. Record units and
the chromatographic method before using either as a matching constraint.

## Fingerprints

`add_fingerprint()` still works in 0.33.1, but it is marked for removal in
matchms 1.0. Prefer the top-level `Fingerprints` class:

```python
from matchms import Fingerprints

fingerprints = Fingerprints(
    fingerprint_algorithm="morgan2",
    fingerprint_method="bit",
    nbits=2048,
)
fingerprints.compute_fingerprints(spectra)
```

`Fingerprints` maps valid InChIKeys to fingerprints and needs a valid InChIKey
plus SMILES or InChI. In 0.33.1, `FingerprintSimilarity` still reads a
`"fingerprint"` field from each spectrum, so bridge deliberately when that
legacy similarity class is needed:

```python
for spectrum in spectra:
    fingerprint = fingerprints.get_fingerprint_by_spectrum(spectrum)
    if fingerprint is not None:
        spectrum.set("fingerprint", fingerprint)
```

Do not describe fingerprint similarity as spectral similarity.

## Custom Filters

A custom filter should accept a spectrum first and return a spectrum or `None`:

```python
def require_fragment(spectrum_in, fragment_mz, tolerance=0.02):
    if spectrum_in is None:
        return None
    if any(abs(mz - fragment_mz) <= tolerance for mz in spectrum_in.peaks.mz):
        return spectrum_in
    return None


processor.parse_and_add_filter(
    (require_fragment, {"fragment_mz": 184.0733, "tolerance": 0.01})
)
```

Avoid mutation unless it is intentional and documented. If the custom filter
modifies a spectrum, either implement a `clone` parameter consistently or clone
inside the function.

## Reproducibility Checklist

- Save `processor.processing_steps`.
- Record matchms and Python versions.
- Record whether metadata harmonization was enabled at import.
- Apply identical peak filters to query and reference collections.
- Preserve counts before and after every requirement filter.
- Preserve original metadata before repair/enrichment.
- Record network calls and external annotation sources.
- Test a few spectra with expected pass/fail behavior before batch processing.
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall