Tests & Configuration

01How tests are selected

For every PR, CodeMole resolves the checks in three stages — the first one that applies wins:

1.  .codemole.yml in the repo?   → use exactly that   (versioned, per branch/PR)
2.  otherwise: repo analysis     → detect the profile automatically
3.  always, for everyone         → secret-scan + conflict-markers + sensitive-files + ai-review

The report header transparently shows what was selected:

## 🧪 CodeMole — automations.yaml
Profile: ha-config · auto-detected · ⚙ overridable

02Auto-detection

Without a .codemole.yml, CodeMole determines the profile via marker files in the repo — deterministic, no guessing:

03Profiles

A profile is just a named bundle of checks. universal always runs on top.

04Check catalog

Each check is a self-contained building block. Quick overview below — the detailed check reference explains every check with an example finding and fix (the check names in PR reports link straight to it):

05Configuration: .codemole.yml

Put the file in the repo root (or under .github/). It is optional — without it, auto-detection kicks in. Complete example with all options:

# force the language of the bot output (otherwise auto-detected from the PR title/text)
lang: en                  # de | en

# force a profile (overrides auto-detection)
profile: ha-config          # ha-config | ha-component | aem-eds | generic

# OR select checks explicitly (instead of profile):
# checks: [yamllint, secret-scan, ai-review]

disable: [diff-size]        # turn off individual checks

ignore:                   # exclude paths globally (glob)
  - "**/*.generated.yaml"
  - "vendor/**"

# per-check options:
yamllint:
  changed-lines-only: true
ai-review:
  focus: "native HA-Trigger statt Jinja, entity_id statt device_id"
  severity: major     # only post major findings (major | minor)

# enable page-audit (frontend repos with branch previews; {branch} is substituted):
page-audit:
  base_url: "https://{branch}--mein-site--meine-org.aem.page"
  pages: ["/de/de/", "/de/de/products/"]   # max. 5 pages; also applies to Lighthouse (label)

🔐 Storing secrets safely (e.g. HA token for entity-exists)

Some checks need a secret — e.g. entity-exists checks entity IDs live against your Home Assistant instance and needs a long-lived access token for that (HA: Profile → Security → create token). Secrets must never go into the repo in plaintext. So: encrypt them in the browser here (nothing leaves your browser — WebCrypto against the CodeMole public key) and put the blob into the .codemole.yml. Only the CodeMole server can decrypt it.

entity-exists:
  ha_url: "http://homeassistant.local:8123"
  ha_token: "enc:v1:<paste the blob from the tool here>"

Prefer a local script? (maximum assurance, no browser)

To be completely certain, encrypt offline on your own machine — just openssl + the embedded public key, no network:

# download the script (plain text — inspect it first!) and run it:
curl -O https://web.skycryer.com/codemole/encrypt-secret.sh
bash encrypt-secret.sh          # prompts for the secret (hidden), prints enc:v1:…

The script is intentionally short and readable: it only calls openssl pkeyutl -encrypt with the public key and makes zero network calls — you can review it fully before running. Result is identical to the browser tool. View the script →

06Overriding & disabling

A check flags nonsense? — fix it yourself

• Findings on generated/vendor files (build bundles, fonts, fixtures) → that belongs in your tool's ignore (.stylelintignore, .eslintignore). CodeMole respects it and never overrides it — nothing to do.
• A lint rule is too strict for you (e.g. at-rule-empty-line-before) → turn it off in your tool config (.stylelintrc, .eslintrc). CodeMole uses your config, not its own.
• The check is simply irrelevant here → disable: [check-name].
• Only certain paths are noisy → ignore: ["path/**"].
• One-off exception (aem-eds) → testing-rules.json in the repo.

Force a profile

Detection got it wrong? Set profile: — then only that counts.

Disable individual checks

disable: [diff-size, visual] removes checks from the resolved set (the profile otherwise stays the same).

Ignore paths

ignore: with glob patterns — generated files, vendor code, fixtures. Findings on these paths are suppressed.

Set the language

By default CodeMole detects the language automatically (German PR → German output, English PR → English). With lang: de or lang: en in the .codemole.yml you force a fixed language for all bot output (report, findings, replies) — useful for very short or mixed-language PR texts.

How auto-detection works: it looks at the PR title + description (not the code diff — code is always English). A fast, dependency-free heuristic assigns scores:

de_score = German stop words (der/die/und/nicht/für…) + umlauts/ß × 3
en_score = English stop words (the/and/with/for/this…)
→ "en" if en_score > de_score, else "de"

Umlauts/ß count triple because a single „ä/ö/ü/ß" is a very strong German signal. On a tie, empty text, or error it falls back to de (fail-safe). The heuristic is deliberately simple — for short or mixed texts, prefer setting lang:.

Pause completely

checks: [] (empty) turns everything off — or simply remove the repo from the app installation.

07Onboarding a new repo

For a repo in your own organization, onboarding is one click — no workflow, no webhook, no token:

1.  Install the app → tick the repo   → done, runs on the next PR
2.  optional: .codemole.yml in the repo    → fine-tune profile/checks (section 05)

The profile is detected automatically (section 02). Without matching markers, generic runs (diff-size + universal checks + AI review) — even that is already useful.

When manual work is needed after all

08Talking to the bot

CodeMole is not a one-way reporter — it responds to you, right in the PR:

⚡ Using Lighthouse properly (best practice)

Lighthouse is the heavy run (~1–2 min per page, two-pass, max. 2 pages) and therefore never runs automatically — only when you set the lighthouse label. It runs entirely on our own infrastructure (no Google/PageSpeed — no API limits).

When to set the label?
→ once, when the PR is performance-relevant:
   CSS/JS rework, image handling, new blocks, lazy loading, fonts

Recommended flow
1.  finish developing the PR (regular checks run on every push anyway)
2.  before review/merge: set the lighthouse label → wait for the score table
3.  regression? fix & push — then remove + re-add the label for the re-run

Good to know
·  new pushes do NOT re-trigger Lighthouse (deliberate — only the label counts)
·  the result comment is updated, not duplicated
·  scores fluctuate by ±2–3 points due to the CDN — only react to larger deltas

09Setting up visual tests

Visual regression means: CodeMole takes a screenshot of a block and compares it pixel by pixel against a stored baseline image. If anything deviates, you get a finding. Technically this is Playwright (@playwright/test) — already set up in the JUMO repo. You only add a small test file + a reference image per block. Step by step (even if you have never done this before):

1 · Create a spec — two layouts (both equally valid)

CodeMole supports two spec layouts on equal footing — pick whichever the repo already uses for the block in question. In both, the baselines live next to the spec in <name>.spec.js-snapshots/.

A · Styleguide / nested — tests/visual-styleguide/<block>/

Folder = block name, the spec is called ex-N.spec.js. A one-liner via a helper, no boilerplate — the helper renders the block demo from the pattern styleguide (/patterns/styleguide/#demo/<block>/<index>/…) and screenshots it:

// tests/visual-styleguide/contact-overlay-role/ex-0.spec.js
import { patternScreenshot } from '../_helpers/pattern-screenshot.js';

// (block id, example index in the styleguide, snapshot name)
patternScreenshot('contact-overlay-role', 0, 'ex-0');

Multiple variants of a block → ex-0, ex-1, … (one call or file per example index).

B · Component page / flat — tests/visual/<block>.spec.js

Filename = block name, a full Playwright spec: opens the real demo page and screenshots the block wrapper. The URL is project-specific (at JUMO it's the component library):

import { test, expect } from '@playwright/test';
import { getBaseUrl } from './helpers/test-setup.js';

// project-specific demo page; ?wcmmode=disabled hides the AEM author UI
const TEST_URL = `${getBaseUrl()}/de/de/components/base/teaser-xl/columns-blockquote`;

test.beforeEach(async ({ page }) => {
  await page.goto(`${TEST_URL}?wcmmode=disabled`);
  await page.waitForLoadState('networkidle');   // (+ dismiss the cookie banner)
});

test('blockquote should match reference', async ({ page }) => {
  const block = page.locator('.blockquote-wrapper').first();
  await expect(block).toHaveScreenshot('blockquote.png', { maxDiffPixels: 500 });
});

2 · Generate the baseline image (once per spec)

The first time around there is no comparison image yet — generate it once and commit it (the npm scripts are already in the repo, applies to both layouts):

# run the comparison:        npm run test:visual
# write/update baselines: npm run test:visual:update
git add tests/**/*-snapshots/   # commit the generated .png baselines too

Without a committed baseline, CodeMole has nothing to compare against.

3 · What CodeMole then does automatically

• PR changes files of a block → CodeMole finds its spec (nested: folder name = block · flat: filename = block) and runs it against the preview hosts (BASE_URL_DEV / BASE_URL_BRANCH).
• Screenshot deviates from the baseline → finding with a diff image. Everything matches → green.
• Atoms (button, text, image, link) automatically trigger the specs of all organisms that embed them, via block-deps.json.
• No spec for a changed block? → CodeMole warns: "No visual spec for: <block>". Then create one (layout A or B).