Skill: /signal-over-noise

A two-phase Python observability audit that improves the ratio of meaningful signal to noise in your error handling and logging. Each phase analyzes the codebase, presents a remediation plan for your review, and — after approval — executes the fixes.

• --mode surface (default): uncovers suppressed exceptions that make failures invisible
• --mode tune: fixes logging anti-patterns that pollute or obscure your log stream

Invocation

/signal-over-noise                                      # surface audit, current directory
/signal-over-noise packages/app-framework/src           # surface audit, specific path
/signal-over-noise --mode surface                       # same as default
/signal-over-noise --mode tune                          # logging quality audit
/signal-over-noise --mode tune --severity high          # HIGH + CRITICAL findings only
/signal-over-noise --mode surface packages/foo/src      # surface audit, specific path

Argument parsing:

• First positional token (not a flag): target path. Default: .
• --mode surface|tune: which phase to run. Default: surface
• --severity <level>: filter findings; levels: critical, high, medium, all. Default: all

---

How each mode works

Both modes follow the same two-stage structure:

1. Analysis — grep the codebase, read context, classify findings, build a remediation plan. Enter plan mode and present the plan for user review. The user can iterate on the plan.

2. Execution — after the user approves the plan, apply the agreed fixes. Run pre-commit on changed files. Leave all changes uncommitted for the user to review before committing.

---

Mode: surface

Finds every pattern where exceptions are caught and silently discarded, converting failures into invisible state.

Detection is now delegated to the conformance suite (E-series). The E001–E018 checks run deterministically via uv run atlan-application-sdk-conformance detect --repo . --series E and produce SARIF output with one finding per rule violation. This skill retains its value for the remediation / fix-authoring pass (Stage 2) — the suite deliberately does not apply fixes. For a quick detection-only scan, prefer the suite runner; use this skill when you want a plan-mode review and automated fix application.

Pattern catalogue, severity criteria, fix templates, and linting rules are in:

.claude/skills/signal-over-noise/references/error-recovery-patterns.md

---

Stage 1 — Analysis

Step 1.1 — Setup

1. Read .claude/skills/signal-over-noise/references/error-recovery-patterns.md in full.
2. Read .claude/skills/signal-over-noise/references/typed-error-prescription.md in full.
3. Resolve TARGET_PATH from arguments (default: current directory). Confirm it exists.
4. Set SEVERITY_FILTER from --severity (default: all).

Step 1.2 — Pattern Detection

Run all 13 grep searches in parallel against TARGET_PATH (*.py files only):

#	Pattern ID	Grep
1	P1	except\s*:\s*pass
2	P2	except\s+\w.*:\s*pass
3	P3	contextlib\.suppress\(
4	P4	except\s+(Exception|BaseException)\s*:
5	P5	logger\.(warning|error|critical)\( inside except blocks — collect hits for Step 1.3
6	P6	^\s*except\s*:
7	P7	except block with return and no log — collect hits for Step 1.3
8	P8	except\s+ImportError
9	P9	except block with only an assignment — collect hits for Step 1.3
10	P10	return_exceptions=True
11	P11	class\s+\w+.*logging\.Filter or def filter\(self inside a Filter subclass
12	P12	raise\s+(ValueError|RuntimeError|Exception|TypeError|NotImplementedError|OSError|KeyError|LookupError)\b
13	P13	raise\s+(ClientError|ApiError|OrchestratorError|WorkflowError|IOError|CommonError|DocGenError|ActivityError|AtlanError)\b

For P5, P7, P9, P11, P12, P13: raw grep over-matches. Collect file:line hits and read context in Step 1.3. P13 context note: IOError is also a Python builtin alias for OSError — confirm the hit imports from application_sdk.common.error_codes before classifying as legacy. Bare AtlanError raises (no constant) require litmus-test classification (§3 of typed-error-prescription.md).

Collect all hits as a flat list: (file, line_number, pattern_id, raw_snippet).

Step 1.3 — Context Reading and Classification

For each hit, read 10–15 lines of surrounding context.

Classify each finding:

• Severity: CRITICAL, HIGH, MEDIUM, or LOW (from §Severity Classification in reference)
• Legitimacy: genuine-bug or acceptable (from §Legitimacy in reference)
• Category: silent-swallow, missing-traceback, overly-broad-catch, error-to-return-value, suppress-operational, optional-import, asyncio-unexamined, untyped-raise, legacy-raise
• Typed-error prescription: for every genuine-bug finding whose fix involves a raise (FT-1b, FT-3b, FT-5b, FT-8, FT-9), look up the prescribed leaf and code from references/typed-error-prescription.md. Record as Prescribed: on the remediation entry so the user sees the target class in the plan.

Apply SEVERITY_FILTER: discard findings below the threshold before proceeding.

P12 classification notes:

• Inside __post_init__, stdlib dataclass validators, or Pydantic @field_validator / @validator methods: check for a comment explaining the stdlib-interop need. If present → acceptable. If absent → flag as genuine-bug (MEDIUM — add the comment).
• Inside an @task-decorated activity body or @activity.defn-decorated function → escalate to CRITICAL.

P13: confirmed hits are genuine-bug (HIGH). After context reading (see over-matches note above), use the §5 migration table from typed-error-prescription.md for the Prescribed: field. When no matching legacy constant is present, use §3 litmus tests to select a leaf directly.

Step 1.4 — Build Remediation Plan

For each genuine-bug finding, produce a remediation entry. Include Prescribed: whenever the fix involves raising an exception:

File:     application_sdk/clients/sql.py:352
Severity: HIGH
Category: untyped-raise
Pattern:  P12
Current:
    raise ValueError("Engine is not initialized. Call load() first.")
Prescribed:
    EngineNotInitializedError(InternalError)  →  code=INTERNAL_ENGINE_NOT_INITIALIZED
    Subclass bakes message + component + invariant. Place in a sibling
    module (e.g. _<area>_errors.py); see typed-error-prescription.md §4.
Fix (FT-8):
    raise EngineNotInitializedError()
Auto-fixable: no  (leaf + subclass choice needs context — manual review)
Priority: P1

For existing-pattern (P1–P11) findings whose fix re-raises, also include Prescribed: with the leaf selected from typed-error-prescription.md §4.

Priority mapping:

• P0 — CRITICAL: fix immediately, block merges
• P1 — HIGH: fix in current sprint
• P2 — MEDIUM/LOW: fix when touching the file

Step 1.5 — Enter Plan Mode

Call EnterPlanMode.

Present the Surface Report as the plan:

# Surface Report — <TARGET_PATH>

## Executive Summary
- Files scanned: N
- Total findings: N (N genuine bugs, N acceptable patterns)
- By severity: CRITICAL: N | HIGH: N | MEDIUM: N | LOW: N
- By category: silent-swallow: N | missing-traceback: N | ...
- Auto-fixable: N | Needs manual review: N

## Findings
| # | File | Line | Severity | Category | Pattern | Auto-fixable | Notes |
|---|------|------|----------|----------|---------|--------------|-------|

## Remediation Plan

### P0 — Fix Immediately (CRITICAL)
[entries]

### P1 — Fix This Sprint (HIGH)
[entries]

### P2 — Fix When Touching (MEDIUM/LOW)
[entries]

## Acceptable Patterns (no action required)
| File | Line | Pattern | Justification |
|------|------|---------|---------------|

Call ExitPlanMode to request user approval. Wait for approval before proceeding to Stage 2.

---

Stage 2 — Execution (after plan approval)

Apply the remediation plan that was approved. Only fix what was listed in the approved plan.

Auto-fixable patterns — apply directly:

• FT-1a: Add logging to silent-swallow (log-and-continue, best-effort paths)
• FT-2: Add exc_info=True to existing log call in except block
• FT-3a: Add logging before error-to-return-value (when caller treats default as valid)
• FT-4: Add exception inspection after asyncio.gather(..., return_exceptions=True)
• FT-5a: Replace broad contextlib.suppress(Exception) with logged try/except (best-effort)

Manual review required — leave TODO comments:

• FT-1b (log + typed re-raise): leaf choice is contextual
• FT-3b (convert swallow to typed raise): default for error-to-return-value when caller trusts result
• FT-5b (typed re-raise for non-best-effort suppress): leaf choice is contextual
• FT-6 (narrow broad catch): # TODO(signal-over-noise): [P4] narrow this catch
• FT-7 (Filter exception safety): # TODO(signal-over-noise): [P11] wrap filter body in try/except
• FT-8 (convert untyped builtin raise to typed AppError): leaf from typed-error-prescription.md §4. Define a subclass for each distinct failure mode in a sibling errors module (SDK: application_sdk/<area>/errors.py or _<area>_errors.py; app: app/failures.py). Bake message and evidence-field defaults when they're stable across raise sites; for one-off sites, the subclass may just override code and let the raise site pass message.
• FT-9 (convert legacy AtlanError raise to typed AppError): leaf from typed-error-prescription.md §5. Same subclassing rule applies when the same error recurs.

TODO comment format for swallow/logging findings:

# TODO(signal-over-noise): [P2] silent swallow — add logging. See references/error-recovery-patterns.md#P2

TODO comment format for untyped/legacy raise findings — include the prescribed leaf:

# TODO(signal-over-noise): [P12] convert to typed AppError. Leaf: InternalError (wire code: INTERNAL_ENGINE_NOT_INITIALIZED). See typed-error-prescription.md §4
# TODO(signal-over-noise): [P13] legacy AtlanError — migrate to DependencyUnavailableError. See typed-error-prescription.md §5

After applying fixes:

Run pre-commit on every file that was changed:

uv run pre-commit run --files <file1> <file2> ...

If pre-commit is not available, skip this step (do not fail).

Leave all changes uncommitted — the user reviews before committing.

Finish

Report a brief summary: N files changed, N auto-fixes applied, N TODO comments added.

Then recommend the next step:

Next: Run /signal-over-noise --mode tune to audit logging quality (f-strings, missing exc_info, log levels, credential leaks, and more).

---

Mode: tune

Finds logging anti-patterns that pollute your log stream, obscure structured fields, or create security risks.

Detection is now delegated to the conformance suite (L-series). The L001–L021 checks run deterministically via uv run atlan-application-sdk-conformance detect --repo . --series L and produce SARIF output with one finding per rule violation. This skill retains its value for the remediation / fix-authoring pass (Stage 2) — the suite deliberately does not apply fixes. For a quick detection-only scan, prefer the suite runner; use this skill when you want a plan-mode review and automated fix application.

Skill ↔ Suite ID crosswalk

The suite renumbers the skill's L-ids. When reading SARIF output and mapping back to the fix templates below, use this table:

Skill ID	Suite ID	Rule name
L1	L001	FStringInLogMessage
L2	L002	InconsistentLoggerFactory
L3	L003	ExtraKwargInNonStdlibCall
L4	L004	MissingExcInfo
L5	L005	PrintInProductionCode
L6	L006	InfoInLoop
L7	L007	CriticalCall
L8	L008	UnguardedExpensiveDebug
L9	L009	WarnThenRaise
L10	L010	CredentialInLog
L11	L011	StringConcatInLogMessage
L13	L012	StdlibExtraReservedKeyCollision
L14	L013	StdlibArbitraryKwargs
L15	L014	StructlogEventKwargOverwrite
L16	L015	DictConfigDisableExistingLoggers
L17	L016	BasicConfigNoopAfterFirstCall
L18	L017	LoggerExceptionUsage
L24	L018	KwargsInApplicationLogCalls
L19	L019	DiscardedBindResult
L23	L020	DeprecatedLoggingWarn
L12, L20, L21, L22	—	not ported (%-style, propagate=False, loguru.remove, loguru kwargs)

Pattern catalogue, severity criteria, fix templates, and linting rules are in:

• .claude/skills/signal-over-noise/references/logging-patterns.md — detectable patterns (L1–L23)
• .claude/skills/signal-over-noise/references/logging-level-guidelines.md — level philosophy and framework awareness notes

---

Stage 1 — Analysis

Step 2.1 — Setup and Discovery

1. Read both reference files listed above in full.
2. Resolve TARGET_PATH from arguments (default: current directory). Confirm it exists.
3. Set SEVERITY_FILTER from --severity (default: all).

Discover the project's logging framework — do not assume any specific one:

Grep for common logger factory patterns and count occurrences:

• structlog\.get_logger\(
• logging\.getLogger\(
• loguru (import or usage)
• get_logger\( (custom wrapper)

The factory with the highest count is CANONICAL_FACTORY.

Check pyproject.toml and requirements.txt / uv.lock for logging framework dependencies:

• structlog in deps → LOGGER_FRAMEWORK = structlog
• loguru in deps → LOGGER_FRAMEWORK = loguru
• Neither → LOGGER_FRAMEWORK = stdlib

Determine SUPPORTS_KWARGS:

• structlog or loguru: SUPPORTS_KWARGS = true
• stdlib: SUPPORTS_KWARGS = false

Determine FRAMEWORK_SPECIFIC_PATTERNS — which additional patterns (L13–L23) to run:

• stdlib: L13, L14, L16, L17, L20, L23
• structlog: L15, L19
• loguru: L19, L21, L22
• All frameworks: L18

Discover logger variable names — grep for assignments:

logger\s*=\s*
log\s*=\s*
_logger\s*=\s*
_log\s*=\s*

Collect distinct prefixes (e.g. logger, log, _logger). Build LOGGER_VAR_PATTERN by joining them: (logger|log|_logger|_log)\. — use this in all Step 2.2 greps.

Step 2.2 — Pattern Detection

Run the universal patterns (L1–L12) plus FRAMEWORK_SPECIFIC_PATTERNS (L13–L23) in parallel against TARGET_PATH. Use LOGGER_VAR_PATTERN in place of hardcoded logger\. in each pattern.

Universal patterns (all frameworks):

Pattern	Grep	Type
L1 f-string	LOGGER_VAR_PATTERN\.\w+\(f["']	mechanical
L2 inconsistent factory	Files not using CANONICAL_FACTORY	mechanical
L3 extra={}	extra=\{ near a logger call	heuristic
L4 missing exc_info	multiline: except + log call without exc_info	heuristic
L5 print()	^\s*print\( (exclude test files, CLI scripts)	mechanical
L6 INFO in loop	LOGGER_VAR_PATTERN\.info\( inside for/while	heuristic
L7 critical()	LOGGER_VAR_PATTERN\.critical\(	mechanical
L8 expensive debug	LOGGER_VAR_PATTERN\.debug\(.*(?:json\.dumps|repr|\.to_dict|\.model_dump)	heuristic
L9 warn-then-raise	multiline: log call within 3 lines before raise	heuristic
L10 credential in log	LOGGER_VAR_PATTERN\.\w+\(.*(?:password|secret|token|api_key|credential|bearer|private_key) (case-insensitive)	heuristic
L11 concatenation	LOGGER_VAR_PATTERN\.\w+\(["'].*["']\s*\+	mechanical
L12 %-style (non-stdlib only)	LOGGER_VAR_PATTERN\.\w+\(["'].*%[sdfr] — skip if LOGGER_FRAMEWORK == stdlib	heuristic
L18 exception() outside except	LOGGER_VAR_PATTERN\.exception\(	heuristic
L24 kwargs in application log calls	LOGGER_VAR_PATTERN\.\w+\([^)]*,\s*\w+= (non-stdlib only) — collect hits for Step 2.3	heuristic

Framework-specific patterns (run only for the indicated framework):

Pattern	Grep	Framework	Type
L13 extra reserved key	extra=\{ — check keys against 22 reserved list	stdlib	heuristic
L14 arbitrary kwargs	LOGGER_VAR_PATTERN\.\w+\([^)]*,\s*\w+=	stdlib	heuristic
L15 event= kwarg	LOGGER_VAR_PATTERN\.\w+\(.*event=	structlog	mechanical
L16 dictConfig disable_existing	dictConfig\(	stdlib	heuristic
L17 basicConfig no-op	basicConfig\( (collect all across codebase)	stdlib	heuristic
L19 bind() discarded	^\s+\w+\.bind\( as bare statement	structlog/loguru	heuristic
L20 propagate=False	propagate\s*=\s*False	stdlib	heuristic
L21 logger.remove()	logger\.remove\(\s*\) (no args)	loguru	mechanical
L22 loguru kwargs as format	LOGGER_VAR_PATTERN\.\w+\("[^"]*"[^)]*,\s*\w+=	loguru	heuristic
L23 warn() deprecated	\.warn\(	stdlib	mechanical

For heuristic patterns: collect file:line hits for Step 2.3 context reading.

For mechanical patterns (L1, L2, L5, L7, L11, L15, L21, L23): record directly as findings.

Step 2.3 — Context Reading and Classification

For each heuristic hit, read 10–15 lines of surrounding context.

Classify each hit:

• Severity: apply the severity from the pattern catalogue, adjusted by context
• Legitimacy: genuine finding or acceptable pattern (see §Legitimacy in logging-patterns.md)

Universal framework adjustments:

• L3: if LOGGER_FRAMEWORK == stdlib → ACCEPTABLE. If structlog/loguru → MEDIUM. Fix direction: %-style message body, not "unwrap to kwargs".
• L12: if LOGGER_FRAMEWORK == stdlib → ACCEPTABLE. If the log call goes through CANONICAL_FACTORY (e.g. get_logger()) AND that factory has a %-style bridge (look for _format_printf_args or similar in the adapter) → ACCEPTABLE. If the log call uses a direct loguru import (from loguru import logger / loguru.logger) even when a bridge adapter exists → HIGH (the bridge is bypassed; positional args are silently dropped because loguru uses str.format() not %). If vanilla loguru (no bridge) → HIGH. If vanilla structlog → MEDIUM.
• L10: read context — token_name=name is acceptable; token=value is CRITICAL.
• L6: if loop is clearly bounded to ≤10 items → ACCEPTABLE.
• L18: read 5 lines before the .exception() call — if inside an except block → ACCEPTABLE.
• L24: skip exc_info=True/False. Skip files that define the logging factory/adapter (the file containing get_logger or the logger adapter class). Flag all other kwargs as MEDIUM.

Framework-specific classification rules:

• L13: Read the extra={} dict literal. Flag CRITICAL for each key that matches any of the 22 reserved LogRecord attributes. Keys not in the reserved list → ACCEPTABLE.
• L14: Read the log call kwargs. Flag CRITICAL for any kwarg not in {exc_info, extra, stack_info, stacklevel}. Do NOT flag structlog or loguru projects.
• L16: Check the config dict passed to dictConfig(). If "disable_existing_loggers" is absent or True → HIGH. If False → ACCEPTABLE.
• L17: Collect all basicConfig() calls across the codebase. If more than one → flag the second+ as HIGH. Single call in if __name__ == "__main__": → ACCEPTABLE.
• L19: Check if the .bind() call is a bare statement (result not assigned) → HIGH. If assigned (x = logger.bind(...)) → ACCEPTABLE.
• L20: Read 10 lines around propagate = False. Check if addHandler() is called on the same logger. If no handler → HIGH. If handler present → ACCEPTABLE.
• L22: Read the message string. If any kwarg name has no matching {kwarg_name} placeholder in the message → MEDIUM. If all kwargs have placeholders → ACCEPTABLE (used for formatting).

Apply SEVERITY_FILTER: drop findings below the threshold.

Step 2.4 — Build Remediation Plan

For each genuine finding, produce an entry:

• Location: file:line
• Severity: CRITICAL / HIGH / MEDIUM / LOW
• Category: L1–L23 code + name
• Current code: offending lines
• Fix template: reference to FT-L1 through FT-L23 from logging-patterns.md
• Auto-fixable: yes/no

Group into priority tiers:

• P0 — Fix Immediately: CRITICAL — L10 (credential leak), L13 (reserved key crash), L14 (stdlib kwargs crash)
• P1 — Fix This Sprint: HIGH — L1, L2, L4, L15, L16, L17, L19, L20, L21
• P2 — Fix When Touching File: MEDIUM/LOW — L3, L5, L6, L7, L8, L9, L11, L12, L18, L22, L23, L24

Also compute linting recommendations: read pyproject.toml for existing ruff or flake8 rule selections. List any missing rules from §Linting Rules in logging-patterns.md:

• G001, G003, G004, T201, LOG009 — recommend if not present
• G002 — do NOT recommend (%-style is the preferred formatting; G002 would produce false positives)
• LOG015 — recommend if stdlib and ruff version supports it

Include the minimal pyproject.toml diff and suggested lint-logging Makefile target in the plan.

Step 2.5 — Enter Plan Mode

Call EnterPlanMode.

Present the Tune Report as the plan:

# Tune Report — <TARGET_PATH>

## Project Logging Profile
- Framework: <LOGGER_FRAMEWORK>
- Canonical factory: <CANONICAL_FACTORY>
- Supports kwargs: <yes/no>
- Logger variable names found: logger, log, ...

## Executive Summary
- Files scanned: N
- Total hits: N (N genuine findings, N acceptable patterns)
- By severity: CRITICAL: N | HIGH: N | MEDIUM: N | LOW: N
- Auto-fixable: N | Needs manual review: N

## Logger Consistency
- Files using canonical factory: N
- Files using non-canonical factory: N (list)

## Findings
| # | File | Line | Severity | Pattern | Auto-fixable | Notes |
|---|------|------|----------|---------|--------------|-------|

## Remediation Plan

### P0 — Fix Immediately (CRITICAL)
[L10, L13, L14 findings with exact lines and fix template]

### P1 — Fix This Sprint (HIGH)
[L1, L2, L4, L15, L16, L17, L19, L20, L21 findings]

### P2 — Fix When Touching File (MEDIUM/LOW)
[L3, L5, L6, L7, L8, L9, L11, L12, L18, L22, L23 findings]

## Linting Recommendations
[pyproject.toml diff]
[Makefile target suggestion]

## Acceptable Patterns (no action required)
| File | Line | Pattern | Justification |
|------|------|---------|---------------|

Call ExitPlanMode to request user approval. Wait for approval before proceeding to Stage 2.

---

Stage 2 — Execution (after plan approval)

Apply the remediation plan that was approved. Only fix what was listed in the approved plan.

Auto-fixable patterns — apply directly:

• FT-L1: f-string → rewrite as %-style message body (embed context in message, not kwargs)
• FT-L2: Import swap to canonical factory
• FT-L4: Add exc_info=True to logger.warning( / logger.error( in except blocks
• FT-L5: Simple print("...") → logger.info("...") using %-style for any embedded values
• FT-L7: .critical( → .error(
• FT-L11: String concatenation → %-style message body
• FT-L14: Wrap stdlib log call kwargs in extra={} (stdlib only)
• FT-L16: Add "disable_existing_loggers": False to dictConfig call
• FT-L19: Capture bind() return value — logger.bind(...) → logger = logger.bind(...)
• FT-L22: loguru kwargs → %-style message body
• FT-L23: .warn( → .warning(
• FT-L24: Application kwargs → %-style message body (keep exc_info=True)

Do not auto-fix L10 (credential leak) — always flag for human review only.

Manual review required — leave TODO comments:

• FT-L3, FT-L6, FT-L8, FT-L9, FT-L10, FT-L12, FT-L13, FT-L15, FT-L17, FT-L18, FT-L20, FT-L21

TODO comment format:

# TODO(signal-over-noise): [L6] INFO inside loop — consider DEBUG + summary. See references/logging-patterns.md#L6

Apply linting config changes (if approved in the plan):

Update pyproject.toml with the recommended ruff rules.

After applying fixes: