Turns a rough podcast transcript, timestamps, and guest notes into publish-ready show notes: an episode title, an SEO summary, key takeaways, chapter markers for YouTube, Spotify, and Podcasting 2.0, timed pull quotes, a guest bio, links, and social posts. Use when a user shares a transcript and asks "write my show notes", "make chapters for this episode", or "pull quotes from this interview". Includes a tested stdlib Python chapter finder.
---
name: podcast-show-notes-structurer
description: Turns a rough podcast transcript, timestamps, and guest notes into publish-ready show notes: an episode title, an SEO summary, key takeaways, chapter markers for YouTube, Spotify, and Podcasting 2.0, timed pull quotes, a guest bio, links, and social posts. Use when a user shares a transcript and asks "write my show notes", "make chapters for this episode", or "pull quotes from this interview". Includes a tested stdlib Python chapter finder.
---
# Podcast Show Notes Structurer
You help podcasters turn a messy transcript into show notes that listeners actually read and search engines can find. You find the real structure of the conversation, name chapters the way a listener would describe them, choose quotes that work out of context, and write a summary that tells someone in two lines why this episode is worth their time. You never invent facts, links, or quotes that are not in the material.
## Files in this skill
- `scripts/chapterize.py` - reads timestamped, WebVTT, headed, or plain transcripts, proposes chapter starts from headings, pause markers, cue phrases ("let's talk about", "after the break"), and vocabulary shifts, drafts keyword titles, lists ad and music markers, and ranks quote candidates (Python 3 standard library only)
- `references/show-notes-structure.md` - what goes in each section, length limits, and how to write titles, summaries, and takeaways
- `references/chapters-and-platforms.md` - chapter rules for YouTube, Spotify, Apple Podcasts, and Podcasting 2.0, plus quote and timestamp ethics
- `templates/show-notes.md` - the show notes document to return
- `examples/example-rooftop-beekeeping-episode.md` - a worked example from a 25-minute interview
## Workflow
### 1. Collect the material
Ask, or assume and say so:
- The transcript (any format) and whether its times match the final edit. If an intro or ad was added after transcription, get the offset in seconds.
- Show name, host and guest names with correct spelling, the guest's role, and links they want included (only links the user gives you).
- Where the notes will be published (podcast host, YouTube, website, newsletter) and any house style: tone, length, emoji, sponsor wording.
- The audience and 2 or 3 search phrases a listener might type to find this episode.
### 2. Run the chapter finder
```bash
python3 scripts/chapterize.py transcript.txt
python3 scripts/chapterize.py transcript.txt --min-chapter 3 --offset 45 --duration 52:30
python3 scripts/chapterize.py episode.vtt --format youtube
python3 scripts/chapterize.py - --format json < transcript.txt
```
The script's chapter starts and quotes are candidates, not answers. Read the transcript around every proposed start and move it to the sentence where the new topic really begins. Rewrite every keyword title (for example "Roof / Hive / Water / Wind") into a short listener-facing title ("Keeping bees on a windy roof"). If there are no timestamps, times are estimates (marked `~`); tell the user to replace them with times from the audio editor. If you cannot run the script, find chapters by hand the same way: headings, breaks, transition phrases, then topic changes.
### 3. Write the notes
1. Title: 40 to 60 characters, the main promise of the episode, guest name if it helps search.
2. Summary: 2 sentences for the app preview (the first 150 characters must work alone), then one short paragraph.
3. Key takeaways: 3 to 5 bullets, each a concrete idea from the episode, not a topic name.
4. Chapters: 4 to 10 for a 20 to 60 minute episode, first at 00:00, titles under 40 characters, in the order they happen.
5. Quotes: 3 to 5 lines that make sense without context, with speaker and time. Remove filler words only; never change meaning. Mark edited quotes so the user can check them against the audio.
6. Guest bio (2 or 3 sentences), links and resources mentioned (only real ones from the material or the user), credits, sponsor disclosure if there was an ad.
7. Social posts: one per platform the user names, each with one quote or takeaway.
### 4. Deliver
Fill in `templates/show-notes.md` and follow `references/show-notes-structure.md` and `references/chapters-and-platforms.md`. Follow the style of `examples/example-rooftop-beekeeping-episode.md`. Add a short "check before publishing" list: estimated times, edited quotes, spellings, and any facts or numbers the guest stated that should be verified.
## Rules
- Only use facts, names, numbers, and links from the transcript or the user; flag anything unclear as [check].
- Quotes must be the speaker's own words; removing "um" and "you know" is fine, rewording is not.
- Keep chapter titles descriptive and honest; no clickbait the episode does not deliver.
- Respect guests: do not quote private remarks, off-the-record moments, or anything the user marks as cut.
- Match the show's voice; default to warm, clear, and specific.
FILE:references/show-notes-structure.md
# Show notes structure
Listeners skim show notes in a podcast app on a phone, search engines read them on the episode page, and the guest shares them. Each section has one job.
| Section | Job | Length |
|---|---|---|
| Episode title | Say the main promise; help search | 40 to 60 characters; most apps cut after about 60 |
| Preview summary | Make someone press play from the app list | First 150 characters must work alone; 2 sentences total |
| Full summary | Explain who the guest is and what you cover | 60 to 120 words |
| Key takeaways | Give skimmers the value | 3 to 5 bullets, one idea each, under 20 words |
| Chapters | Let listeners jump | 4 to 10 for 20 to 60 minutes; titles under 40 characters |
| Pull quotes | Quotable lines for notes and social | 3 to 5, 12 to 30 words each, with speaker and time |
| Guest bio | Credit the guest | 2 or 3 sentences, written in third person |
| Links and resources | Everything mentioned that has a link | Only links the user or guest gave |
| Credits and disclosure | Host, guest, producer, music, sponsor | One line each |
| Call to action | One next step | Subscribe, review, newsletter, or a question for listeners |
## Titles
- Lead with the idea, not the episode number: "Keeping Bees on a Hotel Roof with Daniel Okafor" beats "Ep. 48: Daniel Okafor".
- Use the words a listener would search: "rooftop beekeeping", "city honey".
- Avoid questions the episode does not answer and promises the guest did not make.
## Summaries
- Sentence 1: who and what, in plain words. Sentence 2: the most surprising or useful point.
- Write for people, not keywords; include the 2 or 3 search phrases naturally once each.
- No "In this episode we talk about..." filler; start with the substance.
## Key takeaways
Write each takeaway as a claim or instruction from the episode, not a topic:
- Weak: "Water on roofs."
- Strong: "Give rooftop bees a shallow water tray, or they will find the neighbors' pool."
## Chapter titles
- Describe what the listener gets in that part, in 2 to 6 words: "Why wind is the real enemy".
- Keep them parallel in style (all statements or all questions).
- Name ad breaks only if the platform does not hide them; never call an ad a chapter about content.
## Quotes
- Pick lines that stand alone: a clear claim, a lesson, a number, or a vivid image.
- Avoid greetings, setup questions, and lines that need the previous sentence.
- Removing fillers (um, uh, you know) is fine; mark the quote as lightly edited. Changing words is not.
## Accessibility and trust
- Link a full transcript when one exists; it helps deaf and hard-of-hearing listeners and search.
- Disclose sponsors and affiliate links clearly.
- Put a [check] tag on any number, name, or claim that should be verified before publishing.
FILE:references/chapters-and-platforms.md
# Chapters and platform rules
Rules change; when in doubt, check the platform's current help page. These are the stable basics.
## YouTube (video podcasts and audio uploads)
- Chapters come from timestamps in the description, one per line: `00:00 Intro`.
- The first timestamp must be 00:00.
- At least 3 timestamps, and each chapter at least 10 seconds long.
- Use `MM:SS` for videos under an hour and `H:MM:SS` once any chapter is past one hour.
- `chapterize.py --format youtube` prints lines in this format and warns when a rule is broken.
## Spotify
- Spotify reads chapter timestamps from the episode description in the same `MM:SS Title` style for episodes on its platform; follow the same rules as YouTube (start at 00:00, a few minutes apart).
## Apple Podcasts and most other apps
- Chapters are embedded in the audio file: ID3 CHAP frames in MP3 or a chapter track in M4A/AAC. Add them in the audio editor or a tagging tool before uploading.
- Apple may show automatically generated chapters when an episode has none; providing your own keeps control of the titles.
## Podcasting 2.0 (RSS)
- The `<podcast:chapters>` tag points to a JSON chapters file, supported by many modern apps and hosts:
```json
{
"version": "1.2.0",
"chapters": [
{ "startTime": 0, "title": "Intro" },
{ "startTime": 280, "title": "Keeping bees on a windy roof" }
]
}
```
- `startTime` is in seconds; `chapterize.py --format json` gives `start` in seconds for each chapter.
## Timestamps
- Transcript times often differ from the final file: intros, ads, and edits shift them. Use `--offset` for a fixed shift and recheck a few chapters in the audio.
- Without timestamps, `chapterize.py` estimates times from word count (default 150 words per minute) and marks them with `~`. Never publish estimated times without checking.
- Dynamic ad insertion moves times per listener; put chapters in the file or description only when ads are baked in or the host adjusts chapters automatically.
## How the script finds chapter starts
1. Headings (`# Title`, `Chapter 3: ...`, `Segment: ...`) always start a chapter.
2. Pause markers (`[music]`, `[ad break]`, `[pause]`, `***`, `---`) mark the next spoken block as a candidate.
3. Section cue phrases score high ("let's talk about", "moving on", "switching gears", "welcome back", "before we wrap up"); single-question cues ("next question") score lower so a Q&A segment stays one chapter.
4. Vocabulary shift: how different the words are in the 3 blocks before and after a point.
5. Starts closer than `--min-chapter` minutes are dropped (headings excepted), and nothing starts in the last minute.
## Quote and transcript ethics
- Keep quotes faithful; mark filler removal as an edit.
- Ask before quoting a guest's personal story on social media if it was sensitive.
- Do not publish remarks the user marked as off the record or cut from the episode.
FILE:templates/show-notes.md
# episode_title
**show_name** | Episode episode_number | duration | Guest: guest_name
preview_summary_two_sentences
full_summary_paragraph
## Key takeaways
-
-
-
## Chapters
```
00:00 chapter_1
time chapter_2
```
## Quotes
> "quote" - speaker (time)
## About guest_name
guest_bio
## Links and resources mentioned
- resource - url_or_check
## Credits
Host: host | Guest: guest_name | Produced by: producer | Music: music_credit
sponsor_disclosure
call_to_action
---
## Social posts
- **platform:** post_with_quote_or_takeaway
## Check before publishing
- [ ] Chapter times checked against the final audio (offset: offset)
- [ ] Edited quotes checked against the audio
- [ ] Names and spellings confirmed
- [ ] Facts and numbers marked [check] verified
- [ ] Links tested
FILE:examples/example-rooftop-beekeeping-episode.md
# Example: show notes for a 25-minute interview
## User request
"Here's the transcript of episode 48 of Small Plot Radio with Daniel Okafor, who keeps bees on a hotel roof. We added a 45-second intro in the edit, so the final file runs 26:02. Can you write the show notes with chapters for YouTube and Spotify, a few quotes, and an Instagram caption? The hotel tours and Daniel's course have links on their sites; I'll add them."
## Chapter finder run
```
$ python3 scripts/chapterize.py rooftop-bees.txt
Transcript: 1140 words, about 25:17 long, timestamps found, speakers: Daniel (20 turns), Maya (19 turns)
Chapter candidates (6, min 4 min apart):
1. 00:00 4.7 min [detected] Hotel Manager / Hive / Twelve / Bees
2. 04:40 4.5 min [detected] Roof / Hive / Water / Wind
3. 09:12 5.3 min [detected] Honey Tour / Spring Honey / Late Summer / Guests
4. 14:30 5.0 min [detected] City / Bees / Honey / Roof
5. 19:30 4.7 min [detected] Hive / Balcony / Bees / Priya
6. 24:10 1.1 min [detected] Keeping / Colony / Association / Find
Markers (breaks, music, ads): 13:30 [ad break]
Quote candidates (6):
00:36 Daniel: "I built my first hive from a kit that winter and I made every beginner mistake you can imagine."
01:24 Daniel: "The biggest lesson I learned that first year was that good beekeeping is mostly patience."
05:33 Daniel: "If you don't give them water, they will find the hotel pool or the neighbors' air conditioning drip, and that is how complaints start."
...
$ python3 scripts/chapterize.py rooftop-bees.txt --offset 45 --format youtube
00:00 Hotel Manager / Hive / Twelve / Bees
05:25 Roof / Hive / Water / Wind
09:57 Honey Tour / Spring Honey / Late Summer / Guests
15:15 City / Bees / Honey / Roof
20:15 Hive / Balcony / Bees / Priya
24:55 Keeping / Colony / Association / Find
# note: YouTube needs the first chapter at 00:00; the first one was moved to 00:00
```
Review of the candidates: all six starts land on real transitions ("Let's talk about the roof itself", "Switching gears to the hotel manager", "Welcome back", "Let's move on to listener questions", "Before we wrap up"). The 13:30 ad break stays inside chapter 3 because the ad is baked in. Chapter 1 is long, so it was split by hand at 01:20 ("What was the biggest mistake?"), which becomes 02:05 after the offset. Every keyword title was rewritten.
## Show notes (filled template)
# Keeping Bees on a Hotel Roof with Daniel Okafor
**Small Plot Radio** | Episode 48 | 26:02 | Guest: Daniel Okafor
Daniel Okafor keeps twelve beehives on a downtown hotel roof. He explains how to beat wind, heat, and thirst on a roof, and why planting flowers can help pollinators more than adding hives.
Nine years ago a swarm landed on Daniel's balcony; today he runs twelve hives on a hotel roof, supplies the hotel kitchen with honey, and leads sold-out rooftop honey tours. Maya and Daniel talk about beginner mistakes, windbreaks, water trays, and roof loads, how he won over a hotel manager, whether city honey is safe to eat, and why too many city hives can crowd out wild bees. They finish with listener questions on balcony beekeeping, starter equipment, and honey yields.
## Key takeaways
- Inspect hives every 7 to 10 days in spring, with a plan, and leave them alone in between.
- On a roof, wind is the main enemy: a 1.5 m slatted windbreak on the exposed sides makes the difference.
- Give rooftop bees a shallow water tray with pebbles, or they will find the neighbors' pool.
- Have the roof load checked first: a full hive can weigh 80 to 100 kg.
- Planting flowers helps pollinators more than adding another hive to a crowded city.
## Chapters
```
00:00 How a swarm made Daniel a beekeeper
02:05 The beginner mistake: opening the hive too often
05:25 Keeping bees on a windy roof
09:57 Winning over the hotel (and its kitchen)
15:15 Is city honey safe, and are there too many hives?
20:15 Listener questions: balconies, gear, and honey yields
24:55 The one thing every new beekeeper should know
```
## Quotes
> "The biggest lesson I learned that first year was that good beekeeping is mostly patience." - Daniel Okafor (02:09)
> "If you don't give them water, they will find the hotel pool or the neighbors' air conditioning drip, and that is how complaints start." - Daniel Okafor (06:18)
> "If you want to help pollinators, planting flowers matters more than adding another hive." - Daniel Okafor (16:53)
## About Daniel Okafor
Daniel Okafor is a city beekeeper who has kept bees for nine years and now runs twelve hives on the roof of a downtown hotel. He leads the hotel's monthly rooftop honey tours and teaches a beginner course with the city beekeeping association every spring.
## Links and resources mentioned
- Hotel rooftop honey tours - [link from user]
- Beginner beekeeping course, city beekeeping association - [link from user]
- Mason bee houses for balconies (mentioned at 20:15) - no link given
## Credits
Host: Maya | Guest: Daniel Okafor | Produced by: [check] | Music: [check]
This episode includes an advertisement at 14:15. [check sponsor name for disclosure]
Got a balcony or roof question for Daniel? Send it in and we'll answer it in a future episode.
---
## Social posts
- **Instagram:** "Planting flowers matters more than adding another hive." Daniel Okafor keeps twelve hives on a hotel roof, and his advice for helping city pollinators might surprise you. New episode of Small Plot Radio, link in bio.
## Check before publishing
- [ ] Chapter times checked against the final audio (offset: +45 s added to every transcript time)
- [ ] Quotes checked against the audio (none were edited)
- [ ] Spelling of "Okafor" confirmed with the guest
- [ ] Facts to verify: "zero stings on guests in nine seasons", yearly lab tests below limits, 15 to 30 kg per hive
- [ ] Links added and tested
FILE:scripts/chapterize.py
#!/usr/bin/env python3
"""Split a podcast transcript into rough chapter candidates and pull quote candidates.
Usage:
python3 chapterize.py TRANSCRIPT [options]
python3 chapterize.py - < transcript.txt (read stdin)
Accepted input (detected automatically, can be mixed):
- timestamp lines: "[00:12:30] Ana: text", "00:12:30 text", "(12:30) text", "12:30 - text"
- WebVTT/SRT cue times: "00:01:02.000 --> 00:01:05.000" (the cue text follows)
- headings: "# Intro", "## Part 2", "Chapter 3: Pricing", "SEGMENT: listener questions"
- pause markers: [pause], [music], [break], [ad break], (long pause), ***, ---
- plain paragraphs separated by blank lines (times are estimated from --wpm)
Options:
--min-chapter MIN shortest chapter in minutes (default 4)
--max-chapters N most chapters to return (default 12)
--wpm N speaking rate used to estimate times without timestamps (default 150)
--offset SEC seconds to add to every time, for an intro added in the edit (default 0)
--quotes N number of quote candidates (default 6)
--format FMT text (default), json, or youtube (chapter lines "00:00 Title")
--duration HH:MM:SS total episode length, to check the last chapter and YouTube rules
Chapter boundaries come from headings first, then pause markers and cue phrases
("let's talk about", "moving on", "after the break"), then vocabulary shifts between
neighbouring blocks. Titles are keyword guesses: always rewrite them for the listener.
Exit codes: 0 ok, 1 input too short for chapters, 2 usage or read error.
Python 3 standard library only.
"""
import json
import re
import sys
from collections import Counter
TS_LINE = re.compile(r"^\s*[\[(]?((?:\d{1,2}:)?\d{1,2}:\d{2})(?:[.,]\d+)?[\])]?\s*(?:[-\u2013\u2014|:]\s*)?(.*)$")
CUE_LINE = re.compile(r"^\s*((?:\d{1,2}:)?\d{1,2}:\d{2})[.,]\d{1,3}\s*-->\s*((?:\d{1,2}:)?\d{1,2}:\d{2})?")
LAST_CUE_END = [0]
HEADING = re.compile(r"^\s*(?:#{1,6}\s+(.+)|(?:chapter|part|segment|section|topic)\s*\d*\s*[:.\-\u2013]\s*(.+))\s*$", re.I)
PAUSE = re.compile(r"^\s*(?:\[(?:pause|long pause|music|break|ad break|ad|sponsor|interlude|silence)[^\]]*\]|"
r"\((?:pause|long pause|music|break|ad break|silence)[^)]*\)|\*{3,}|-{3,}|={3,})\s*$", re.I)
SPEAKER = re.compile(r"^([A-Z][\w.'-]*(?: [A-Z][\w.'-]*){0,2}):\s+(.*)$")
# section-level cue phrases start a new chapter; item-level ones (single questions) only nudge
CUES = ["let's talk about", "lets talk about", "let's move on", "moving on", "switching gears", "change gears",
"after the break", "welcome back", "before we wrap", "to wrap up", "let's get into", "let's dive into",
"let's start with", "rapid fire", "listener questions"]
ITEM_CUES = ["next question", "first question", "last question", "final question", "the next thing",
"another thing i want", "speaking of"]
INTRO = re.compile(r"\b(welcome (to|back)|my guest|today i'm talking|today i am talking|thanks for (having|listening)|"
r"see you (next|in)|where can people find)\b", re.I)
STOP = set("""a about above after again against all also am an and any are aren't as at be because been before being
below between both but by can can't could couldn't did didn't do does doesn't doing don't down during each even ever
every few for from further get gets getting go going gonna got had hadn't has hasn't have haven't having he he'd he'll
he's her here here's hers herself him himself his how how's i i'd i'll i'm i've if in into is isn't it it's its itself
just know kind kinda let let's like lot really right said say says see so some such sort than that that's the their
theirs them themselves then there there's these they they'd they'll they're they've thing things think this those
though through to too um uh under until up us very want was wasn't way we we'd we'll we're we've well were weren't what
what's when when's where where's which while who who's whom why why's will with won't would wouldn't yeah yes you you'd
you'll you're you've your yours yourself yourselves okay ok actually basically maybe mean pretty probably stuff sure
talk talking yeah one two three lot lots much many more most other others same back new good great time people
something anything everything nothing make made making take took come came look looking need needs needed thank
thanks today episode podcast show welcome not first question questions listener listeners she she's she'd she'll now""".split())
QUOTE_WORDS = re.compile(r"\b(never|always|nobody|everyone|the secret|the key|the biggest|the hardest|the best|the worst|"
r"mistake|learned|lesson|truth|matters|changed|surprised|realized|if you|you have to|don't)\b", re.I)
FILLER = re.compile(r"\b(um+|uh+|you know|i mean|sort of|kind of|like,)\s*", re.I)
class UsageError(Exception):
pass
def to_sec(ts):
parts = [int(p) for p in ts.split(":")]
while len(parts) < 3:
parts.insert(0, 0)
h, m, s = parts
return h * 3600 + m * 60 + s
def fmt(sec, force_hours=False):
sec = int(round(sec))
h, rem = divmod(sec, 3600)
m, s = divmod(rem, 60)
return f"{h:d}:{m:02d}:{s:02d}" if h or force_hours else f"{m:02d}:{s:02d}"
def words(text):
return re.findall(r"[A-Za-z][A-Za-z'-]+", text)
def parse(text):
"""Return a list of units: dict(kind=text|heading|pause, text, time or None, speaker)."""
units, pending_time = [], None
lines = text.replace("\r\n", "\n").replace("\r", "\n").lstrip("\ufeff").split("\n")
para = []
def flush():
if para:
joined = " ".join(p["text"] for p in para)
units.append({"kind": "text", "text": joined, "time": para[0]["time"], "speaker": para[0]["speaker"]})
para.clear()
for raw in lines:
line = raw.strip()
if not line or line.upper() == "WEBVTT" or line.isdigit():
if not line:
flush()
continue
m = CUE_LINE.match(line)
if m:
pending_time = to_sec(m.group(1))
if m.group(2):
LAST_CUE_END[0] = max(LAST_CUE_END[0], to_sec(m.group(2)))
continue
h = HEADING.match(line)
if h:
flush()
units.append({"kind": "heading", "text": (h.group(1) or h.group(2)).strip(), "time": pending_time,
"speaker": None})
continue
if PAUSE.match(line):
flush()
units.append({"kind": "pause", "text": line, "time": pending_time, "speaker": None})
continue
t = TS_LINE.match(line)
time = pending_time
if t and t.group(2) is not None and (":" in t.group(1)):
time, line = to_sec(t.group(1)), t.group(2).strip()
flush()
if not line:
pending_time = time
continue
h = HEADING.match(line)
if h:
units.append({"kind": "heading", "text": (h.group(1) or h.group(2)).strip(), "time": time, "speaker": None})
pending_time = None
continue
if PAUSE.match(line):
units.append({"kind": "pause", "text": line, "time": time, "speaker": None})
pending_time = None
continue
sp = SPEAKER.match(line)
speaker = None
if sp and len(sp.group(1)) <= 30:
speaker, line = sp.group(1), sp.group(2)
flush()
if time is not None or pending_time is not None:
flush()
para.append({"text": line, "time": time, "speaker": speaker})
pending_time = None
flush()
return units
def assign_times(units, wpm, offset):
"""Fill missing times by interpolating between known ones or from the word count."""
known = [i for i, u in enumerate(units) if u["time"] is not None]
clock = 0.0
timed = bool(known)
for i, u in enumerate(units):
if u["time"] is not None:
clock = float(u["time"])
u["estimated"] = False
else:
u["time"] = clock
u["estimated"] = True
clock = u["time"] + len(words(u["text"])) / wpm * 60 if u["kind"] == "text" else u["time"]
for u in units:
u["time"] += offset
return timed
EXTRA_STOP = set()
def keywords(text, n=4):
ws = [w.lower().strip("'-") for w in words(text)]
ws = [w for w in ws if len(w) > 2 and w not in STOP and w not in EXTRA_STOP]
counts = Counter(ws)
bigrams = Counter(f"{a} {b}" for a, b in zip(ws, ws[1:]) if a != b)
out = [b for b, c in bigrams.most_common(3) if c >= 2]
for w, _c in counts.most_common(12):
if len(out) >= n:
break
if not any(w in b.split() for b in out):
out.append(w)
return out[:n]
def similarity(a, b):
wa = {w.lower() for w in words(a) if w.lower() not in STOP and len(w) > 2}
wb = {w.lower() for w in words(b) if w.lower() not in STOP and len(w) > 2}
if not wa or not wb:
return 0.0
return len(wa & wb) / len(wa | wb)
def boundaries(units, min_gap, max_chapters, end_time):
"""Score each unit as a chapter start; return chosen indexes (always includes the first text unit)."""
scores = {}
for i, u in enumerate(units):
if u["kind"] == "heading":
scores[i] = 10.0
elif u["kind"] == "pause":
nxt = next((j for j in range(i + 1, len(units)) if units[j]["kind"] == "text"), None)
if nxt is not None:
scores[nxt] = max(scores.get(nxt, 0), 3.0)
elif u["kind"] == "text":
low = u["text"].lower()
cue = 4.0 if any(c in low[:160] for c in CUES) else 1.5 if any(c in low[:160] for c in ITEM_CUES) else 0.0
before = " ".join(x["text"] for x in units[max(0, i - 3):i] if x["kind"] == "text")
after = " ".join(x["text"] for x in units[i:i + 3] if x["kind"] == "text")
shift = 1.0 - similarity(before, after) if before and after else 0.0
score = shift * 2.0 + cue
if score > 1.6:
scores[i] = max(scores.get(i, 0), score)
first = next((i for i, u in enumerate(units) if u["kind"] in ("text", "heading")), 0)
chosen = [first]
for i in sorted(scores, key=lambda k: (-scores[k], k)):
if len(chosen) >= max_chapters:
break
t = units[i]["time"]
forced = units[i]["kind"] == "heading"
strong = forced or scores[i] >= 4.0 # heading or cue phrase
if end_time - t < (60 if strong else min_gap / 2) and not forced:
continue # too close to the end of the episode
if all(abs(t - units[c]["time"]) >= min_gap for c in chosen) or (forced and i not in chosen and
all(abs(t - units[c]["time"]) >= 10 for c in chosen)):
chosen.append(i)
return sorted(set(chosen)), scores
def sentences(text):
return [s.strip() for s in re.split(r"(?<=[.!?])\s+", text) if s.strip()]
def quote_candidates(units, n):
cands = []
for u in units:
if u["kind"] != "text":
continue
for s in sentences(u["text"]):
clean = re.sub(r"\s{2,}", " ", FILLER.sub("", s)).strip()
wc = len(clean.split())
if wc < 8 or wc > 40 or clean.endswith("?"):
continue
score = 0.0
score += 2.0 if 12 <= wc <= 28 else 0.5
score += 1.5 * len(QUOTE_WORDS.findall(clean))
score += 1.0 if re.search(r"\b\d+(?:[.,]\d+)?\s*(%|percent|x|times|years|hours|days)\b", clean, re.I) else 0
score += 0.5 if re.search(r"\b(I|we)\b", clean) else 0
score -= 1.0 if FILLER.search(s) else 0
score -= 3.0 if INTRO.search(clean) else 0
cands.append({"time": u["time"], "speaker": u["speaker"], "text": clean, "score": round(score, 1),
"edited": clean != s})
cands.sort(key=lambda c: (-c["score"], c["time"]))
out, seen = [], set()
for c in cands:
key = c["text"][:40].lower()
if key in seen:
continue
seen.add(key)
out.append(c)
if len(out) >= n:
break
return sorted(out, key=lambda c: c["time"])
def build(units, chosen, end_time):
chapters = []
for k, start in enumerate(chosen):
stop = chosen[k + 1] if k + 1 < len(chosen) else len(units)
seg = units[start:stop]
text = " ".join(u["text"] for u in seg if u["kind"] == "text")
heading = next((u["text"] for u in seg[:1] if u["kind"] == "heading"), None)
t0 = units[start]["time"]
t1 = units[stop]["time"] if stop < len(units) else end_time
speakers = Counter(u["speaker"] for u in seg if u["speaker"])
first = sentences(text)[:1]
chapters.append({
"n": k + 1, "start": round(t0), "start_hms": fmt(t0), "end": round(t1), "minutes": round((t1 - t0) / 60, 1),
"estimated_time": any(u.get("estimated") for u in seg[:1]),
"source": "heading" if heading else "detected",
"title_draft": heading or " / ".join(keywords(text)).title() or "(untitled)",
"keywords": keywords(text, 6), "words": len(words(text)),
"speakers": [s for s, _ in speakers.most_common()],
"opening_line": first[0][:160] if first else "",
})
return chapters
def youtube_lines(chapters):
lines, problems = [], []
if chapters and chapters[0]["start"] != 0:
problems.append("YouTube needs the first chapter at 00:00; the first one was moved to 00:00")
for c in chapters:
start = 0 if c is chapters[0] else c["start"]
lines.append(f"{fmt(start, force_hours=chapters[-1]['start'] >= 3600)} {c['title_draft']}")
if len(chapters) < 3:
problems.append("YouTube shows chapters only with at least 3 of them")
short = [c["n"] for c in chapters if c["minutes"] * 60 < 10]
if short:
problems.append(f"chapters {short} are shorter than 10 seconds (YouTube minimum)")
return lines, problems
def parse_args(argv):
opts = {"file": None, "min_chapter": 4.0, "max_chapters": 12, "wpm": 150.0, "offset": 0.0, "quotes": 6,
"format": "text", "duration": None}
numeric = {"--min-chapter": ("min_chapter", float), "--max-chapters": ("max_chapters", int),
"--wpm": ("wpm", float), "--offset": ("offset", float), "--quotes": ("quotes", int)}
i = 0
while i < len(argv):
a = argv[i]
if a in ("-h", "--help"):
print(__doc__)
sys.exit(0)
if a in numeric or a in ("--format", "--duration"):
if i + 1 >= len(argv):
raise UsageError(f"{a} needs a value")
v = argv[i + 1]
if a == "--format":
if v not in ("text", "json", "youtube"):
raise UsageError("--format must be text, json, or youtube")
opts["format"] = v
elif a == "--duration":
if not re.fullmatch(r"(?:\d{1,2}:)?\d{1,2}:\d{2}", v):
raise UsageError("--duration must look like 52:30 or 1:05:00")
opts["duration"] = to_sec(v)
else:
key, typ = numeric[a]
try:
opts[key] = typ(v)
except ValueError:
raise UsageError(f"{a} needs a number, got {v!r}")
if opts[key] < 0 or (key in ("wpm", "max_chapters") and opts[key] == 0):
raise UsageError(f"{a} must be positive")
i += 2
elif a.startswith("--"):
raise UsageError(f"unknown option {a}")
elif opts["file"] is None:
opts["file"] = a
i += 1
else:
raise UsageError(f"unexpected argument {a!r}")
if opts["file"] is None:
raise UsageError("give a transcript file, or - for stdin")
return opts
def main(argv):
try:
opts = parse_args(argv)
if opts["file"] == "-":
text = sys.stdin.read()
else:
with open(opts["file"], encoding="utf-8") as f:
text = f.read()
except UsageError as e:
print(f"error: {e}", file=sys.stderr)
return 2
except (OSError, UnicodeDecodeError) as e:
print(f"error: cannot read {opts['file']}: {getattr(e, 'strerror', None) or e}", file=sys.stderr)
return 2
units = parse(text)
total_words = sum(len(words(u["text"])) for u in units if u["kind"] == "text")
if total_words < 50:
print(f"error: only {total_words} words of transcript; need at least 50 to suggest chapters", file=sys.stderr)
return 1
timed = assign_times(units, opts["wpm"], opts["offset"])
last = units[-1]
end_time = last["time"] + (len(words(last["text"])) / opts["wpm"] * 60 if last["kind"] == "text" else 0)
end_time = max(end_time, LAST_CUE_END[0] + opts["offset"] if LAST_CUE_END[0] else 0)
if opts["duration"]:
end_time = max(end_time, opts["duration"])
EXTRA_STOP.update(w.lower() for u in units if u["speaker"] for w in u["speaker"].split())
chosen, _scores = boundaries(units, opts["min_chapter"] * 60, opts["max_chapters"], end_time)
chapters = build(units, chosen, end_time)
quotes = quote_candidates(units, opts["quotes"])
speakers = Counter(u["speaker"] for u in units if u["speaker"])
notes = []
if not timed:
notes.append(f"no timestamps found: times are estimates at {opts['wpm']:g} words per minute; replace them "
"with real times from the audio editor before publishing")
if opts["duration"] and opts["duration"] < end_time - 30 and timed:
notes.append("the transcript runs past --duration; check the timestamps or the offset")
markers = [{"time": round(u["time"]), "time_hms": fmt(u["time"]), "marker": u["text"]}
for u in units if u["kind"] == "pause"]
yt_lines, yt_problems = youtube_lines(chapters)
report = {"markers": markers, "words": total_words, "timestamps": "found" if timed else "estimated", "end": round(end_time),
"end_hms": fmt(end_time), "speakers": dict(speakers.most_common()), "chapters": chapters,
"quotes": quotes, "youtube": yt_lines, "youtube_problems": yt_problems, "notes": notes}
if opts["format"] == "json":
print(json.dumps(report, indent=2, ensure_ascii=False))
return 0
if opts["format"] == "youtube":
print("\n".join(yt_lines))
for p in yt_problems + notes:
print(f"# note: {p}")
return 0
spk = ", ".join(f"{s} ({c} turns)" for s, c in speakers.most_common()) or "not labeled"
print(f"Transcript: {total_words} words, about {fmt(end_time)} long, timestamps {report['timestamps']}, "
f"speakers: {spk}")
print(f"\nChapter candidates ({len(chapters)}, min {opts['min_chapter']:g} min apart):")
for c in chapters:
est = "~" if c["estimated_time"] else " "
print(f" {c['n']:>2}. {est}{c['start_hms']:>8} {c['minutes']:>5.1f} min [{c['source']}] {c['title_draft']}")
print(f" keywords: {', '.join(c['keywords']) or '-'}")
if c["opening_line"]:
print(f" opens: \"{c['opening_line']}\"")
if markers:
print("\nMarkers (breaks, music, ads): " + ", ".join(f"{m['time_hms']} {m['marker']}" for m in markers))
print(f"\nQuote candidates ({len(quotes)}):")
for q in quotes:
who = f"{q['speaker']}: " if q["speaker"] else ""
edited = " (fillers removed, check against audio)" if q["edited"] else ""
print(f" {fmt(q['time']):>8} {who}\"{q['text']}\"{edited}")
print("\nYouTube chapter lines:")
for line in yt_lines:
print(f" {line}")
for p in yt_problems + notes:
print(f"note: {p}")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))Reviews a schema and slow SQL queries, proposes composite indexes in the right column order (equality, sort, range) with covering columns and tradeoffs, flags non-sargable filters like functions on columns and leading-wildcard LIKE, and warns about duplicate, redundant, and low-selectivity indexes. Use when a user shares a slow query, EXPLAIN output, or schema and asks "what index do I need?" or "too many indexes?". Includes a tested stdlib Python helper.
---
name: sql-index-recommendation-reviewer
description: Reviews a schema and slow SQL queries, proposes composite indexes in the right column order (equality, sort, range) with covering columns and tradeoffs, flags non-sargable filters like functions on columns and leading-wildcard LIKE, and warns about duplicate, redundant, and low-selectivity indexes. Use when a user shares a slow query, EXPLAIN output, or schema and asks "what index do I need?" or "too many indexes?". Includes a tested stdlib Python helper.
---
# SQL Index Recommendation Reviewer
You help developers make slow queries fast without burying their tables in indexes. You read the schema and the query the way a database engineer would: which columns are compared with equality, which with ranges, what the query sorts by, what it joins on, and which indexes already exist. You propose the smallest set of indexes that serves the workload, in the right column order, and you are honest that only EXPLAIN on real data can confirm it.
## Files in this skill
- `scripts/suggest_indexes.py` - parses CREATE TABLE / CREATE INDEX statements and SELECT, UPDATE, and DELETE queries (subqueries and CTE bodies separately), then prints candidate indexes with DDL, the queries each one serves, covering columns, overlaps, and schema findings (Python 3 standard library only; no database connection)
- `references/indexing-rules.md` - column order, covering indexes, partial and expression indexes, sargability, and over-indexing rules
- `references/explain-reading-guide.md` - how to read EXPLAIN in PostgreSQL, MySQL, and SQLite to confirm or reject a suggestion
- `templates/index-review.md` - the review report to return
- `examples/example-shop-orders-review.md` - a worked review of an online shop's slow queries
## Workflow
### 1. Collect the facts
Ask, or assume and say so:
- The engine and version (PostgreSQL, MySQL/MariaDB, SQLite, SQL Server), and the schema with existing indexes (`\d table`, `SHOW CREATE TABLE`, or the migration files).
- The slow queries exactly as the app sends them, with typical parameter values, how often each runs, and how slow it is.
- Table sizes (rows), how write-heavy each table is, and any EXPLAIN or EXPLAIN ANALYZE output.
- Constraints: can the team build indexes online, disk space, and replication lag concerns.
### 2. Run the helper
```bash
python3 scripts/suggest_indexes.py --schema schema.sql slow-queries.sql
python3 scripts/suggest_indexes.py all-in-one.sql --dialect mysql
python3 scripts/suggest_indexes.py --schema schema.sql - --json < query.sql
python3 scripts/suggest_indexes.py --schema schema.sql queries.sql --strict # exit 1 on non-sargable filters or duplicate indexes
```
The helper is a heuristic parser, not a planner: it does not know row counts, data distribution, or which plan the database will choose. It can miss predicates in complex SQL (it lists them as "not understood"). Read every query yourself and treat each suggestion as a candidate to test. If you cannot run the script, apply `references/indexing-rules.md` by hand in the same order.
### 3. Decide
1. Fix the query before adding an index: rewrite non-sargable filters (functions on columns, arithmetic, leading-wildcard LIKE, OR across columns) so an index can be used at all.
2. For each query, choose column order: equality columns (most selective first when you know it), then ORDER BY or GROUP BY columns if the query sorts with LIMIT, then one range column.
3. Merge: one composite index can serve several queries when they share its leftmost columns. Prefer extending an existing index over adding a new one.
4. Add covering (INCLUDE) columns only for hot, read-heavy queries, and keep them few.
5. Remove before adding: drop duplicate and redundant-prefix indexes after checking usage statistics (`pg_stat_user_indexes`, `sys.schema_unused_indexes`).
6. Count the cost: every index slows inserts and updates of its columns and uses disk and memory. Write-heavy tables deserve fewer, broader indexes.
### 4. Deliver
Fill in `templates/index-review.md`: per-query diagnosis, the recommended DDL in build order with the queries each serves, indexes to drop, query rewrites, tradeoffs, and how to verify with EXPLAIN before and after (see `references/explain-reading-guide.md`). Follow the style of `examples/example-shop-orders-review.md`.
## Rules
- Never claim a speed-up without saying how to measure it; give expected plan changes ("index scan instead of sequential scan and sort"), not invented timings.
- Use safe build commands for production: `CREATE INDEX CONCURRENTLY` in PostgreSQL, online DDL (`ALGORITHM=INPLACE, LOCK=NONE`) in MySQL, and a maintenance window for SQLite.
- Do not recommend an index on a low-selectivity column alone (status, booleans); use it inside a composite or as a partial index condition.
- Keep primary keys and unique constraints; never suggest dropping them to remove "redundancy".
- Tell the user to build one index at a time and compare plans, and to remove indexes that EXPLAIN never uses.
FILE:references/indexing-rules.md
# Indexing rules
The rules the helper applies, plus the judgment it cannot make. Examples use PostgreSQL syntax unless noted.
## 1. Column order in a composite index
A B-tree index is sorted by its first column, then the second, and so on. A query can seek on a leftmost run of columns compared with equality, then use one more column for a range or for sorting.
Order: **equality columns, then sort columns, then one range column**.
```sql
-- WHERE customer_id = ? AND status = 'paid' ORDER BY created_at DESC LIMIT 20
CREATE INDEX ON orders (customer_id, status, created_at);
```
- Among equality columns, put the one used by the most queries first so the index serves them all; selectivity matters less than reuse once all of them are equalities.
- After a range column (`>`, `<`, `BETWEEN`, `LIKE 'abc%'`), later columns cannot be used for seeking, only for filtering inside the index.
- A sort can use the index only if every ORDER BY column comes from the index in the same order and directions are all the same (or all reversed). Mixed directions need a matching index definition (`created_at DESC, id ASC`).
- With a range and an ORDER BY on a different column, the database must choose: seek with the range and sort, or walk the sort order and filter. With LIMIT, walking the sort order often wins.
## 2. Leftmost prefix and merging
An index on `(a, b, c)` serves queries on `a`, on `a, b`, and on `a, b, c`, but not on `b` or `c` alone. Two candidates where one is a leftmost prefix of the other become one index. An existing index `(a)` is redundant once `(a, b)` exists, unless it is unique, partial, or much smaller and very hot.
## 3. Joins and foreign keys
- Index the column on the side that is looked up: usually the foreign key (`orders.customer_id`). The primary key side is already indexed.
- PostgreSQL does not index foreign keys automatically (MySQL InnoDB does). Missing FK indexes also make deletes on the parent table slow.
- A join column plus that table's own filters can form one composite: `(customer_id, created_at)`.
## 4. Covering indexes
If the index holds every column the query reads, the database can skip the table (index-only scan).
- PostgreSQL 11+ and SQL Server: `INCLUDE (col, ...)` stores extra columns without sorting by them.
- MySQL and SQLite: append the columns to the key; InnoDB secondary indexes already carry the primary key.
- Use it for hot, read-mostly queries with few selected columns. Wide covering indexes are big and slow to write.
- PostgreSQL index-only scans also need a well-vacuumed table (visibility map).
## 5. Partial and expression indexes
- Partial (PostgreSQL, SQLite; filtered index in SQL Server): `CREATE INDEX ON orders (created_at) WHERE status = 'paid' AND shipped_at IS NULL;` Tiny and fast when the condition matches few rows. The query must repeat the same condition.
- Expression: `CREATE INDEX ON users (lower(email));` serves `WHERE lower(email) = ?`. MySQL 8 supports functional indexes `((lower(email)))` or a generated column with an index.
## 6. Sargability: filters an index cannot use
| Pattern | Problem | Fix |
|---|---|---|
| `WHERE date(created_at) = '2026-10-12'` | function on the column | `created_at >= '2026-10-12' AND created_at < '2026-10-13'` |
| `WHERE YEAR(updated_at) = 2026` | function on the column | range on the column |
| `WHERE price * 1.2 > 100` | arithmetic on the column | `price > 100 / 1.2` |
| `WHERE name LIKE '%lamp%'` | leading wildcard | full-text search, or `pg_trgm` GIN index |
| `WHERE lower(name) = ?` | function | expression index on `lower(name)` or a case-insensitive collation |
| `WHERE a = 1 OR b = 2` | OR across columns | `UNION` of two indexed queries, or one index per column (bitmap OR in PostgreSQL) |
| `WHERE status <> 'done'`, `NOT IN` | negative filter | positive list `IN (...)` or a partial index |
| `WHERE id = '42'` on an integer column | implicit cast (MySQL may skip the index on string columns compared to numbers) | compare with the right type |
## 7. Over-indexing
- Every index is updated on INSERT, on DELETE, and on UPDATE of its columns; it also uses disk, memory (buffer cache), and backup and replication bandwidth.
- Duplicates (same columns, same condition) are pure cost: drop one.
- Single-column indexes on low-selectivity columns (booleans, status, type) are rarely used; the planner prefers a scan when a value matches more than about 5 to 20 percent of rows. Use them inside composites or as partial-index conditions.
- As a rule of thumb, more than 5 or 6 indexes on a write-heavy table deserves a review; read-mostly tables can carry more.
- Find unused indexes before dropping: PostgreSQL `pg_stat_user_indexes.idx_scan = 0` (since the last stats reset, on every replica), MySQL `sys.schema_unused_indexes`. Drop with `DROP INDEX CONCURRENTLY` in PostgreSQL.
## 8. Building safely
- PostgreSQL: `CREATE INDEX CONCURRENTLY` avoids blocking writes; it cannot run inside a transaction and leaves an INVALID index on failure (drop and retry).
- MySQL 8: `ALTER TABLE ... ADD INDEX ..., ALGORITHM=INPLACE, LOCK=NONE`; watch replication lag on large tables.
- SQLite: index builds lock the database; run during low traffic.
- Build one index at a time, check the plan, then the next.
FILE:references/explain-reading-guide.md
# Reading EXPLAIN to confirm an index
Always compare the plan before and after, on production-like data and with realistic parameter values. Small test tables make the planner choose scans that are right for small tables.
## PostgreSQL
```sql
EXPLAIN (ANALYZE, BUFFERS) SELECT ...; -- runs the query; wrap writes in BEGIN ... ROLLBACK
```
| Plan node | Meaning |
|---|---|
| Seq Scan | reads the whole table; fine for small tables or when most rows match |
| Index Scan | seeks the index, then reads matching table rows |
| Index Only Scan | answered from the index; check "Heap Fetches" is low (vacuum if not) |
| Bitmap Index Scan + Bitmap Heap Scan | collects many matches, then reads pages in order; common for medium selectivity and OR |
| Sort (with "external merge") | sorting spilled to disk; an index in ORDER BY order can remove it |
| Rows Removed by Filter | rows read and thrown away; high numbers mean the index does not cover the filter |
Look at: actual time of the top node, rows estimated versus actual (a 10x gap means stale statistics: run `ANALYZE table`), and shared hit/read buffers. Confirm a new index is used with `pg_stat_user_indexes.idx_scan` after a day of traffic.
## MySQL 8
```sql
EXPLAIN FORMAT=TREE SELECT ...;
EXPLAIN ANALYZE SELECT ...; -- 8.0.18+, runs the query
```
In classic `EXPLAIN` output check: `type` (ALL = full scan, ref/eq_ref/range = index use, index = full index scan), `key` (the index chosen), `rows` (estimate), and `Extra` ("Using index" = covering, "Using filesort" = extra sort, "Using temporary" = temporary table for GROUP BY or DISTINCT). Refresh statistics with `ANALYZE TABLE t`.
## SQLite
```sql
EXPLAIN QUERY PLAN SELECT ...;
```
"SCAN t" is a full scan; "SEARCH t USING INDEX idx (a=? AND b>?)" shows which columns seek; "USING COVERING INDEX" means no table access; "USE TEMP B-TREE FOR ORDER BY" means an extra sort. Run `ANALYZE;` so the planner has statistics.
## What success looks like
- The slow query's plan switches from a full scan plus sort to an index (only) scan with no sort, and the rows read drop close to the rows returned.
- Writes on the table do not get noticeably slower (check p95 insert and update latency after deploying).
- No other important query changed plan for the worse; run the top queries by total time (`pg_stat_statements`, MySQL performance schema digest tables) before and after.
## When the index is not used
1. Statistics are stale: run ANALYZE.
2. The filter is not sargable (see indexing-rules.md section 6).
3. The value matches too many rows; the planner is right to scan.
4. Column types or collations differ between the query and the column (implicit cast).
5. A parameterized query uses a generic plan; test with the real prepared statement.
6. The index columns are in the wrong order for the query.
FILE:templates/index-review.md
# Index review: app_or_service
**Engine:** engine_and_version | **Tables:** tables_with_row_counts | **Write load:** write_profile
**Helper:** n_queries queries analyzed, n_suggested candidates, n_findings schema findings
## Summary
Two or three sentences: the main cause of slowness, the indexes to add and drop, and the expected plan change.
## Per-query diagnosis
| Query | Runs | Now (plan / time) | Problem | Fix |
|---|---|---|---|---|
| Q1 | per_minute | Seq Scan + Sort | | |
## Query rewrites (do these first)
```sql
-- before
-- after
```
## Indexes to add (build in this order)
| # | DDL | Serves | Why this column order | Cost |
|---|---|---|---|---|
| 1 | | | | |
## Indexes to drop (after checking usage)
| Index | Reason | Check first |
|---|---|---|
| | duplicate / redundant prefix / unused / low selectivity | `idx_scan` on primary and replicas |
## Net effect per table
| Table | Indexes before | After | Note |
|---|---|---|---|
## How to verify
1. Capture `EXPLAIN (ANALYZE, BUFFERS)` for each query before.
2. Build one index (`CONCURRENTLY` / online DDL), run `ANALYZE`, capture the plan again.
3. Expect: (plan change per query).
4. Watch write latency and replication lag for a day; then drop the indexes listed above.
## Caveats
- Suggestions come from query text and schema only; row counts and data skew can change the best choice.
FILE:examples/example-shop-orders-review.md
# Example: index review for an online shop
## User request
"Our PostgreSQL 16 shop database is getting slow. orders has 12 million rows and gets about 40 inserts a second, customers has 900k rows, products 20k. Here are the schema and the six queries that show up in pg_stat_statements. What indexes should we add, and do we have too many already?"
## Helper run
```
$ python3 scripts/suggest_indexes.py --schema shop-schema.sql slow-queries.sql
Schema: 4 table(s), 12 existing index(es) incl. keys | dialect: postgres
Queries analyzed: 7
Q1 SELECT: SELECT o.id, o.total, o.created_at FROM orders o WHERE o.customer_id = $1 AND o.status = 'paid' ORDER BY o.cre...
equality: orders.customer_id (=), orders.status (=)
order by: orders.created_at DESC (LIMIT)
Q2 SELECT: SELECT c.email, count(*) AS orders FROM customers c JOIN orders o ON o.customer_id = c.id WHERE c.country = 'D...
equality: customers.country (=)
range: orders.created_at (range)
joins: orders.customer_id = customers.id
group by: customers.email
Q3 SELECT: SELECT * FROM products WHERE lower(name) LIKE '%lamp%'
WARN non-sargable: lower(name) in the WHERE clause hides the column from a normal index; ... the LIKE pattern '%lamp%' also starts with a wildcard, so only full-text search or a trigram index can help
Q4 SELECT: ... WHERE oi.order_id IN (SELECT id FROM orders WHERE customer_id = 42 AND created_at > now() - interval '30 days')
note: subquery analyzed as Q5
Q6 UPDATE: UPDATE orders SET status = 'shipped', shipped_at = now() WHERE status = 'paid' AND shipped_at IS NULL AND crea...
Q7 SELECT: SELECT id FROM customers WHERE email = 'ann@example.com' OR phone = '+491701234567'
WARN OR across different columns (...): one composite index cannot serve both sides; consider UNION ALL of two indexed queries or one index per side
Suggested indexes (4):
1. CREATE INDEX CONCURRENTLY idx_customers_country_email ON customers (country, email);
2. CREATE INDEX CONCURRENTLY idx_orders_customer_id_created_at ON orders (customer_id, created_at);
serves: Q2, Q5 | overlap: same leading column as #3
3. CREATE INDEX CONCURRENTLY idx_orders_customer_id_status_created_at ON orders (customer_id, status, created_at) INCLUDE (id, total);
serves: Q1 | overlap: same leading column as #2
4. CREATE INDEX CONCURRENTLY idx_orders_status_shipped_at_created_at ON orders (status, shipped_at, created_at);
serves: Q6 | alternative: ... ON orders (status, created_at) WHERE shipped_at IS NULL;
Schema findings (5):
WARN redundant-prefix customers: idx_customers_country (country) is a leftmost prefix of idx_customers_country_created ...
WARN low-selectivity customers: idx_customers_active indexes only 'is_active' ...
HIGH duplicate-index orders: idx_orders_created and orders_created_at_idx have the same columns (created_at); drop one
WARN low-selectivity orders: idx_orders_status indexes only 'status' ...
WARN many-indexes orders: 4 existing + 3 suggested = 7 indexes (limit 6) ...
```
## Review (filled template, shortened)
**Summary:** orders has no index that starts with customer_id, so the order history page (Q1) and the recent-orders lookup (Q5) scan millions of rows, and two of its four indexes are a duplicate and a low-selectivity status index. Add one composite and one partial index on orders, a phone index and a trigram index elsewhere, and drop three indexes: orders ends with the same count of 4 but every one is useful.
| Query | Problem | Fix |
|---|---|---|
| Q1 order history | no index on customer_id; sort on created_at | (customer_id, created_at) with status and total included: newest rows for the customer, status filtered inside the index, LIMIT stops after 20 |
| Q2 country report | join needs orders.customer_id; customers.country is already served by idx_customers_country_created | same orders index; no new customers index (the helper's #1 only helps the GROUP BY, and the report runs once a night) |
| Q3 product search | `lower(name) LIKE '%lamp%'` cannot use a B-tree | pg_trgm GIN index on lower(name) |
| Q4/Q5 recent items | subquery needs (customer_id, created_at); order_items PK (order_id, product_id) serves the IN | same orders index |
| Q6 shipping job | scans by status | partial index on created_at for paid, unshipped orders only |
| Q7 support lookup | OR across email and phone | UNION of two lookups; email is unique already, add a phone index |
Why not the helper's #3 as well: #2 and #3 share customer_id, and on a table with 40 inserts a second one index is better than two. The user confirmed that about 95 percent of orders are paid, so Q1 throws away few rows, and (customer_id, created_at) with status in INCLUDE keeps Q1 fast. If the share of non-paid orders grows, switch to #3's column order.
**Indexes to add (in this order):**
```sql
CREATE INDEX CONCURRENTLY idx_orders_customer_created
ON orders (customer_id, created_at) INCLUDE (id, status, total); -- Q1, Q2, Q5
CREATE INDEX CONCURRENTLY idx_orders_paid_unshipped
ON orders (created_at) WHERE status = 'paid' AND shipped_at IS NULL; -- Q6, holds only paid, unshipped orders
CREATE INDEX CONCURRENTLY idx_customers_phone ON customers (phone); -- Q7 after the rewrite
CREATE EXTENSION IF NOT EXISTS pg_trgm;
CREATE INDEX CONCURRENTLY idx_products_name_trgm
ON products USING gin (lower(name) gin_trgm_ops); -- Q3, query unchanged
```
**Query rewrite (Q7):**
```sql
SELECT id FROM customers WHERE email = 'ann@example.com'
UNION
SELECT id FROM customers WHERE phone = '+491701234567';
```
**Indexes to drop (after checking `pg_stat_user_indexes.idx_scan` on the primary and both replicas):**
| Index | Reason |
|---|---|
| orders_created_at_idx | duplicate of idx_orders_created |
| idx_orders_status | low selectivity; Q6 now uses the partial index |
| idx_customers_country | leftmost prefix of idx_customers_country_created |
| idx_customers_active (check) | boolean; keep only if idx_scan shows real use |
**Net effect:** orders 4 -> 4 indexes (two replaced by better ones), customers 5 -> 5 (one dropped, phone added; 4 if is_active goes too), products 2 -> 3.
**Recheck with the helper** (schema after the change, queries after the rewrite; query numbers follow the new file, so Q3 is the old Q5 and Q4/Q5 are the two UNION branches):
```
$ python3 scripts/suggest_indexes.py --schema shop-schema-after.sql shop-queries-after.sql --no-covering
Already served by existing indexes (5):
customers (email) -> customers_email_key (Q4)
customers (id, country) -> customers_pkey (Q2)
customers (phone) -> idx_customers_phone (Q5)
orders (created_at) -> idx_orders_created (Q2)
orders (customer_id, created_at) -> idx_orders_customer_created (Q2, Q3)
```
The helper still lists (customer_id, status, created_at) for Q1 and (country, email) for Q2; both were rejected on purpose for the reasons above. There are no duplicate or redundant-prefix findings left.
**How to verify:** before and after each build, run `EXPLAIN (ANALYZE, BUFFERS)` for Q1 with a real customer id. Expect "Index Scan using idx_orders_customer_created" (or Index Only Scan) with no Sort node and about 20 rows read instead of a Bitmap Heap Scan over thousands. Watch insert latency on orders for a day before dropping the old indexes.
FILE:scripts/suggest_indexes.py
#!/usr/bin/env python3
"""Suggest candidate indexes for slow SQL queries from a schema and the query text (no database needed).
Usage:
python3 suggest_indexes.py --schema schema.sql queries.sql [options]
python3 suggest_indexes.py all-in-one.sql (CREATE statements and queries in one file)
python3 suggest_indexes.py --schema schema.sql - < slow.sql
Options:
--schema FILE CREATE TABLE / CREATE INDEX statements (may also be inside the query file)
--dialect NAME postgres (default), mysql, or sqlite; changes INCLUDE and online-build hints
--max-indexes N warn when a table would have more than N indexes (default 6)
--no-covering do not suggest covering (INCLUDE) columns
--json print the result as JSON
--strict exit 1 if any query has a non-sargable predicate or the schema has duplicate indexes
What it does: lightweight parsing of SELECT, UPDATE, and DELETE statements (subqueries are analyzed
separately) to find equality filters, range filters, join columns, ORDER BY and GROUP BY columns,
and selected columns per table. It builds composite candidates (equality columns, then sort, then
range), merges candidates that are prefixes of each other, skips those already served by an existing
index, and reports duplicate and redundant indexes, low-selectivity single-column indexes, and tables
with many indexes. It does not know row counts or data distribution: confirm every suggestion with
EXPLAIN on real data.
Exit codes: 0 ok, 1 --strict findings, 2 usage or read error.
Python 3 standard library only.
"""
import json
import re
import signal
import sys
if hasattr(signal, "SIGPIPE"):
signal.signal(signal.SIGPIPE, signal.SIG_DFL) # quiet exit when piped into head
KEYWORDS = set("""select from where and or not in is null like ilike between join inner left right full outer cross on
using group by order having limit offset as asc desc distinct union all exists case when then else end update set
delete insert into values returning with true false interval current_date current_timestamp now lateral natural
fetch first rows only for share nowait skip locked""".split())
LOW_CARD = re.compile(r"^(is_|has_|can_|should_)|^(status|state|type|kind|flag|active|enabled|deleted|archived|gender|"
r"visible|published|verified|priority|level|tier|role)$", re.I)
CLAUSE_END = r"(?=\b(?:where|group\s+by|order\s+by|having|limit|offset|union|returning|for\s+update|window)\b|$)"
class UsageError(Exception):
pass
# ---------------------------------------------------------------- text helpers
def strip_comments(sql):
sql = re.sub(r"/\*.*?\*/", " ", sql, flags=re.S)
return re.sub(r"--[^\n]*", " ", sql)
def split_statements(sql):
out, buf, quote = [], [], None
for ch in sql:
if quote:
buf.append(ch)
if ch == quote:
quote = None
continue
if ch in "'\"`":
quote = ch
if ch == ";":
out.append("".join(buf))
buf = []
else:
buf.append(ch)
out.append("".join(buf))
return [s.strip() for s in out if s.strip()]
def unquote_idents(sql):
sql = re.sub(r'"([A-Za-z_][\w$]*)"', r"\1", sql)
sql = re.sub(r"`([A-Za-z_][\w$]*)`", r"\1", sql)
return re.sub(r"\[([A-Za-z_][\w$]*)\]", r"\1", sql)
def mask_strings(sql):
"""Replace string literals with placeholders, remembering LIKE patterns."""
lits = []
def rep(m):
lits.append(m.group(0)[1:-1])
return f" __lit{len(lits) - 1}__ "
return re.sub(r"'(?:[^']|'')*'", rep, sql), lits
def split_top(text, sep_regex):
"""Split on a keyword at parenthesis depth 0."""
parts, depth, last = [], 0, 0
for m in re.finditer(r"\(|\)|" + sep_regex, text, flags=re.I):
tok = m.group(0)
if tok == "(":
depth += 1
elif tok == ")":
depth -= 1
elif depth == 0:
parts.append(text[last:m.start()])
last = m.end()
parts.append(text[last:])
return [p.strip() for p in parts if p.strip()]
def split_and(text):
"""Split on top-level AND, keeping 'x BETWEEN a AND b' together."""
parts = split_top(text, r"\band\b")
out = []
for part in parts:
if out and re.search(r"\bbetween\s+[^\s]+$", out[-1], re.I):
out[-1] = f"{out[-1]} AND {part}"
else:
out.append(part)
return out
def extract_subqueries(sql):
"""Pull out innermost (SELECT ...) blocks; return (outer_sql, [subqueries])."""
subs = []
pattern = re.compile(r"\(\s*(select\b[^()]*(?:\([^()]*\)[^()]*)*)\)", re.I)
while True:
m = pattern.search(sql)
if not m:
break
subs.append(m.group(1))
sql = sql[:m.start()] + " __subq__ " + sql[m.end():]
return sql, subs
# ---------------------------------------------------------------- schema
def parse_columns_list(text):
return [re.sub(r"\s+(asc|desc)\b.*$", "", c.strip(), flags=re.I).strip().lower()
for c in split_top(text, r",")]
def parse_schema(statements):
tables, indexes = {}, []
for st in statements:
s = unquote_idents(strip_comments(st)).strip()
m = re.match(r"create\s+table\s+(?:if\s+not\s+exists\s+)?([\w.]+)\s*\((.*)\)\s*[^)]*$", s, re.I | re.S)
if m:
name = m.group(1).split(".")[-1].lower()
cols, pk = [], None
for part in split_top(m.group(2), r","):
p = part.strip()
low = p.lower()
c = re.match(r"(?:constraint\s+\w+\s+)?primary\s+key\s*\(([^)]*)\)", low)
if c:
pk = parse_columns_list(c.group(1))
continue
c = re.match(r"(?:constraint\s+(\w+)\s+)?unique\s*(?:key\s+\w+\s*)?\(([^)]*)\)", low)
if c:
indexes.append({"name": c.group(1) or f"{name}_unique", "table": name,
"columns": parse_columns_list(c.group(2)), "unique": True, "kind": "unique"})
continue
c = re.match(r"(?:key|index)\s+(\w+)\s*\(([^)]*)\)", low)
if c:
indexes.append({"name": c.group(1), "table": name, "columns": parse_columns_list(c.group(2)),
"unique": False, "kind": "index"})
continue
if re.match(r"(constraint|foreign\s+key|check|exclude)\b", low):
continue
col = re.match(r"([a-z_][\w$]*)\s+", low + " ")
if col:
cols.append(col.group(1))
if re.search(r"\bprimary\s+key\b", low):
pk = [col.group(1)]
elif re.search(r"\bunique\b", low):
indexes.append({"name": f"{name}_{col.group(1)}_key", "table": name,
"columns": [col.group(1)], "unique": True, "kind": "unique"})
tables[name] = {"columns": cols}
if pk:
indexes.append({"name": f"{name}_pkey", "table": name, "columns": pk, "unique": True,
"kind": "primary"})
continue
m = re.match(r"create\s+(unique\s+)?index\s+(?:concurrently\s+)?(?:if\s+not\s+exists\s+)?(\w+)\s+on\s+"
r"([\w.]+)\s*(?:using\s+\w+\s*)?\(([^;]*?)\)\s*(include\s*\(([^)]*)\))?\s*(where\s+.*)?$",
s, re.I | re.S)
if m:
cols = parse_columns_list(m.group(4))
indexes.append({"name": m.group(2), "table": m.group(3).split(".")[-1].lower(), "columns": cols,
"unique": bool(m.group(1)), "kind": "index",
"include": parse_columns_list(m.group(6)) if m.group(6) else [],
"partial": (m.group(7) or "").strip() or None})
return tables, indexes
# ---------------------------------------------------------------- queries
def parse_from(from_text):
"""Return (tables {alias: table}, order list, join predicates text list)."""
aliases, order, ons = {}, [], []
pieces = re.split(r"\b((?:natural\s+)?(?:inner\s+|left\s+(?:outer\s+)?|right\s+(?:outer\s+)?|full\s+(?:outer\s+)?|"
r"cross\s+)?join)\b", from_text, flags=re.I)
items = [pieces[0]] + [pieces[i + 1] for i in range(1, len(pieces) - 1, 2)]
for item in items:
on_split = re.split(r"\bon\b", item, maxsplit=1, flags=re.I)
using = re.search(r"\busing\s*\(([^)]*)\)", on_split[0], re.I)
head = re.sub(r"\busing\s*\([^)]*\)", "", on_split[0], flags=re.I)
for ref in head.split(","):
toks = [t for t in re.split(r"\s+", ref.strip()) if t and t.lower() not in ("as", "lateral", "only")]
if not toks or toks[0] == "__subq__":
continue
table = toks[0].split(".")[-1].lower()
alias = toks[1].lower() if len(toks) > 1 else table
aliases[alias] = table
order.append(alias)
if len(on_split) > 1:
ons.append(on_split[1])
if using and len(order) >= 2:
for col in parse_columns_list(using.group(1)):
ons.append(f"{order[-2]}.{col} = {order[-1]}.{col}")
return aliases, order, ons
COL = r"((?:[a-z_][\w$]*\.)?[a-z_][\w$]*)"
VALUE = r"(?:__lit\d+__|\?|:\w+|\$\d+|@\w+|%s|-?\d+(?:\.\d+)?|true|false|null|__subq__|current_date|current_timestamp|now\(\))"
def resolve(ref, aliases, tables):
ref = ref.lower()
if "." in ref:
a, c = ref.split(".", 1)
return (aliases.get(a, a), c)
if len(set(aliases.values())) == 1:
return (next(iter(aliases.values())), ref)
owners = [t for t in set(aliases.values()) if ref in tables.get(t, {}).get("columns", [])]
return (owners[0], ref) if len(owners) == 1 else (None, ref)
def analyze_predicate(pred, lits, aliases, tables, q):
p = pred.strip()
while p.startswith("(") and p.endswith(")") and len(split_top(p[1:-1], r"\band\b")) >= 1 and \
p.count("(") == p.count(")") and _balanced_wrap(p):
p = p[1:-1].strip()
if len(split_and(p)) > 1:
for sub in split_and(p):
analyze_predicate(sub, lits, aliases, tables, q)
return
ors = split_top(p, r"\bor\b")
if len(ors) > 1:
cols = set()
for o in ors:
m = re.match(rf"^\(?\s*{COL}\s*(=|in\b)", o.strip(), re.I)
cols.add(m.group(1).lower() if m else None)
if len(cols) == 1 and None not in cols:
t, c = resolve(cols.pop(), aliases, tables)
q["filters"].append({"table": t, "column": c, "kind": "eq", "op": "OR of equalities (same as IN)"})
else:
shown = re.sub(r"__lit(\d+)__", lambda m: "'" + lits[int(m.group(1))] + "'", p)
shown = shown if len(shown) <= 70 else shown[:67] + "..."
q["warnings"].append(f"OR across different columns ({shown}): one composite index cannot serve "
"both sides; consider UNION ALL of two indexed queries or one index per side")
return
if re.match(r"^not\b", p, re.I) or re.search(r"(<>|!=|\bnot\s+in\b|\bnot\s+like\b)", p, re.I):
m = re.match(rf"^(?:not\s+)?{COL}", p, re.I)
shown = re.sub(r"__lit(\d+)__", lambda m: "'" + lits[int(m.group(1))] + "'", p)[:50]
q["notes"].append(f"negative predicate '{shown}' rarely uses an index; it is applied as a filter")
return
m = re.match(rf"^{COL}\s*(=|<=>)\s*{COL}$", p, re.I)
if m and not re.fullmatch(VALUE, m.group(3), re.I) and m.group(3).lower() not in KEYWORDS:
left, right = resolve(m.group(1), aliases, tables), resolve(m.group(3), aliases, tables)
if left[0] and right[0] and left[0] != right[0]:
q["joins"].append((left, right))
return
m = re.match(rf"^{VALUE}\s*(=|<|>|<=|>=)\s*{COL}$", p, re.I)
if m:
flip = {"<": ">", ">": "<", "<=": ">=", ">=": "<=", "=": "="}
p = f"{m.group(2)} {flip[m.group(1)]} __v__"
m = re.match(rf"^{COL}\s*(=|<=>)\s*(.+)$", p, re.I)
if m and m.group(1).lower() not in KEYWORDS:
t, c = resolve(m.group(1), aliases, tables)
q["filters"].append({"table": t, "column": c, "kind": "eq", "op": "="})
return
m = re.match(rf"^{COL}\s+in\s+__subq__$", p, re.I)
if m:
t, c = resolve(m.group(1), aliases, tables)
q["filters"].append({"table": t, "column": c, "kind": "eq", "op": "IN (subquery)"})
return
m = re.match(rf"^{COL}\s+in\s*\((.*)\)$", p, re.I | re.S)
if m:
t, c = resolve(m.group(1), aliases, tables)
q["filters"].append({"table": t, "column": c, "kind": "eq", "op": "IN"})
return
m = re.match(rf"^{COL}\s+is\s+(not\s+)?null$", p, re.I)
if m:
t, c = resolve(m.group(1), aliases, tables)
if m.group(2):
q["notes"].append(f"'{c} IS NOT NULL' is usually not selective; a partial index can help if most rows are NULL")
else:
q["filters"].append({"table": t, "column": c, "kind": "eq", "op": "IS NULL"})
return
m = re.match(rf"^{COL}\s*(<|>|<=|>=)\s*(.+)$", p, re.I) or re.match(rf"^{COL}\s+between\s+(.+)\s+and\s+(.+)$", p, re.I)
if m and m.group(1).lower() not in KEYWORDS:
t, c = resolve(m.group(1), aliases, tables)
q["filters"].append({"table": t, "column": c, "kind": "range", "op": "range"})
return
m = re.match(rf"^{COL}\s+(i?like)\s+__lit(\d+)__", p, re.I)
if m:
t, c = resolve(m.group(1), aliases, tables)
pattern = lits[int(m.group(3))]
if pattern.startswith(("%", "_")):
q["warnings"].append(f"non-sargable: {c} {m.group(2).upper()} '{pattern}' starts with a wildcard, so a "
"B-tree index cannot seek; use full-text search or a trigram index (PostgreSQL pg_trgm)")
q["non_sargable"] = True
else:
q["filters"].append({"table": t, "column": c, "kind": "range", "op": f"LIKE '{pattern}'"})
if m.group(2).lower() == "ilike":
q["notes"].append(f"ILIKE on {c}: a plain B-tree index is case-sensitive; index lower({c}) instead")
return
m = re.match(rf"^([a-z_]+)\s*\(\s*{COL}[^)]*\)\s*(=|<|>|<=|>=|like|between)", p, re.I)
if m and m.group(1).lower() not in KEYWORDS:
func, col = m.group(1).lower(), m.group(2)
t, c = resolve(col, aliases, tables)
wild = re.search(r"like\s+__lit(\d+)__", p, re.I)
extra = ""
if wild and lits[int(wild.group(1))].startswith(("%", "_")):
extra = (f"; the LIKE pattern '{lits[int(wild.group(1))]}' also starts with a wildcard, so only full-text "
"search or a trigram index can help")
q["warnings"].append(f"non-sargable: {func}({c}) in the WHERE clause hides the column from a normal index; "
f"rewrite as a range on {c} or add an expression index on {func}({c}){extra}")
q["non_sargable"] = True
return
m = re.match(rf"^{COL}\s*[-+*/]\s*\S+\s*(=|<|>|<=|>=)", p, re.I)
if m:
t, c = resolve(m.group(1), aliases, tables)
q["warnings"].append(f"non-sargable: arithmetic on {c} ('{p[:40]}'); move the arithmetic to the other side")
q["non_sargable"] = True
return
if "__subq__" in p and re.search(r"\bexists\b", p, re.I):
return
q["notes"].append(f"predicate not understood, check by hand: {p[:60]}")
def _balanced_wrap(p):
depth = 0
for i, ch in enumerate(p):
depth += ch == "("
depth -= ch == ")"
if depth == 0 and i < len(p) - 1:
return False
return True
def parse_query(raw, tables, number, lits=None, outer_aliases=None):
if lits is None:
sql, lits = mask_strings(unquote_idents(strip_comments(raw)))
else:
sql = raw
sql = re.sub(r"\s+", " ", sql).strip()
outer, subs = extract_subqueries(sql)
shown = re.sub(r"__lit(\d+)__", lambda m: "'" + lits[int(m.group(1))] + "'", re.sub(r"\s+", " ", raw.strip()))
shown = re.sub(r"\s+", " ", shown)
q = {"n": number, "text": shown[:110] + ("..." if len(shown) > 110 else ""), "kind": None, "aliases": {},
"filters": [], "joins": [], "order_by": [], "group_by": [], "selected": {}, "select_star": False,
"limit": False, "warnings": [], "notes": [], "non_sargable": False, "subqueries": subs, "lits": lits}
low = outer.lower()
cte_names = set()
if low.startswith("with "):
cte_names = {n.lower() for n in re.findall(r"(?:with|,)\s*(?:recursive\s+)?(\w+)\s+as\b", outer, re.I)}
q["notes"].append(f"CTE {', '.join(sorted(cte_names))} analyzed as separate queries; the final SELECT reads from it")
m = re.search(r"(?:\)|__subq__)\s*((?:select|update|delete)\b.*)$", outer, re.I)
outer = m.group(1) if m else outer
low = outer.lower()
if low.startswith("select"):
q["kind"] = "SELECT"
m = re.match(r"select\s+(?:distinct\s+)?(.*?)\s+from\s+(.*?)" + CLAUSE_END, outer, re.I)
if not m:
q["notes"].append("no FROM clause found")
return q
select_list, from_text = m.group(1), m.group(2)
elif low.startswith("update"):
q["kind"] = "UPDATE"
m = re.match(r"update\s+(.*?)\s+set\s+", outer, re.I)
select_list, from_text = "", m.group(1) if m else ""
elif low.startswith("delete"):
q["kind"] = "DELETE"
m = re.match(r"delete\s+from\s+(.*?)" + CLAUSE_END, outer, re.I)
select_list, from_text = "", m.group(1) if m else ""
else:
return None
own, _order, ons = parse_from(from_text)
own = {a: t for a, t in own.items() if t not in cte_names}
q["aliases"] = own
aliases = {**(outer_aliases or {}), **own} # outer aliases resolve correlated references
unknown = [t for t in set(own.values()) if tables and t not in tables]
if unknown:
q["notes"].append(f"table(s) not in the schema: {', '.join(sorted(unknown))}; existing indexes unknown")
for on in ons:
for pred in split_and(on):
analyze_predicate(pred, lits, aliases, tables, q)
w = re.search(r"\bwhere\b(.*?)(?=\b(?:group\s+by|order\s+by|having|limit|offset|returning|for\s+update)\b|$)", outer, re.I)
if w:
for pred in split_and(w.group(1)):
analyze_predicate(pred, lits, aliases, tables, q)
for clause, key in (("group\\s+by", "group_by"), ("order\\s+by", "order_by")):
m = re.search(rf"\b{clause}\b(.*?)(?=\b(?:having|order\s+by|limit|offset|for\s+update)\b|$)", outer, re.I)
if m:
for item in split_top(m.group(1), r","):
mm = re.match(rf"^{COL}\s*(asc|desc)?(?:\s+nulls\s+(?:first|last))?$", item.strip(), re.I)
if mm:
t, c = resolve(mm.group(1), aliases, tables)
q[key].append({"table": t, "column": c, "dir": (mm.group(2) or "asc").lower()})
else:
q["notes"].append(f"{key.replace('_', ' ')} expression '{item.strip()[:40]}' cannot use a plain column index")
q["limit"] = bool(re.search(r"\b(limit|fetch\s+first)\b", outer, re.I))
if select_list:
if re.search(r"(^|,)\s*(\w+\.)?\*\s*(,|$)", select_list):
q["select_star"] = True
for ref in re.findall(rf"\b{COL}\b(?!\s*\()", select_list, re.I):
if ref.lower() in KEYWORDS or ref.startswith("__") or re.fullmatch(r"\d+", ref):
continue
t, c = resolve(ref, aliases, tables)
if t and (not tables or c in tables.get(t, {}).get("columns", [c])):
q["selected"].setdefault(t, [])
if c not in q["selected"][t]:
q["selected"][t].append(c)
for f in q["filters"]:
if f["table"] is None:
q["notes"].append(f"column '{f['column']}' could belong to several tables; qualify it with an alias")
return q
# ---------------------------------------------------------------- candidates
def candidates_for(q, dialect, covering, pk_cols=None):
out = []
by_table = {}
for f in q["filters"]:
if f["table"]:
by_table.setdefault(f["table"], {"eq": [], "range": []})
lst = by_table[f["table"]][f["kind"]]
if f["column"] not in lst:
lst.append(f["column"])
tables_in_q = set(q["aliases"].values())
correlated = set()
for left, right in q["joins"]:
if left[0] in tables_in_q and right[0] not in tables_in_q:
correlated.add(left[0])
if right[0] in tables_in_q and left[0] not in tables_in_q:
correlated.add(right[0])
order_tables = {o["table"] for o in q["order_by"]}
for t in tables_in_q:
if t in correlated:
continue # reached through the correlated join below
eq = by_table.get(t, {}).get("eq", [])
rng = [c for c in by_table.get(t, {}).get("range", []) if c not in eq]
sort = []
if q["order_by"] and order_tables == {t} and len({o["dir"] for o in q["order_by"]}) == 1:
sort = [o["column"] for o in q["order_by"] if o["column"] not in eq]
elif q["group_by"] and {g["table"] for g in q["group_by"]} == {t}:
sort = [g["column"] for g in q["group_by"] if g["column"] not in eq]
key, why = list(eq), []
if eq:
why.append(f"equality on {', '.join(eq)} first")
if sort and (not rng or q["limit"]):
key += [c for c in sort if c not in key]
purpose = "already sorted" if q["order_by"] else "already grouped"
why.append(f"then {', '.join(sort)} so rows come out {purpose}" + (" (stops early with LIMIT)" if q["limit"] else ""))
if rng and rng[0] not in key:
key.append(rng[0])
why.append(f"{rng[0]} last as an in-index filter")
elif rng:
key.append(rng[0])
why.append(f"then the range on {rng[0]}" if eq else f"range on {rng[0]}")
if len(rng) > 1:
q["notes"].append(f"several range filters on {t} ({', '.join(rng)}): only the first can seek; the rest filter")
if sort:
q["notes"].append(f"ORDER BY on {t} cannot use the same index as the range on {rng[0]}; expect a sort step")
if not key:
continue
include = []
if covering and not q["select_star"] and q["kind"] == "SELECT":
extra = [c for c in q["selected"].get(t, []) if c not in key
and not (dialect == "mysql" and c in (pk_cols or {}).get(t, []))]
if 0 < len(extra) <= 3:
include = extra
out.append({"table": t, "columns": key, "include": include, "why": "; ".join(why), "queries": [q["n"]],
"source": "filter", "eq": list(eq),
"null_filter": [f["column"] for f in q["filters"] if f["table"] == t and f["op"] == "IS NULL"]})
for left, right in q["joins"]:
for (t, c), (ot, oc) in ((left, right), (right, left)):
if t not in tables_in_q:
continue # outer table of a correlated subquery
eq = [e for e in by_table.get(t, {}).get("eq", []) if e != c]
rng = [r for r in by_table.get(t, {}).get("range", []) if r != c and r not in eq][:1]
why = f"join column {t}.{c} = {ot}.{oc} first (lets the planner look up matching rows)"
if eq or rng:
why += f", then {', '.join(eq + rng)} to filter inside the index"
if t in correlated:
why = f"correlated lookup {t}.{c} = outer {ot}.{oc} first" + (f", then {', '.join(eq + rng)}" if eq or rng else "")
out.append({"table": t, "columns": [c] + eq + rng, "include": [], "queries": [q["n"]], "source": "join",
"eq": [c] + eq, "why": why})
return out
def served_by(cand, indexes):
for ix in indexes:
if ix["table"] != cand["table"]:
continue
cols = ix["columns"]
if cand["source"] == "join" and ix["unique"] and cols == cand["columns"][:1] and not ix.get("partial"):
return ix # joining on a unique key: one row per lookup, extra columns add little
if cols[:len(cand["columns"])] == cand["columns"]:
return ix
n_eq = len(cand.get("eq", []))
if n_eq > 1 and len(cols) >= len(cand["columns"]) and set(cols[:n_eq]) == set(cand["columns"][:n_eq]) \
and cols[n_eq:len(cand["columns"])] == cand["columns"][n_eq:]:
return ix # same equality columns in another order serve the query equally well
return None
def served_by_named(c, indexes):
ix = served_by(c, indexes)
return {"existing": ix["name"] if ix else "?"}
def merge(cands):
merged = []
for c in sorted(cands, key=lambda c: -len(c["columns"])):
for m in merged:
if m["table"] == c["table"] and m["columns"][:len(c["columns"])] == c["columns"]:
m["queries"] = sorted(set(m["queries"]) | set(c["queries"]))
m["include"] = m["include"] or []
break
else:
merged.append(dict(c))
return sorted(merged, key=lambda c: (c["table"], c["columns"]))
def schema_findings(indexes, max_indexes, new_by_table):
out = []
by_table = {}
for ix in indexes:
by_table.setdefault(ix["table"], []).append(ix)
for t, ixs in by_table.items():
for i, a in enumerate(ixs):
for b in ixs[i + 1:]:
if a["columns"] == b["columns"] and (a.get("partial") == b.get("partial")):
out.append({"level": "high", "rule": "duplicate-index",
"message": f"{t}: {a['name']} and {b['name']} have the same columns ({', '.join(a['columns'])}); drop one"})
elif a["columns"] != b["columns"]:
short, long_ = (a, b) if len(a["columns"]) < len(b["columns"]) else (b, a)
if long_["columns"][:len(short["columns"])] == short["columns"] and not short["unique"] \
and not short.get("partial"):
out.append({"level": "warn", "rule": "redundant-prefix",
"message": f"{t}: {short['name']} ({', '.join(short['columns'])}) is a leftmost prefix of "
f"{long_['name']} ({', '.join(long_['columns'])}); usually droppable"})
for ix in ixs:
if len(ix["columns"]) == 1 and LOW_CARD.search(ix["columns"][0]) and ix["kind"] == "index" and not ix.get("partial"):
out.append({"level": "warn", "rule": "low-selectivity",
"message": f"{t}: {ix['name']} indexes only '{ix['columns'][0]}', which sounds like a flag or status "
"with few values; use a composite or partial index instead"})
total = len(ixs) + new_by_table.get(t, 0)
if total > max_indexes:
out.append({"level": "warn", "rule": "many-indexes",
"message": f"{t}: {len(ixs)} existing + {new_by_table.get(t, 0)} suggested = {total} indexes "
f"(limit {max_indexes}); every insert and update maintains all of them"})
return out
def ddl(c, dialect):
name = f"idx_{c['table']}_{'_'.join(c['columns'])}"[:63]
cols = ", ".join(c["columns"])
if c["include"]:
if dialect == "postgres":
return f"CREATE INDEX CONCURRENTLY {name} ON {c['table']} ({cols}) INCLUDE ({', '.join(c['include'])});"
cols += ", " + ", ".join(c["include"])
if dialect == "postgres":
return f"CREATE INDEX CONCURRENTLY {name} ON {c['table']} ({cols});"
if dialect == "mysql":
return f"ALTER TABLE {c['table']} ADD INDEX {name} ({cols}), ALGORITHM=INPLACE, LOCK=NONE;"
return f"CREATE INDEX {name} ON {c['table']} ({cols});"
# ---------------------------------------------------------------- main
def parse_args(argv):
opts = {"schema": None, "files": [], "dialect": "postgres", "max_indexes": 6, "covering": True,
"json": False, "strict": False}
i = 0
while i < len(argv):
a = argv[i]
if a in ("-h", "--help"):
print(__doc__)
sys.exit(0)
if a in ("--schema", "--dialect", "--max-indexes"):
if i + 1 >= len(argv):
raise UsageError(f"{a} needs a value")
v = argv[i + 1]
if a == "--schema":
opts["schema"] = v
elif a == "--dialect":
if v not in ("postgres", "mysql", "sqlite"):
raise UsageError("--dialect must be postgres, mysql, or sqlite")
opts["dialect"] = v
else:
try:
opts["max_indexes"] = int(v)
except ValueError:
raise UsageError(f"--max-indexes needs a whole number, got {v!r}")
i += 2
elif a == "--no-covering":
opts["covering"] = False; i += 1
elif a == "--json":
opts["json"] = True; i += 1
elif a == "--strict":
opts["strict"] = True; i += 1
elif a.startswith("--"):
raise UsageError(f"unknown option {a}")
else:
opts["files"].append(a); i += 1
if not opts["files"] and not opts["schema"]:
raise UsageError("give a query file (or - for stdin), optionally with --schema FILE")
return opts
def read(path):
if path == "-":
return sys.stdin.read()
with open(path, encoding="utf-8") as f:
return f.read()
def main(argv):
try:
opts = parse_args(argv)
schema_text = read(opts["schema"]) if opts["schema"] else ""
query_texts = [read(p) for p in opts["files"]]
except UsageError as e:
print(f"error: {e}", file=sys.stderr)
return 2
except OSError as e:
print(f"error: cannot read {e.filename}: {e.strerror}", file=sys.stderr)
return 2
statements = split_statements(strip_comments(schema_text))
query_stmts = []
for text in query_texts:
clean = strip_comments(text)
for st in split_statements(clean):
(statements if re.match(r"\s*create\s", st, re.I) else query_stmts).append(st)
tables, indexes = parse_schema(statements)
queries, n = [], 0
expanded = []
for st in query_stmts:
masked, _l = mask_strings(st)
if len(split_top(masked, r"\bunion(?:\s+all)?\b")) > 1: # a real top-level UNION, not one inside a string
expanded.extend(split_top(st, r"\bunion(?:\s+all)?\b")) # each branch is analyzed as its own query
else:
expanded.append(st)
for st in expanded:
pending = [(st, None, None)]
while pending:
cur, lits, outer = pending.pop(0)
n += 1
q = parse_query(cur, tables, n, lits, outer)
if q is None:
n -= 1
continue
queries.append(q)
for sub in q["subqueries"]:
pending.append((sub, q["lits"], {**(outer or {}), **q["aliases"]}))
q["notes"].append(f"subquery analyzed as Q{n + len(pending)}")
if not queries:
print("error: no SELECT, UPDATE, or DELETE statements found", file=sys.stderr)
return 2
pks = {ix["table"]: ix["columns"] for ix in indexes if ix["kind"] == "primary"}
cands = []
for q in queries:
cands += candidates_for(q, opts["dialect"], opts["covering"], pks)
covered, open_cands = [], []
for c in cands:
ix = served_by(c, indexes)
if ix:
covered.append({**c, "existing": ix["name"]})
else:
open_cands.append(c)
suggested = merge(open_cands)
covered = merge(covered)
for c in covered:
c["existing"] = next(x["existing"] for x in [served_by_named(c, indexes)])
new_by_table = {}
for c in suggested:
new_by_table[c["table"]] = new_by_table.get(c["table"], 0) + 1
c["ddl"] = ddl(c, opts["dialect"])
lead = c["columns"][0]
c["tradeoff"] = ("every INSERT, UPDATE of these columns, and DELETE on " + c["table"] +
" also writes this index; size grows with the row count and column widths")
if LOW_CARD.search(lead) and len(c["columns"]) == 1:
c["tradeoff"] += f"; '{lead}' looks low-selectivity, so the planner may still prefer a scan"
for a in suggested:
others = [k for k, b in enumerate(suggested, 1) if b is not a and b["table"] == a["table"]
and b["columns"][0] == a["columns"][0]]
a["overlaps"] = others
a["makes_redundant"] = [ix["name"] for ix in indexes if ix["table"] == a["table"] and not ix["unique"]
and ix["kind"] != "primary" and len(ix["columns"]) < len(a["columns"])
and a["columns"][:len(ix["columns"])] == ix["columns"]]
if a.get("null_filter") and opts["dialect"] == "postgres":
col = a["null_filter"][0]
rest = [c for c in a["columns"] if c != col]
name = f"idx_{a['table']}_{'_'.join(rest)}_{col}_null"[:63]
a["partial_hint"] = (f"CREATE INDEX CONCURRENTLY {name} ON {a['table']} ({', '.join(rest)}) "
f"WHERE {col} IS NULL; -- smaller if few rows match")
findings = schema_findings(indexes, opts["max_indexes"], new_by_table)
report = {"dialect": opts["dialect"], "tables": {t: len(v["columns"]) for t, v in tables.items()},
"existing_indexes": indexes, "queries": [{k: v for k, v in q.items() if k not in ("subqueries", "lits")}
for q in queries],
"suggested": suggested, "already_covered": covered, "schema_findings": findings}
for q in report["queries"]:
q["joins"] = [f"{a[0]}.{a[1]} = {b[0]}.{b[1]}" for a, b in q["joins"]]
strict_fail = opts["strict"] and (any(q["non_sargable"] for q in queries) or
any(f["rule"] == "duplicate-index" for f in findings))
if opts["json"]:
print(json.dumps(report, indent=2))
return 1 if strict_fail else 0
print(f"Schema: {len(tables)} table(s), {len(indexes)} existing index(es) incl. keys | dialect: {opts['dialect']}")
print(f"Queries analyzed: {len(queries)}")
for q in report["queries"]:
print(f"\nQ{q['n']} {q['kind']}: {q['text']}")
tabs = ", ".join(f"{t} {a}" if a != t else t for a, t in q["aliases"].items())
print(f" tables: {tabs or '-'}")
eqs = [f"{f['table'] or '?'}.{f['column']} ({f['op']})" for f in q["filters"] if f["kind"] == "eq"]
rngs = [f"{f['table'] or '?'}.{f['column']} ({f['op']})" for f in q["filters"] if f["kind"] == "range"]
if eqs:
print(f" equality: {', '.join(eqs)}")
if rngs:
print(f" range: {', '.join(rngs)}")
if q["joins"]:
print(f" joins: {', '.join(q['joins'])}")
if q["order_by"]:
print(" order by: " + ", ".join(f"{o['table']}.{o['column']} {o['dir'].upper()}" for o in q["order_by"])
+ (" (LIMIT)" if q["limit"] else ""))
if q["group_by"]:
print(" group by: " + ", ".join(f"{g['table']}.{g['column']}" for g in q["group_by"]))
for w in q["warnings"]:
print(f" WARN {w}")
for nt in q["notes"]:
print(f" note: {nt}")
print(f"\nSuggested indexes ({len(suggested)}):")
if not suggested:
print(" none: existing indexes already lead with the needed columns")
for k, c in enumerate(suggested, 1):
print(f" {k}. {c['ddl']}")
print(f" serves: {', '.join('Q' + str(x) for x in c['queries'])} | why: {c['why']}")
if c["include"]:
print(f" covering: {', '.join(c['include'])} added so the query can skip the table (optional)")
print(f" tradeoff: {c['tradeoff']}")
if c.get("overlaps"):
print(f" overlap: same leading column as #{', #'.join(map(str, c['overlaps']))}; "
"check whether one index can serve both before creating two")
if c.get("makes_redundant"):
print(f" makes redundant: {', '.join(c['makes_redundant'])} (leftmost prefix of this index; drop after checking usage)")
if c.get("partial_hint"):
print(f" alternative: {c['partial_hint']}")
if covered:
print(f"\nAlready served by existing indexes ({len(covered)}):")
for c in covered:
print(f" {c['table']} ({', '.join(c['columns'])}) -> {c['existing']} (Q{', Q'.join(map(str, c['queries']))})")
print(f"\nSchema findings ({len(findings)}):")
if not findings:
print(" none")
for f in findings:
print(f" {f['level'].upper():<4} {f['rule']:<17} {f['message']}")
print("\nNext: run EXPLAIN (ANALYZE, BUFFERS) or EXPLAIN on real data before and after; build one index at a time.")
return 1 if strict_fail else 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))Describe your region, season, skill level, transport, and target birds or habitats, and get a realistic weekend field plan: site shortlist with access notes, a dawn-to-dusk itinerary, packing and ethics checklist, ID tips for likely species, weather contingencies, and a simple log sheet template.
Act as a birdwatching weekend field trip planner. You have guided beginners and intermediate birders on day trips and overnight weekends across wetlands, forests, coasts, and urban parks. You plan around real dawn chorus times, tide tables when relevant, public access rules, and ethical viewing distances. You are encouraging, precise, and safety-minded: you never invent rare birds that are not plausible for the place and season, and you always put habitat and bird welfare ahead of a "life list" chase. About my trip: - Region / country and nearest town: southern England, near Salisbury - Season and dates: mid-April weekend, Saturday and Sunday - Experience level: beginner-intermediate; I know common garden birds, struggle with warblers and waders - Target birds or habitats (optional): reedbeds, farmland edges, and any spring migrants; would love to hear a nightingale - How I travel: car shared with one friend; can walk 8 to 10 km a day on easy paths - Start base / overnight: camping near the first reserve, or a B&B if rain is heavy - Budget feel: modest; free reserves preferred, happy to pay one paid reserve entry - Constraints: no early Friday departure; must be back by Sunday 18:00; friend has a mild knee issue so avoid steep climbs - Optics and kit I already have: 8x42 binoculars, phone with Merlin Bird ID, light rain jacket, no spotting scope - Weather forecast I have so far: cool, partly cloudy, light NW wind, 40% chance of showers Sunday afternoon Please do the following: 1. Start with a short trip read: 3 sentences on what this weekend is good for in my region and season, the 2 biggest constraints you see, and any assumption you are making about access or tides. 2. Shortlist 3 to 5 sites (reserves, parks, coastal stretches, or farmland viewpoints) with: habitat type, typical spring birds I can realistically expect, access notes (parking, toilets, path difficulty, fees), best time of day, and why it fits my level. 3. Build a dawn-to-dusk itinerary for Saturday and Sunday with travel times between sites, buffer for weather, and a clear "if X fails, go to Y" backup for each half-day. 4. Give a packing checklist: clothing layers, optics care, notebook or app setup, food and water, and items I should not bring (drones, playback speakers, bright colors near hides if relevant). 5. Cover field craft and ethics: approach distances, hide etiquette, when not to use birdcall playback, dogs/litter rules, and how to share a hide without spoiling someone else's view. 6. List 8 to 12 likely species for the planned habitats in this season, each with one field ID tip (song, silhouette, behavior, or habitat cue) suited to my level; mark which ones need a scope versus binoculars. 7. Add weather contingencies: wind, heavy rain, heat shimmer, and "everything is quiet" plans (sit-and-wait spots, urban backup parks). 8. Provide a simple log sheet template I can copy: date, site, start/end time, weather, species, count or abundance, notes, and a confidence column (certain / probable / heard-only). 9. Finish with a one-page summary: the chosen route, the 5 species most worth listening for, the 3 packing must-haves, and the single biggest etiquette rule for the weekend. Rules: - Prefer publicly accessible sites; if a site needs a permit or membership, say so and offer an alternative. - Do not invent rarity chases or private land access; label uncertain access as something to verify. - Keep advice practical for binoculars-only birding unless I said I have a scope. - Never recommend harassment, nest disturbance, baiting, or illegal recording of protected species. - If something essential is missing (tide times, reserve opening hours), make a sensible assumption and say so instead of asking many questions.
Describe your talk topic, audience, length, and takeaways, and get structured JSON with a narrative arc, timed slide outline, speaker notes cues, demo or story beats, audience interaction points, backup slides, and a rehearsal checklist.
1{2 "role": "You are a conference talk coach who has helped engineers, researchers, and practitioners turn messy expertise into clear 20 to 45 minute talks. You care about audience takeaways, honest timeboxing, and slides that support speaking rather than replace it. You never invent fake case studies, metrics, or employer claims the speaker did not provide.",3 "task": "Build a complete, timed slide outline for the conference talk described below: narrative arc, slide-by-slide plan with speaker-note cues, demo or story beats, audience interaction points, backup slides, and a rehearsal checklist, returned as JSON only.",4 "inputs": {5 "talk_title_working": "${title:Shipping Feature Flags Without Melting Prod}",6 "topic_and_angle": "${topic:practical lessons from rolling out flags at a mid-size SaaS; focus on kill switches, ownership, and what broke}",7 "audience": "${audience:mid-level backend and platform engineers; mixed experience with LaunchDarkly-style tools}",8 "length_minutes": "${length:30}",9 "format": "${format:single speaker, on-stage with slides, optional live CLI demo}",10 "must_cover": "${must:why flags fail in prod, a simple ownership model, one war story, three rules to take home}",...+95 more lines
Describe your target language goals, partner level, time box, and topics to avoid, and get a YAML session plan with warm-ups, timed conversation blocks, correction style, vocabulary harvest, homework micro-tasks, and a gentle feedback script, all conducted in English planning notes for practice that happens in the target language.
1role: >2 You are a language-exchange session coach. You help two people who are not professional teachers run a fair, useful conversation practice session.3 You plan in clear English so both partners know the structure, while the spoken practice happens in the target language(s).4 You keep sessions kind, timeboxed, and balanced so neither person monopolizes the hour.5task: >6 Build a complete language-exchange session plan as YAML for the inputs below: warm-ups, timed conversation blocks, correction agreements,7 vocabulary harvest, micro-homework, and a short feedback script.8inputs:9 my_native_language: "${native:English}"10 target_language: "${target:Spanish}"...+42 more lines

A two-color risograph print look of a packed subway carriage at night: fluorescent teal and warm red ink on cream paper, passengers as simplified shapes, overlapping misregistration, visible halftone dots, handrail reflections, and a quiet tired mood, framed as a finished print on a table.
A finished risograph print of a crowded night subway carriage, teal and warm brick-red ink only on soft cream paper, 16:9 landscape, the print filling most of the frame as if photographed flat on a pale desk. Key subjects first: standing passengers packed along a central aisle, silver handrails, overhead fluorescent strips, and seated figures along both benches, all built from flat limited colors with bold outlines. The carriage is simplified graphic shapes: far doors with circular windows, vertical poles, hanging straps, and one open phone glow as a small red rectangle. People are anonymous silhouettes and half-tones: coats, backpacks, a sleeping head against the window, a child gripping a pole. Print artifacts: slight teal-red misregistration on the poles, coarse halftone dots in the shadows, paper tooth, a faint second ink pass on the floor. Soft desk-photo framing with the cream sheet edge visible, gentle even light. Mood quiet and tired, late commute, graphic poster energy. Not photoreal, not 3D, not watercolor. No readable text, no logos, no watermark, no hands.

A photoreal studio shot of a round cloisonné enamel dish: two white and black cranes flying over a deep blue mountain lake, fine gold wire cells filled with saturated enamel, floral rim border, glossy kiln finish, on dark linen with soft side light.
A close studio photograph of a round cloisonné enamel dish centered in a 16:9 frame on dark charcoal linen. Key subject first: two Japanese-style cranes with white bodies, black necks, and red crowns flying left to right above a deep cobalt mountain lake, every shape outlined in thin raised gold wire cells filled with glossy enamel. Behind the birds, layered indigo and teal mountains, a pale cream cloud band, and a small island pine in dark green enamel. The dish rim is a continuous border of peony and chrysanthemum motifs in coral, gold, and jade green. Metalwork details: tiny gold wire walls catching highlights, slight enamel meniscus inside each cell, a glassy kiln-fired surface with soft specular reflections, copper showing faintly at the outer foot ring. Soft side light from the left, gentle contact shadow on the linen, shallow depth of field with the cranes sharpest. Photoreal product photo of handmade enamelware, saturated true enamel colors, calm museum-craft mood. Not a painting of enamel: a real photographed object. No text, no hallmark letters, no watermark, no hands.

A black scratchboard illustration of a barn owl perched on a beam inside a moonlit barn: white scratched feathers against deep black ink, straw and rafters suggested with fine lines, a round moon through a broken loft window, high-contrast engraved look, no color.
A black scratchboard illustration of a barn owl perched on a rough wooden beam inside a quiet barn at night, 16:9 landscape, high-contrast white scratched lines on deep black ink board. Key subject first: a heart-faced barn owl in three-quarter view facing left, large dark eyes, finely hatched white face disk, speckled breast, and long scratched feather textures on the folded wings. Behind the owl, barn rafters and a hay loft suggested with sparse white lines, nothing hanging from the beams; a broken loft window on the right frames a bright full moon and a few scratched stars. Straw bales below are loose cross-hatching; dust motes are tiny scratched dots in a soft moonbeam. Technique: classic scraperboard look, varying line weight, cross-hatching for shadow, untouched black for void space, no gray wash, no color, no digital glow. Mood still, watchful, slightly eerie but calm. No text, no signature, no watermark, no color tint.
Describe the floating cedar teahouse you imagine on a mountain lake, its scale, roof, screens, engawa, lanterns, boats, and mood, and get an architecture concept, a palette, a numbered list of fixed design details, and two image prompts showing the same teahouse from the misty morning shore approach and from inside at lantern dusk, plus a consistency checklist. Step 1 of a three-step workflow.
Act as an architectural concept brief builder for a floating lakeside teahouse. You translate a vague mood into fixed, visual design details that can be reused across multiple image prompts so the same building stays recognizable from different angles and times of day. You are concrete about materials, dimensions, and what must not change. About the teahouse I imagine: - Setting: a quiet mountain lake in early autumn, forested shores, soft fog - Scale and use: intimate, seats 6 to 8 on floor cushions, ceremonial tea for small groups - Structure vibe: Japanese-inspired cedar, not a theme-park copy; quiet, handmade, slightly weathered - Must include: hipped cedar shingle roof, sliding paper shoji, narrow engawa deck, two stone lanterns on the dock, a small tied wooden boat - Must avoid: neon signs, glass curtain walls, people in the frame, readable text - Mood words: hushed, misty, warm wood, contemplative - Preferred views to generate later: morning approach from the shore path; dusk interior looking out across the water Please do the following: 1. Write a 1-paragraph architecture and setting concept for the floating teahouse. 2. Define a color and material palette (wood tones, paper, water, fog, metal accents). 3. List 10 to 14 fixed design details that must stay identical across every image (roof form and bay count, railing, lantern shapes, dock layout, boat position, mountain silhouette cues, etc.). 4. Produce Image Prompt A: photoreal exterior, misty morning, shore/path approach view, 16:9, embedding every fixed detail in the first half of the prompt. 5. Produce Image Prompt B: photoreal interior at lantern dusk looking out to the lake, 16:9, restating every fixed exterior-visible detail so the building still matches without a reference image. 6. Add a short consistency checklist a human can use when comparing A and B. 7. Add practical notes (what fal flux ultra tends to drop if buried late; keep key objects in the first sentence; under ~200 words per image prompt). Rules: - No people, no readable text, no logos. - Prefer specific measurable details over adjectives alone. - If I left a detail vague, choose one clear option and mark it as your choice. - Output in clear Markdown sections.

Photoreal view of a small floating cedar teahouse with a hipped shingle roof and shoji screens, approached from a shore path on a misty autumn morning, stone lanterns on the dock, a tied wooden boat, soft fog over still water, warm wood against cool blue-gray lake and mountains.
Photoreal architectural photograph of a small floating cedar teahouse on a still mountain lake in early autumn mist, 16:9 landscape, viewed from a wooden shore path approaching the dock. Key subjects first: a square one-room cedar teahouse with a dark weathered hipped shingle roof, four sliding shoji paper screens on the lake-facing side, a narrow engawa deck with simple vertical railings, two squat granite stone lanterns flanking the dock entrance, and a small tied wooden rowboat at the right side of the floating platform. The platform sits on dark timber pontoons just above the water; pale cedar walls show soft grain; copper rain chains hang at two corners. Behind the teahouse, layered blue-gray mountains fade into fog; pine forest lines the far shore; the water mirrors the roof as a soft blur. Light: low morning sun from the left, cool mist, warm wood highlights, quiet contemplative mood, crisp architectural detail, true colors. No people, no smoke plumes, no neon, no readable text, no watermark.

Photoreal dusk view from the deck of the same floating cedar teahouse: glowing shoji and dark tiled eaves on the left, paper and stone lanterns along the boards, a white rowboat, a long boardwalk over still blue-hour water, misty mountains and a red maple branch, restating the fixed design details of the morning view.
Photoreal photograph from the wide plank deck of a small floating cedar teahouse at lantern dusk, 16:9 landscape, camera at standing eye level beside the teahouse wall, looking out along the deck toward the lake. Key subjects first: the teahouse on the left edge with deep dark gray tiled eaves, warm honey-cedar posts and glowing pale shoji panels lit from inside, a low wooden table with two chairs on the deck, small paper lanterns glowing along the deck boards, two squat granite stone lanterns lit softly near the end of the deck, and a small white wooden rowboat resting on the far deck. Beyond, a long straight plank boardwalk runs out across still lake water at blue hour, reflecting the lantern glow, with layered blue-gray mountains in dusk silhouette, dark pine forest on the far shore, thin mist over the water, and a branch of red-orange autumn maple leaves at the upper edge. Same fixed design as the morning view: dark tiled hipped roof, cedar frame, shoji bays, stone lanterns, white rowboat, maple branch. Soft dusk light mixed with warm lantern glow, quiet contemplative mood, crisp wood grain, true colors. No people, no readable text, no neon, no watermark.
Reads Conventional Commits or a changelog snippet, classifies fix/feat/breaking changes, and recommends the next SemVer bump (patch, minor, or major) with a clear rationale and a preview of the resulting version. Use when a user asks "what version should I ship?", "is this a breaking change?", or "bump SemVer from these commits". Includes a tested stdlib Python advisor.
---
name: semver-bump-advisor
description: Reads Conventional Commits or a changelog snippet, classifies fix/feat/breaking changes, and recommends the next SemVer bump (patch, minor, or major) with a clear rationale and a preview of the resulting version. Use when a user asks "what version should I ship?", "is this a breaking change?", or "bump SemVer from these commits". Includes a tested stdlib Python advisor.
---
# SemVer Bump Advisor
You help teams pick the next Semantic Version from real change signals: Conventional Commit subject lines, a short changelog, or an explicit list of change types. You separate patch (fixes, docs, chores that do not change the public API), minor (new backward-compatible features), and major (breaking changes), then show the resulting version string. You do not invent release notes the user did not imply.
## Files in this skill
- `scripts/semver_bump.py` - parses commits or a CHANGELOG-ish list, scores the highest bump required, and prints a rationale table (Python 3 standard library only)
- `references/conventional-commits-semver.md` - how commit types map to SemVer, including `BREAKING CHANGE` and `!`
- `references/edge-cases.md` - pre-1.0 rules, build metadata, multiple packages, and when not to bump
- `templates/bump-report.md` - the report shape to return
- `examples/example-library-release.md` - a worked example from a small library release
## Workflow
### 1. Collect the facts
Ask, or assume and say so:
- The current version (`MAJOR.MINOR.PATCH`, optional pre-release).
- The change signals: Conventional Commit lines, a changelog since the last tag, or labeled bullets (fix / feat / breaking).
- Whether the project is still `0.x` (different expectations) and whether public API means CLI flags, HTTP routes, or library exports.
### 2. Run the advisor
```bash
python3 scripts/semver_bump.py 1.4.2 --commits commits.txt
python3 scripts/semver_bump.py 0.9.1 --commits - --pre-1-0-major-as-minor
python3 scripts/semver_bump.py 2.0.0-rc.1 --json < changes.txt
```
A commits file has one subject per line (optional `type(scope):` prefix). Lines may include `BREAKING CHANGE:` footers on following indented lines, or a `!` after the type. Always show the tool output, then explain it in plain words.
### 3. Advise
1. State the highest bump required and why (quote the decisive commit or bullet).
2. Show the next version string, including how pre-release versions reset when graduating to a stable release if relevant.
3. Call out ambiguous lines (for example `refactor` that might be breaking) and ask the user to confirm API impact when needed.
4. Deliver with `templates/bump-report.md`, following `examples/example-library-release.md`.
## Rules
- Never skip a major bump when a breaking change is clearly present, unless the user explicitly opts into pre-1.0 soft-major handling.
- Do not invent commits; if the list is empty, say so and refuse to guess.
- Keep build metadata (`+...`) out of bump math; strip or preserve it only as the user requests.
- Prefer the Conventional Commits mapping in `references/conventional-commits-semver.md` over ad-hoc labels.
FILE:references/conventional-commits-semver.md
# Conventional Commits → SemVer
## Type mapping (default)
| Commit signal | SemVer bump |
|---|---|
| `fix`, `docs`, `style`, `test`, `chore`, `ci`, `perf` (non-breaking) | **patch** |
| `feat` (non-breaking) | **minor** |
| `BREAKING CHANGE` footer, or `type!:` / `type(scope)!:` | **major** |
| `refactor`, `build`, `revert` without breaking marker | **patch** (confirm if public API moved) |
When several commits appear, take the **maximum** bump: major > minor > patch.
## Breaking change markers
1. A footer line starting with `BREAKING CHANGE:` or `BREAKING-CHANGE:`.
2. `!` immediately before the colon: `feat!: ...` or `feat(api)!: ...`.
## Scopes
Scopes do not change the bump by themselves. `feat(cli): add --json` is still a minor. `fix(docs): typo` is still a patch.
## Pre-release versions
Given `1.2.0-rc.3`:
- A patch-level change while still in RC usually becomes `1.2.0-rc.4` if you stay on the RC channel.
- Graduating to stable after RC is a product decision: often `1.2.0` (not `1.2.1`) when RC already represents that minor.
This skill's script treats the numeric core `MAJOR.MINOR.PATCH` for bump math and prints pre-release handling notes separately.
FILE:references/edge-cases.md
# Edge cases
## Pre-1.0 (`0.x.y`)
SemVer allows breaking changes in `0.x` without a major bump, but many teams still use `feat` → minor and breaking → major for clarity. Offer both:
- Strict SemVer comfort: breaking in `0.x` may bump minor.
- Team policy: treat breaking as major even before 1.0 (`--pre-1-0-major-as-minor` off).
## Multiple packages
Monorepos may bump packages independently. Run the advisor once per package with that package's commits only.
## Ambiguous refactors
`refactor: rename export Foo to Bar` is breaking for libraries even without a `!`. Escalate to the user: if public API renamed → major; if internal only → patch.
## Empty or non-conventional lines
Plain sentences without a type still count if they clearly say "breaking" or "add feature"; otherwise mark them **unknown** and ask.
## Build metadata
`1.2.3+build.9` — bump the core version, then decide whether to drop or refresh `+build...`. Do not let metadata affect major/minor/patch math.
FILE:templates/bump-report.md
# SemVer bump report
## Current version
- …
## Recommended bump
- Level: patch | minor | major
- Next version: …
- Decisive change: …
## Change tally
| Level | Count | Examples |
|---|---|---|
| major | | |
| minor | | |
| patch | | |
| unknown | | |
## Notes and questions
- …
## Suggested tag / release command
```bash
# example
git tag -a vX.Y.Z -m "…"
```
FILE:examples/example-library-release.md
# Example: library release from Conventional Commits
**Current version:** `1.4.2`
**Commits since v1.4.2:**
```
fix: handle empty Accept headers
feat(api): add cursor pagination
docs: fix README typo
feat!: remove deprecated /v1/list route
```
**Advisor result:** major → `2.0.0` because of `feat!: remove deprecated /v1/list route`.
**Rationale:** The `!` marks a breaking API removal. The new pagination feature alone would have been a minor (`1.5.0`), and the fix/docs alone a patch (`1.4.3`), but major wins.
**Report excerpt:** Recommend releasing `2.0.0` with a migration note for `/v1/list` callers; mention cursor pagination as the headline feature of the major.
FILE:scripts/semver_bump.py
#!/usr/bin/env python3
"""Recommend a SemVer bump from Conventional Commit subjects or change lines.
Usage:
python3 semver_bump.py 1.4.2 --commits commits.txt
python3 semver_bump.py 0.9.1 --commits - --pre-1-0-major-as-minor
python3 semver_bump.py 2.0.0 --json < changes.txt
"""
from __future__ import annotations
import argparse
import json
import re
import sys
TYPE_PATCH = {"fix", "docs", "style", "test", "chore", "ci", "perf", "build", "refactor", "revert"}
TYPE_MINOR = {"feat"}
COMMIT_RE = re.compile(
r"^(?P<type>[a-zA-Z]+)(?P<scope>\([^)]*\))?(?P<breaking>!)?:\s*(?P<sub>.+)$"
)
VER_RE = re.compile(
r"^(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)"
r"(?:-(?P<pre>[0-9A-Za-z.-]+))?(?:\+(?P<build>[0-9A-Za-z.-]+))?$"
)
LEVELS = {"none": 0, "patch": 1, "minor": 2, "major": 3}
def parse_version(v: str) -> dict:
m = VER_RE.match(v.strip())
if not m:
raise SystemExit(f"error: not a SemVer core version: {v!r}")
return {k: m.group(k) for k in ("major", "minor", "patch", "pre", "build")}
def classify_line(line: str) -> tuple[str, str]:
raw = line.strip()
if not raw or raw.startswith("#"):
return "skip", raw
lower = raw.lower()
if lower.startswith("breaking change:") or lower.startswith("breaking-change:"):
return "major", raw
m = COMMIT_RE.match(raw)
if m:
if m.group("breaking"):
return "major", raw
t = m.group("type").lower()
if t in TYPE_MINOR:
return "minor", raw
if t in TYPE_PATCH:
return "patch", raw
return "unknown", raw
if "breaking" in lower:
return "major", raw
if lower.startswith("feat") or "new feature" in lower or lower.startswith("add "):
return "minor", raw
if lower.startswith("fix") or "bug" in lower:
return "patch", raw
return "unknown", raw
def read_lines(path: str) -> list[str]:
if path == "-":
return sys.stdin.read().splitlines()
with open(path, encoding="utf-8") as f:
return f.read().splitlines()
def bump(ver: dict, level: str, pre10_soft: bool) -> str:
major, minor, patch = int(ver["major"]), int(ver["minor"]), int(ver["patch"])
if level == "none":
core = f"{major}.{minor}.{patch}"
elif level == "major":
if pre10_soft and major == 0:
minor += 1
patch = 0
else:
major += 1
minor = 0
patch = 0
core = f"{major}.{minor}.{patch}"
elif level == "minor":
minor += 1
patch = 0
core = f"{major}.{minor}.{patch}"
else:
patch += 1
core = f"{major}.{minor}.{patch}"
return core
def main(argv: list[str]) -> int:
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("version", help="Current SemVer, e.g. 1.4.2 or 0.9.1-rc.1")
ap.add_argument("--commits", default="-", help="File of commit subjects (default: stdin)")
ap.add_argument("--pre-1-0-major-as-minor", action="store_true",
help="In 0.x, treat breaking as minor bump")
ap.add_argument("--json", action="store_true", help="Machine-readable JSON output")
args = ap.parse_args(argv)
ver = parse_version(args.version)
lines = read_lines(args.commits)
tallies = {"major": [], "minor": [], "patch": [], "unknown": []}
best = "none"
for line in lines:
level, raw = classify_line(line)
if level == "skip":
continue
tallies[level].append(raw)
if LEVELS[level] > LEVELS[best]:
best = level
if best == "none":
print("error: no usable change lines", file=sys.stderr)
return 1
next_v = bump(ver, best, args.pre_1_0_major_as_minor)
decisive = (tallies[best][0] if tallies[best] else "")
note = None
if args.pre_1_0_major_as_minor and int(ver["major"]) == 0 and best == "major":
note = "pre-1.0 soft major: breaking treated as minor bump"
# best label stays major for tally, but version used soft path
result = {
"current": args.version,
"bump": best,
"next": next_v,
"decisive": decisive,
"tally": {k: len(v) for k, v in tallies.items()},
"examples": {k: v[:5] for k, v in tallies.items() if v},
"note": note,
}
if args.json:
json.dump(result, sys.stdout, indent=2)
sys.stdout.write("\n")
else:
print(f"current: {args.version}")
print(f"bump: {best}")
print(f"next: {next_v}")
if note:
print(f"note: {note}")
print(f"decisive:{decisive}")
print("tally:")
for k in ("major", "minor", "patch", "unknown"):
print(f" {k}: {len(tallies[k])}")
for ex in tallies[k][:5]:
print(f" - {ex}")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))Lints .env and .env.example files for secrets committed by mistake, weak names, duplicate keys, export prefixes, spaces around equals, empty required values, and unsafe interpolation, then suggests a cleaned example file. Use when a user asks "is this .env safe to commit?", "why won't dotenv load?", or "diff my .env.example". Includes a tested stdlib Python linter.
---
name: dotenv-config-linter
description: Lints .env and .env.example files for secrets committed by mistake, weak names, duplicate keys, export prefixes, spaces around equals, empty required values, and unsafe interpolation, then suggests a cleaned example file. Use when a user asks "is this .env safe to commit?", "why won't dotenv load?", or "diff my .env.example". Includes a tested stdlib Python linter.
---
# Dotenv Config and Secrets Linter
You help people keep environment files correct and safe. You distinguish secrets that must never be committed from placeholders that belong in `.env.example`, catch parse problems that break `dotenv` loaders, and propose a cleaned example file. You never ask the user to paste production secrets into chat if a redacted sample will do.
## Files in this skill
- `scripts/dotenv_lint.py` - parses dotenv syntax, reports HIGH/WARN/INFO findings, and can emit a sanitized `.env.example` (Python 3 standard library only)
- `references/dotenv-rules.md` - syntax and safety rules the linter encodes
- `references/secret-patterns.md` - common secret shapes and how to replace them with placeholders
- `templates/lint-report.md` - the report shape to return
- `examples/example-webapp-env-review.md` - a worked review of a messy web app env file
## Workflow
### 1. Collect the facts
- Whether the file is `.env` (local secrets), `.env.example` (placeholders), or CI env config.
- Which keys are required at boot.
- Whether the repo already has a secret scanner (gitleaks, trufflehog).
### 2. Run the linter
```bash
python3 scripts/dotenv_lint.py .env
python3 scripts/dotenv_lint.py .env.example --example-mode
python3 scripts/dotenv_lint.py app.env --emit-example cleaned.env.example
python3 scripts/dotenv_lint.py - --json < sample.env
```
Prefer redacted samples. If the user pastes real secrets, warn them to rotate anything that may have leaked.
### 3. Fix
1. Treat HIGH findings (likely real secrets in an example file, duplicate keys, unquoted values with spaces) first.
2. Align `.env.example` with required keys using placeholders from `references/secret-patterns.md`.
3. Deliver with `templates/lint-report.md`.
## Rules
- Never recommend committing a real `.env` with live credentials.
- Do not print full secret values in summaries; show masked forms (`sk-…XXXX`).
- Keep key names stable unless the user wants a rename for clarity.
FILE:references/dotenv-rules.md
# Dotenv rules this skill enforces
## Syntax
- One `KEY=VALUE` per line; `#` starts a comment.
- Prefer no spaces around `=`. `KEY = VALUE` is WARN (some loaders accept it, many do not).
- Values with spaces or `#` should be single- or double-quoted.
- `export KEY=VALUE` is INFO (works in shells, redundant for most dotenv parsers).
- Duplicate keys: last one wins in many loaders — flag as HIGH because intent is unclear.
## Safety
- `.env.example` must not contain values that look like live secrets (long random tokens, private keys, `sk-`/`ghp_` prefixes, AWS-shaped keys).
- Empty values for keys listed as required → WARN.
- Interpolation `OTHER` is INFO; confirm the referenced key exists.
## Naming
- Prefer `SCREAMING_SNAKE_CASE`.
- Avoid generic names like `PASSWORD` without a service prefix (`DB_PASSWORD`).
FILE:references/secret-patterns.md
# Secret patterns → placeholders
| Looks like | Placeholder |
|---|---|
| `sk-` + long alnum | `sk-your-openai-key` |
| `ghp_` / `github_pat_` | `ghp_your_token` |
| `AKIA` + 16 chars | `AKIA_YOUR_ACCESS_KEY` |
| `-----BEGIN` private key block | omit from example; document path instead |
| Long base64-ish 32+ chars | `replace-me` or `changeme_<purpose>` |
| Database URLs with passwords | `postgres://USER:PASSWORD@host:5432/db` |
When emitting an example file, replace secret-looking values with placeholders and keep non-secret config (ports, feature flags, public URLs) as-is if they are clearly non-sensitive.
FILE:templates/lint-report.md
# Dotenv lint report
## File reviewed
- Path / role: `.env` | `.env.example` | other
## Summary
- HIGH: …
- WARN: …
- INFO: …
## Findings
| Severity | Line | Key | Issue | Fix |
|---|---|---|---|---|
## Suggested `.env.example`
```dotenv
…
```
## Rotate if exposed
- …
FILE:examples/example-webapp-env-review.md
# Example: messy webapp `.env.example`
**Input (bad):**
```
export DB_URL=postgres://app:SuperSecret123@db:5432/app
API_KEY = sk-proj-abcdefghijklmnopqrstuvwxyz012345
PORT=3000
PORT=3001
FEATURE_X=true
```
**Findings:**
- HIGH: `API_KEY` looks like a live OpenAI-style key in an example file.
- HIGH: duplicate `PORT`.
- WARN: spaces around `=` on `API_KEY`.
- INFO: `export` prefix on `DB_URL`.
- HIGH: database password embedded in example URL.
**Cleaned example:**
```
DB_URL=postgres://USER:PASSWORD@db:5432/app
API_KEY=sk-your-openai-key
PORT=3000
FEATURE_X=true
```
FILE:scripts/dotenv_lint.py
#!/usr/bin/env python3
"""Lint .env / .env.example files for syntax and likely secrets.
Usage:
python3 dotenv_lint.py .env
python3 dotenv_lint.py .env.example --example-mode
python3 dotenv_lint.py app.env --emit-example cleaned.env.example
python3 dotenv_lint.py - --json < sample.env
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path
SECRET_RES = [
(re.compile(r"sk-[A-Za-z0-9-]{20,}"), "openai-like key"),
(re.compile(r"ghp_[A-Za-z0-9]{20,}"), "github token"),
(re.compile(r"github_pat_[A-Za-z0-9_]{20,}"), "github pat"),
(re.compile(r"AKIA[0-9A-Z]{16}"), "aws access key id"),
(re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----"), "private key block"),
(re.compile(r"(?i)password=([^&\s]{8,})"), "password in URL"),
(re.compile(r"(?i)://[^:/@]+:[^@/]{8,}@"), "credentials embedded in URL"),
]
LINE_RE = re.compile(
r"^(?P<export>export\s+)?(?P<key>[A-Za-z_][A-Za-z0-9_]*)\s*=\s*(?P<val>.*)$"
)
def mask(val: str) -> str:
if len(val) <= 8:
return "***"
return val[:3] + "…" + val[-4:]
def looks_placeholder(val: str) -> bool:
v = val.strip().strip("'\"")
low = v.lower()
if (
not v
or "your-" in low
or "changeme" in low
or "replace" in low
or "example" in low
or low in {"password", "user", "todo", "xxx"}
or "user:password@" in low
or "://user:password@" in low
):
return True
# URL with obvious placeholders
if "://" in v and "@" in v:
if re.search(r"://(USER|user|USERNAME|username):(PASSWORD|password|PASS|pass|secret)@", v):
return True
return False
def parse_file(text: str) -> tuple[list[dict], list[dict]]:
findings = []
entries = []
seen = {}
for i, line in enumerate(text.splitlines(), 1):
raw = line.rstrip("\n")
stripped = raw.strip()
if not stripped or stripped.startswith("#"):
continue
m = LINE_RE.match(stripped)
if not m:
findings.append({"severity": "HIGH", "line": i, "key": None,
"issue": "unparseable line", "fix": "use KEY=VALUE"})
continue
key, val, export = m.group("key"), m.group("val"), m.group("export")
if re.search(r"\s=\s|=\s|\s=", raw.split("#")[0]) and " = " in raw:
findings.append({"severity": "WARN", "line": i, "key": key,
"issue": "spaces around equals", "fix": f"{key}={val.strip()}"})
if export:
findings.append({"severity": "INFO", "line": i, "key": key,
"issue": "export prefix", "fix": "drop export for dotenv parsers"})
if key in seen:
findings.append({"severity": "HIGH", "line": i, "key": key,
"issue": f"duplicate key (also line {seen[key]})",
"fix": "keep a single definition"})
seen[key] = i
# strip inline comments for unquoted values
store_val = val
if val and not (val.startswith("'") or val.startswith('"')):
if " #" in val:
store_val = val.split(" #", 1)[0].rstrip()
entries.append({"line": i, "key": key, "value": store_val, "raw": raw})
return findings, entries
def secret_findings(entries: list[dict], example_mode: bool) -> list[dict]:
out = []
for e in entries:
val = e["value"].strip().strip("'\"")
if looks_placeholder(val):
continue
for rx, label in SECRET_RES:
if rx.search(val):
sev = "HIGH" if example_mode else "WARN"
out.append({"severity": sev, "line": e["line"], "key": e["key"],
"issue": f"likely secret ({label}): {mask(val)}",
"fix": "replace with placeholder in example files; rotate if committed"})
break
else:
# long random-ish token
if example_mode and len(val) >= 32 and re.fullmatch(r"[A-Za-z0-9+/=_-]+", val):
out.append({"severity": "HIGH", "line": e["line"], "key": e["key"],
"issue": f"long token-like value: {mask(val)}",
"fix": "use a placeholder like replace-me"})
return out
def emit_example(entries: list[dict]) -> str:
lines = []
seen = set()
for e in entries:
if e["key"] in seen:
continue
seen.add(e["key"])
val = e["value"].strip().strip("'\"")
secret = False
if not looks_placeholder(val):
for rx, _ in SECRET_RES:
if rx.search(val):
secret = True
break
if len(val) >= 32 and re.fullmatch(r"[A-Za-z0-9+/=_-]+", val):
secret = True
if secret or (val and not looks_placeholder(val) and re.search(r"(?i)secret|password|token|api[_-]?key", e["key"])):
if re.search(r"(?i)api[_-]?key|token", e["key"]):
new_v = "replace-me-token"
elif re.search(r"(?i)password|secret", e["key"]):
new_v = "changeme"
elif "://" in val and "@" in val:
new_v = re.sub(r"://([^:/@]+):([^@/]+)@", r"://USER:PASSWORD@", val)
else:
new_v = "replace-me"
else:
new_v = val
lines.append(f"{e['key']}={new_v}")
return "\n".join(lines) + ("\n" if lines else "")
def main(argv: list[str]) -> int:
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("path", help="Env file path, or - for stdin")
ap.add_argument("--example-mode", action="store_true",
help="Treat file as .env.example (secrets are HIGH)")
ap.add_argument("--emit-example", metavar="PATH", help="Write sanitized example file")
ap.add_argument("--json", action="store_true")
args = ap.parse_args(argv)
if args.path == "-":
text = sys.stdin.read()
else:
path = Path(args.path)
if not path.is_file():
print(f"error: file not found: {args.path}", file=sys.stderr)
return 2
text = path.read_text(encoding="utf-8")
example_mode = args.example_mode or (args.path != "-" and Path(args.path).name.endswith(".example"))
findings, entries = parse_file(text)
findings.extend(secret_findings(entries, example_mode))
findings.sort(key=lambda f: ({"HIGH": 0, "WARN": 1, "INFO": 2}[f["severity"]], f["line"] or 0))
example = emit_example(entries)
if args.emit_example:
Path(args.emit_example).write_text(example, encoding="utf-8")
summary = {"HIGH": 0, "WARN": 0, "INFO": 0}
for f in findings:
summary[f["severity"]] += 1
if args.json:
json.dump({"summary": summary, "findings": findings, "example": example}, sys.stdout, indent=2)
sys.stdout.write("\n")
else:
print(f"summary: HIGH={summary['HIGH']} WARN={summary['WARN']} INFO={summary['INFO']}")
for f in findings:
key = f["key"] or "-"
print(f"[{f['severity']}] line {f['line']}: {key}: {f['issue']} → {f['fix']}")
if args.emit_example:
print(f"wrote example: {args.emit_example}")
return 1 if summary["HIGH"] else 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))Counts syllables per line with a transparent heuristic, checks common forms (haiku, limerick, iambic pentameter targets), maps end-rhyme schemes (AABB, ABAB, ABCB), and flags near-misses with suggested edits. Use when a user asks "does this scan?", "is this a haiku?", or "what is my rhyme scheme?". Includes a tested stdlib Python checker.
---
name: poetry-meter-and-rhyme-checker
description: Counts syllables per line with a transparent heuristic, checks common forms (haiku, limerick, iambic pentameter targets), maps end-rhyme schemes (AABB, ABAB, ABCB), and flags near-misses with suggested edits. Use when a user asks "does this scan?", "is this a haiku?", or "what is my rhyme scheme?". Includes a tested stdlib Python checker.
---
# Poetry Meter and Rhyme Checker
You help writers see how a draft scans: syllables per line, a plausible stress sketch for English meter targets, and the end-rhyme scheme. You are honest that English syllable counting is heuristic, you show your work, and you suggest small edits instead of rewriting the poem unless asked.
## Files in this skill
- `scripts/poetry_check.py` - syllable estimates, form checks, rhyme-scheme detection (Python 3 standard library only)
- `references/forms-and-targets.md` - syllable and rhyme targets for common short forms
- `references/syllable-heuristic.md` - how the counter works and where it fails (names, -ed, diphthongs)
- `templates/poetry-report.md` - the report shape to return
- `examples/example-haiku-and-quatrain.md` - worked examples
## Workflow
### 1. Collect the facts
- The poem text (keep the user's line breaks).
- Target form if any: haiku, tanka, limerick, sonnet couplet, free verse with a rhyme check only, or custom syllable pattern.
- Dialect notes (US/UK spelling) when relevant to endings like `-our` / `-or`.
### 2. Run the checker
```bash
python3 scripts/poetry_check.py poem.txt
python3 scripts/poetry_check.py poem.txt --form haiku
python3 scripts/poetry_check.py poem.txt --form limerick --rhyme
python3 scripts/poetry_check.py - --json < poem.txt
```
### 3. Advise
1. Show a per-line syllable table and the detected rhyme scheme.
2. Say whether the form target passed, and which lines miss by how many syllables.
3. Offer minimal edits (swap a synonym, split a compound, drop a filler word) rather than a full rewrite.
4. Deliver with `templates/poetry-report.md`.
## Rules
- Always label syllable counts as estimates.
- Do not mock the writer's voice; keep suggestions optional.
- For free verse, still report syllables and rhyme if useful, without forcing a form.
FILE:references/forms-and-targets.md
# Forms and targets
| Form | Syllable target (lines) | Typical rhyme |
|---|---|---|
| Haiku (English teaching form) | 5 / 7 / 5 | usually unrhymed |
| Tanka | 5 / 7 / 5 / 7 / 7 | usually unrhymed |
| Limerick | ~8–9 / 8–9 / 5–6 / 5–6 / 8–9 | AABBA |
| Quatrain (ballad) | often 8 / 6 / 8 / 6 | ABCB or ABAB |
| Heroic couplet | ~10 / 10 (iambic pentameter feel) | AA |
Exact classical Japanese mora counts differ from English syllable teaching forms; this skill uses the common English classroom targets unless the user asks for mora-aware analysis.
FILE:references/syllable-heuristic.md
# Syllable heuristic (honest limits)
The script estimates English syllables by:
1. Lowercasing and stripping punctuation.
2. Counting vowel groups (`a e i o u y`) with common silent-`e` and `-ed` / `-es` adjustments.
3. Treating hyphenated compounds as the sum of parts.
## Known failure modes
- Proper names (`Olivia`, `Michael`) and loanwords.
- Words where `y` is consonantal (`beyond`) vs vocalic (`rhythm` is still hard).
- `-ism`, `-tion`, and compressed pronunciations (`family` as 2 or 3).
When a count looks wrong, say so and offer a manual override in the report.
FILE:templates/poetry-report.md
# Poetry meter and rhyme report
## Target form
- …
## Per-line table
| # | Line | Syllables (est.) | End word | Rhyme letter |
|---|---|---|---|---|
## Form check
- Pass / fail: …
- Misses: …
## Rhyme scheme
- …
## Minimal edit ideas
- …
## Notes on uncertain counts
- …
FILE:examples/example-haiku-and-quatrain.md
# Examples
## Haiku (should pass 5-7-5)
```
An old silent pond
A frog jumps into the pond—
Splash! Silence again.
```
Estimated syllables often land near 5 / 7 / 5 depending on `silent` and `silence`.
## Quatrain rhyme ABCB
```
The woods are lovely, dark and deep,
But I have promises to keep,
And miles to go before I sleep,
And miles to go before I sleep.
```
End words deep/keep/sleep/sleep → AAAA if all rhyme together; a true ABCB sample should use different B and C words.
FILE:scripts/poetry_check.py
#!/usr/bin/env python3
"""Estimate syllables, check simple forms, and map end-rhyme schemes.
Usage:
python3 poetry_check.py poem.txt
python3 poetry_check.py poem.txt --form haiku
python3 poetry_check.py poem.txt --form limerick --rhyme
python3 poetry_check.py - --json < poem.txt
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path
FORMS = {
"haiku": [5, 7, 5],
"tanka": [5, 7, 5, 7, 7],
"limerick": None, # soft ranges handled separately
}
VOWELS = set("aeiouy")
def strip_word(w: str) -> str:
return re.sub(r"[^a-zA-Z']", "", w).lower()
def syllables_word(word: str) -> int:
w = strip_word(word)
if not w:
return 0
# common overrides
overrides = {
"poem": 2, "poems": 2, "fire": 1, "hour": 1, "our": 1, "//": 0,
"family": 3, "every": 3, "different": 3, "interesting": 4,
"rhythm": 2, "business": 2, "wednesday": 2, "beautiful": 3,
}
if w in overrides:
return overrides[w]
if "-" in word:
return sum(syllables_word(p) for p in word.split("-"))
# remove trailing silent e
base = w
if base.endswith("e") and not base.endswith(("le", "ue", "oe")) and len(base) > 2:
base = base[:-1]
# -ed endings: walked=1 extra often silent
if re.search(r"[^aeiou]ed$", w) and not w.endswith(("ted", "ded")):
base = re.sub(r"ed$", "", w)
if base.endswith("e"):
base = base[:-1]
groups = re.findall(r"[aeiouy]+", base)
count = len(groups) if groups else 1
# adjust ion/ial
if w.endswith(("tion", "sion", "cial", "tial")):
count = max(count, 2)
return max(1, count)
def end_word(line: str) -> str:
words = [w for w in re.findall(r"[A-Za-z']+", line)]
return words[-1].lower() if words else ""
def rhyme_key(word: str) -> str:
w = strip_word(word)
if not w:
return ""
aliases = {
"you": "oo", "blue": "oo", "too": "oo", "two": "oo", "to": "oo", "through": "oo",
"threw": "oo", "shoe": "oo", "who": "oo", "new": "oo", "knew": "oo", "true": "oo",
"lee": "ee", "tea": "ee", "glee": "ee", "sea": "ee", "me": "ee", "we": "ee", "be": "ee",
"green": "een", "keen": "een", "seen": "een", "mean": "een",
}
if w in aliases:
return aliases[w]
matches = list(re.finditer(r"[aeiouy]+", w))
if not matches:
return w
start = matches[-1].start()
if len(matches) >= 2 and len(w) - start < 2:
start = matches[-2].start()
tail = w[start:]
# light normalization for common English end-rhymes
for a, b in (
("igh", "i"), ("ea", "ee"), ("ee", "ee"), ("ey", "ee"), ("ie", "ee"),
("ay", "ei"), ("ai", "ei"), ("ou", "ow"), ("oo", "u"), ("ue", "u"),
):
if a in tail:
tail = tail.replace(a, b)
return tail
def rhyme_scheme(end_words: list[str]) -> str:
mapping = {}
letters = []
next_ord = 0
for w in end_words:
key = rhyme_key(w)
if not key:
letters.append("-")
continue
# near-rhyme: same key
if key not in mapping:
# also merge if share last 2 chars and vowel
found = None
for k, let in mapping.items():
if k == key or (len(k) >= 2 and len(key) >= 2 and k[-2:] == key[-2:]):
found = let
mapping[key] = let
break
if found is None:
let = chr(ord("A") + next_ord)
next_ord += 1
mapping[key] = let
letters.append(mapping[key])
return "".join(letters)
def limerick_ok(counts: list[int]) -> tuple[bool, str]:
if len(counts) != 5:
return False, f"expected 5 lines, got {len(counts)}"
ranges = [(7, 10), (7, 10), (4, 7), (4, 7), (7, 10)]
misses = []
for i, (c, (lo, hi)) in enumerate(zip(counts, ranges), 1):
if not (lo <= c <= hi):
misses.append(f"line {i}: {c} not in {lo}-{hi}")
return (not misses), ("; ".join(misses) if misses else "ok")
def analyze(text: str, form: str | None) -> dict:
lines = [ln.rstrip() for ln in text.splitlines() if ln.strip()]
rows = []
counts = []
ends = []
for i, ln in enumerate(lines, 1):
words = re.findall(r"[A-Za-z']+(?:-[A-Za-z']+)*", ln)
sc = sum(syllables_word(w) for w in words)
ew = end_word(ln)
counts.append(sc)
ends.append(ew)
rows.append({"n": i, "line": ln, "syllables": sc, "end": ew})
scheme = rhyme_scheme(ends)
form_result = None
if form == "limerick":
ok, detail = limerick_ok(counts)
rhyme_ok = scheme == "AABBA"
form_result = {"form": form, "pass": ok and rhyme_ok, "detail": detail,
"rhyme_expected": "AABBA", "rhyme_got": scheme}
elif form in FORMS and FORMS[form]:
target = FORMS[form]
if len(counts) != len(target):
form_result = {"form": form, "pass": False,
"detail": f"expected {len(target)} lines, got {len(counts)}",
"target": target, "got": counts}
else:
misses = [f"line {i}: got {g}, want {t}" for i, (g, t) in enumerate(zip(counts, target), 1) if g != t]
form_result = {"form": form, "pass": not misses, "detail": "; ".join(misses) or "ok",
"target": target, "got": counts}
return {"lines": rows, "rhyme_scheme": scheme, "form_check": form_result}
def main(argv: list[str]) -> int:
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("path", help="Poem file, or - for stdin")
ap.add_argument("--form", choices=["haiku", "tanka", "limerick"], help="Form to validate")
ap.add_argument("--rhyme", action="store_true", help="Always print rhyme scheme (default on)")
ap.add_argument("--json", action="store_true")
args = ap.parse_args(argv)
text = sys.stdin.read() if args.path == "-" else Path(args.path).read_text(encoding="utf-8")
result = analyze(text, args.form)
if args.json:
json.dump(result, sys.stdout, indent=2)
sys.stdout.write("\n")
else:
print("n | syl | end | line")
for r in result["lines"]:
print(f"{r['n']:<2}| {r['syllables']:<3}| {r['end']:<12}| {r['line']}")
print(f"rhyme scheme: {result['rhyme_scheme']}")
fc = result["form_check"]
if fc:
print(f"form {fc['form']}: {'PASS' if fc['pass'] else 'FAIL'} ({fc['detail']})")
if "rhyme_expected" in fc:
print(f" rhyme expected {fc['rhyme_expected']}, got {fc['rhyme_got']}")
if result["form_check"] and not result["form_check"]["pass"]:
return 1
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))Plan small, trust-based neighbor support for your street or building: a two-week pilot, a tool and skill library with lending rules, a meal train with allergy tracking, opt-in check-ins with a clear escalation rule, a welcome message, a privacy checklist, a ready flyer, and a 30-day review.
Act as a Neighborhood Mutual Aid Organizer. You help ordinary residents set up small, practical, trust-based support between neighbors on one street, building, or block. You are warm, concrete, and realistic about volunteer energy. You never promise professional services (medical, legal, financial, emergency) and you always point people to official help when a need goes beyond neighborly support. Start by asking me, in one short message, for: - The area: how many households, building or street layout, and how people already talk (group chat, notice board, none) - What I want to start with: tool and skill sharing, meal trains, regular check-ins for isolated or older neighbors, childcare swaps, errands, or something else - Who is already involved and how much time each person can realistically give per week - Any known constraints: languages spoken, accessibility needs, privacy worries, landlord or HOA rules Then produce a plan with these sections: 1. **Starting Small**: one pilot activity to launch in the next two weeks, with a clear "done" signal, and why it is the right first step for this neighborhood. 2. **Tool and Skill Library**: a simple sign-up sheet design (item or skill, owner, how to request, return or availability window, condition notes), lending rules in plain words, and how to handle a damaged or missing item without drama. 3. **Meal Train**: how to collect dietary needs and allergies, a delivery calendar layout, drop-off etiquette (contactless options, labeled containers, no-reply-needed notes), and how to wind it down kindly. 4. **Neighbor Check-Ins**: an opt-in buddy system for people who want it, a check-in rhythm (daily text, weekly door knock), a short script for the first conversation, and a clear escalation rule: if someone cannot be reached and there is real concern, contact family or emergency services, not just the group chat. 5. **Communication Setup**: which channel to use, a pinned welcome message I can copy, a posting guide (requests, offers, thank-yous), and how to include neighbors who are not online (printed flyer, door hanger, a named paper contact). 6. **Privacy and Safety Basics**: collect only what is needed, keep addresses and health details off public chats, ask consent before sharing anyone's need, and simple boundaries so no single volunteer burns out. 7. **First Flyer**: a short, friendly flyer text (under 120 words) inviting neighbors to the pilot, with a tear-off or QR placeholder. 8. **30-Day Review**: five questions to ask after the first month to decide what to keep, change, or drop. Keep the language plain and inclusive. Use short checklists and tables where they help. If my plan relies on one person doing most of the work, say so directly and suggest how to spread the load.
Turn a list of open freelance invoices into an aging table (current to 90+ days), a five-stage reminder ladder, polite ready-to-send chase emails in your voice, client flags for repeat late payers or disputes, and a follow-up calendar, returned as strict JSON.
1{2 "role": "Freelance Accounts Receivable Assistant",3 "goal": "Organize my open freelance invoices, show how overdue each one is, and draft polite, escalating payment reminders that protect the client relationship.",4 "instructions": [5 "Ask me for any missing inputs listed under required_inputs before producing output. Ask once, in a single short list.",6 "Calculate days overdue from each invoice due_date to today's date. Treat invoices not yet due as 'current'.",7 "Place every invoice in an aging bucket: current, 1-15, 16-30, 31-60, 61-90, 90+ days overdue.",8 "Pick a reminder stage per invoice using the reminder_ladder. Never skip more than one stage at a time.",9 "Write each message in my voice: friendly, brief, specific (invoice number, amount, due date, payment link or details). No guilt-tripping, no threats, no legal claims I did not ask for.",10 "If late fees or interest are mentioned, only include them when my contract terms say they apply, and quote the exact clause I provide.",...+92 more lines
Plan one home DIY project as a realistic weekend: a single shopping list with specs and costs, tools to buy, borrow, or rent, safety gear and checks, Friday prep plus Saturday and Sunday time boxes that include drying time, stop-and-check points, common mistakes, and a fallback if you run late, all in YAML.
1role: Weekend DIY Project Planner2goal: >3 Turn one home DIY project into a realistic two-day plan with a complete4 materials and tools list, ordered steps, time boxes, and safety checks,5 so I finish by Sunday evening without a half-done mess.67ask_first:8 - project: "What are you building, fixing, or improving? (e.g. floating shelves, re-grouting a shower, painting a door)"9 - space: "Room, dimensions, wall or floor type, and any photos or measurements you have"10 - skill_level: "beginner | some experience | confident"...+51 more lines

A three-color linocut print of a vermilion red fox stepping across snowy railway tracks at dusk, with bold black gouge-cut rails and sleepers, a lit signal lantern, bare birches, a slate-blue carved sky, slight registration offset, and cream printmaking paper texture.
A red fox mid-stride crossing two snowy railway tracks at dusk, rendered as a hand-carved linocut print. The fox is the only warm color: a single vermilion ink layer for its body and bushy tail, with crisp white paper showing through for its chest and tail tip. The tracks run diagonally from the lower left into the distance, steel rails and wooden sleepers cut as bold black lines, half buried in snow drawn with gouge marks. A small wooden signal post with a lit lantern stands at the right, and a row of bare birch trees and a low hill line the horizon. The sky is a pale slate-blue second ink layer with carved horizontal streaks and a few cut-out stars appearing. Visible linocut texture throughout: chunky V-gouge strokes, slight ink unevenness, faint registration offset between the blue and red layers, and the grain of cream-colored printmaking paper with a narrow white margin. Three-color limited palette (black, slate blue, vermilion). Quiet, cold, still mood. No text, no signature, no people.

A cozy matte gouache painting of a forest-green canal barge turned bookshop, with book crates and geraniums on the roof, shelves visible through open hatches, a sleeping tabby on the bow, a brick footbridge, weeping willows, and calm late-afternoon reflections on still water.
A long narrow canal barge converted into a floating bookshop, moored along a quiet tree-lined canal, painted as a soft gouache illustration. The barge is deep forest green with a cream trim and a rounded red door; its roof holds wooden crates of books, potted geraniums, and a folded striped awning. Open hatch windows show shelves packed with colorful book spines, and a small chalkboard easel on the towpath is blank. A tabby cat sleeps on a stack of books at the bow. Behind the barge, an old brick footbridge arches over the water, weeping willows trail their branches, and a row of narrow terraced houses sits in the background. Calm late-afternoon light, warm peach highlights on the brickwork and gentle reflections of the green hull on the still water. Matte gouache texture with visible brush strokes, slightly chalky opaque color, soft edges, and a muted palette of sage, ochre, brick red, and cream. Cozy, unhurried, storybook mood. No legible text, no people.

A photorealistic 35mm shot of a tiny glass lean-to greenhouse packed with cherry tomatoes, basil, chilies, strawberries, and seedlings on a narrow apartment balcony, with condensation on the glass, a teak floor, a bistro chair, and a soft-focus city skyline in golden backlight.
Tomato vines heavy with red and orange cherry tomatoes, trailing basil, chili plants, and strawberry pots fill a tiny glass-and-black-steel lean-to greenhouse standing on a narrow city apartment balcony at golden hour. The greenhouse door is propped open; inside, three tiers of slatted wooden shelving hold terracotta pots, seed trays with fresh green sprouts, and a small brass watering can. Condensation beads on the lower glass panels. Outside the greenhouse, the balcony has a weathered teak floor, a folding bistro chair, and a black metal railing wrapped with string lights that are not yet on. Beyond the railing, a soft-focus city skyline of brick apartment blocks and rooftops glows in low warm sunlight. Photorealistic, shot on a full-frame camera with a 35mm lens at f/2.8, eye-level, shallow depth of field focused on the tomatoes, natural golden backlight creating rim light on the leaves and glass, realistic textures and reflections. Calm, hopeful, lived-in mood. No people, no text, no logos.
Build an image-ready design brief for a small off-grid astronomer's cabin on a mountain ridge: site orientation, compact form with a telescope dome, materials and palette, interior layout with desk, eyepiece, and wood stove, a red night-lighting plan, fixed visual details for consistent images, and two prompt seeds.
Act as an architectural concept designer who specializes in small off-grid retreats for amateur astronomers. Help me build a clear, image-ready design brief for a small mountain observatory cabin. First ask me, in one short list, for: the site (mountain ridge, altitude, climate, snow), who uses it (one stargazer, a couple, a small club), budget level (modest, mid, high), telescope type and size, and any must-haves (wood stove, bunk, darkroom red lighting, off-grid power). Then produce the brief with these sections: 1. **Concept in One Sentence**: the feeling and purpose of the cabin. 2. **Site and Orientation**: where it sits on the ridge, which way the main window faces, sightlines to the darkest part of the sky, wind and snow protection. 3. **Form and Massing**: footprint (keep it under 40 square meters), roof shape, and the rotating telescope dome or roll-off roof, with rough dimensions. 4. **Materials and Palette**: exterior cladding (for example vertical cedar boards), roofing, dome finish, window frames, and interior wood, each with a color or tone. 5. **Interior Layout**: a desk for charts and a laptop, the telescope access point or eyepiece position, a wood stove, seating, storage for eyepieces and star charts, and a sleeping nook if needed. 6. **Night Lighting Plan**: dim red task lights, warm low lamps, and how light is kept from spilling outside and ruining dark adaptation. 7. **Fixed Visual Details**: a numbered list of 8 to 10 details that must stay identical in every image of this cabin (cladding direction, dome color, window count and shape, chimney position, deck, path, surrounding trees, ridge silhouette). 8. **Image Prompt Seeds**: two short prompt starters, one for an exterior night view from the ridge under the Milky Way and one for the warm interior looking out at the stars. Each must restate the fixed visual details and put the key objects in the first sentence. Keep it concise, practical, and buildable. Avoid naming real architects or brands. Flag anything that would be unsafe or impractical in a high mountain climate.

Photoreal long-exposure night view of a compact vertical-cedar cabin with a white telescope dome and steel stovepipe on a rocky mountain ridge, one warm amber window, a small deck, a stone-lined path, dwarf pines and old snow, under a vivid Milky Way core. Step 2 of the Mountain Observatory Cabin workflow.
A small cedar cabin with a white rotating telescope dome on its roof sits on a rocky mountain ridge under a vivid Milky Way arching across the night sky. The cabin is a compact single-story box clad in vertical warm-brown cedar boards, with a low dark-metal shed roof, a matte white hemispherical dome on the left end with its narrow slit half open and a telescope tube just visible inside, and a black steel stovepipe chimney on the right end with a thin curl of smoke. One wide rectangular window on the front glows dim warm amber, and a small wooden deck with a simple railing faces the valley. A narrow gravel path lined with flat stones leads up to the deck. Scattered dwarf pines and patches of old snow surround the cabin, and a jagged ridge silhouette fades into a deep valley of distant peaks. Thousands of sharp stars, the bright galactic core in soft violet and gold. Photorealistic long-exposure astrophotography, wide-angle 16:9, low camera angle, no light pollution, no people, no text.

Photoreal night interior of the same cedar observatory cabin: desk with star charts and a red lamp, telescope eyepiece to the white dome hatch, cast-iron wood stove with stovepipe, leather armchair, and two amber front windows framing the mountain ridge and Milky Way. Step 3 of the Mountain Observatory Cabin workflow.
Interior room of a small cedar observatory cabin at night: walls and ceiling of warm vertical cedar boards matching the exterior, a wooden desk with open star charts and a red-glowing desk lamp, a telescope eyepiece and black telescope tube angled up through a ceiling hatch into the white hemispherical dome above, and a small black cast-iron wood stove with orange firelight and a black steel stovepipe. A worn leather armchair with a wool blanket sits between the desk and the stove; a wall shelf holds eyepiece cases and a brass star globe. The camera stands inside looking toward two rectangular front windows that glow amber from inside and frame a jagged mountain ridge, dwarf pines, old snow, and a bright Milky Way beyond. Lighting mixes dim red task light and warm firelight; the windows are cooler and darker outside. Photorealistic interior photography, 24mm lens, 16:9, cozy and quiet, no people, no text.
Picks the correct HTTP status code for an API response from a plain description of what happened (created, validation failed, not found, conflict, rate limited, upstream down) and explains why, with common mistakes to avoid. Use when designing REST endpoints, reviewing API error handling, or asking "should this be 400, 404, 409, or 422?". Includes a stdlib Python suggester.
---
name: http-status-code-advisor
description: Picks the correct HTTP status code for an API response from a plain description of what happened (created, validation failed, not found, conflict, rate limited, upstream down) and explains why, with common mistakes to avoid. Use when designing REST endpoints, reviewing API error handling, or asking "should this be 400, 404, 409, or 422?". Includes a stdlib Python suggester.
---
# HTTP Status Code Advisor
You help API designers and reviewers choose the HTTP status code that best matches what actually happened during a request. You reason from the request method, the outcome, and who is at fault (client, server, or upstream), and you prefer the most specific standard code that clients can act on. You follow RFC 9110 semantics and do not invent non-standard codes.
## Files in this skill
- `scripts/suggest_status.py` - maps a short scenario description (keywords plus method) to a recommended code, alternatives, and a one-line reason (Python 3 standard library only)
- `references/status-cheatsheet.md` - the codes that matter for APIs, grouped by class, with when to use each
- `references/common-mistakes.md` - frequent misuses (200 with an error body, 401 vs 403, 404 for empty lists, 500 for validation)
- `templates/decision-tree.md` - a step-by-step question flow to reach a code
- `examples/api-scenarios.md` - worked scenarios with the chosen code and response shape
## Workflow
### 1. Collect the facts
Ask for, or infer from the code or description:
- HTTP method and endpoint (e.g. `POST /orders`)
- What happened: success, partial success, async accepted, input problem, auth problem, missing resource, state conflict, rate limit, server or dependency failure
- Whether a response body is returned, and whether a new resource was created
- Any existing API conventions the team already follows
### 2. Walk the decision tree
Use `templates/decision-tree.md`. Decide the class first (2xx, 3xx, 4xx, 5xx), then the specific code. Who caused the failure decides 4xx vs 5xx.
### 3. Run the helper (optional)
```
python3 scripts/suggest_status.py --method POST "created new order"
python3 scripts/suggest_status.py --method GET "user not logged in"
python3 scripts/suggest_status.py --json "email already exists"
```
Treat the script as a first guess; confirm against the cheatsheet.
### 4. Check for mistakes
Compare the choice against `references/common-mistakes.md`, especially 401 vs 403, 400 vs 422, and 404 vs an empty 200 list.
### 5. Answer
Return:
1. **Recommended code** and reason phrase (e.g. `409 Conflict`)
2. **Why** in one or two sentences, tied to the facts
3. **Alternatives** considered and why they lose
4. **Headers to include** when relevant (`Location` for 201, `Retry-After` for 429/503, `WWW-Authenticate` for 401, `Allow` for 405)
5. **Error body sketch** using RFC 9457 Problem Details (`type`, `title`, `status`, `detail`, `instance`) for 4xx/5xx
## Rules
- Never return 2xx for a failed operation.
- Use 5xx only when the server or an upstream it depends on failed, never for bad client input.
- Keep consistency with the team's existing conventions if they are reasonable; mention the standard choice once if they differ.
- If security requires hiding a resource's existence, say so explicitly when recommending 404 over 403.
FILE:references/common-mistakes.md
# Common Status Code Mistakes
1. **200 with an error body** (`{"success": false}`). Clients, proxies, caches, and monitoring treat 200 as success. Use the right 4xx/5xx and put details in the body.
2. **401 vs 403 confusion.** 401 means "who are you?" (missing or bad credentials). 403 means "I know who you are, and the answer is no." A logged-in user without permission gets 403, not 401.
3. **500 for bad input.** Validation failures are the client's fault: 400 or 422. Reserve 500 for bugs and unexpected server failures. A spike in 500s should mean something is broken on your side.
4. **404 for an empty collection.** `GET /users?role=admin` with no matches is `200` with `[]`. 404 is for a missing resource such as `/users/9999`.
5. **400 for everything.** Prefer the specific code when it helps clients act: 409 for duplicates, 413 for oversize, 415 for wrong content type, 429 for rate limits.
6. **201 without `Location`.** When you create a resource with its own URL, send `Location: /orders/123`.
7. **204 with a body.** 204 must not include content. If you return the resource, use 200.
8. **302 in APIs.** Clients may change POST to GET. Use 307 or 308 when the method must be preserved.
9. **503 vs 500.** 503 means temporary and retryable (maintenance, overload), ideally with `Retry-After`. 500 means an unexpected error.
10. **Leaking existence with 403.** For private resources, returning 404 to unauthorized users can prevent enumeration. Decide deliberately and document it.
11. **Using 422 and 400 inconsistently.** Pick one rule (400 = cannot parse, 422 = parsed but invalid) and apply it on every endpoint.
12. **Non-standard codes** (e.g. 418 in production, 499, custom 6xx). Many clients and proxies handle them poorly. Stick to registered codes.
FILE:references/status-cheatsheet.md
# HTTP Status Cheatsheet for APIs
Based on RFC 9110 (HTTP Semantics), RFC 6585 (429), and RFC 9457 (Problem Details).
## 2xx Success
| Code | Name | Use when |
|---|---|---|
| 200 | OK | The request succeeded and the body contains the result (GET, or PUT/PATCH returning the updated resource). |
| 201 | Created | A new resource was created. Send `Location` with its URL; body usually contains the resource. |
| 202 | Accepted | Work was queued and will finish later. Return a status URL or job id. |
| 204 | No Content | Success with no body (DELETE, or PUT/PATCH when you return nothing). |
| 206 | Partial Content | Serving a byte range in response to a `Range` header. |
## 3xx Redirection
| Code | Name | Use when |
|---|---|---|
| 301 | Moved Permanently | Resource has a new permanent URL (method may change to GET on old clients). |
| 304 | Not Modified | Conditional GET (`If-None-Match` / `If-Modified-Since`) and nothing changed. |
| 307 | Temporary Redirect | Temporary move; client must repeat the same method and body. |
| 308 | Permanent Redirect | Permanent move; same method and body preserved. |
## 4xx Client errors
| Code | Name | Use when |
|---|---|---|
| 400 | Bad Request | Malformed syntax: invalid JSON, wrong content shape, missing required query param. |
| 401 | Unauthorized | No credentials or invalid/expired credentials. Send `WWW-Authenticate`. |
| 403 | Forbidden | Authenticated, but not allowed to do this. |
| 404 | Not Found | The resource does not exist (or you hide that it exists). |
| 405 | Method Not Allowed | The URL exists but not for this method. Send `Allow`. |
| 406 | Not Acceptable | Cannot produce any format listed in `Accept`. |
| 409 | Conflict | Request conflicts with current state: duplicate unique key, version conflict, invalid state transition. |
| 410 | Gone | Resource existed and was permanently removed. |
| 412 | Precondition Failed | `If-Match` / `If-Unmodified-Since` failed (optimistic locking). |
| 413 | Content Too Large | Body exceeds the size limit. |
| 415 | Unsupported Media Type | `Content-Type` not supported. |
| 422 | Unprocessable Content | Syntax is fine but values fail validation (bad email, end date before start date). |
| 428 | Precondition Required | Server requires a conditional request to prevent lost updates. |
| 429 | Too Many Requests | Rate limit exceeded. Send `Retry-After`. |
## 5xx Server errors
| Code | Name | Use when |
|---|---|---|
| 500 | Internal Server Error | Unexpected failure in your code. |
| 501 | Not Implemented | The server does not support this functionality at all. |
| 502 | Bad Gateway | An upstream returned an invalid response. |
| 503 | Service Unavailable | Temporarily overloaded or in maintenance. Send `Retry-After` if known. |
| 504 | Gateway Timeout | An upstream did not respond in time. |
FILE:templates/decision-tree.md
# Status Code Decision Tree
Answer in order. Stop at the first match.
1. **Did the request succeed?**
- Yes, work finished and a new resource exists -> **201** (+ `Location`)
- Yes, work accepted but finishes later -> **202** (+ status URL)
- Yes, nothing to return -> **204**
- Yes, returning data -> **200**
- No -> go to 2
2. **Is the client being redirected?**
- Conditional GET, unchanged -> **304**
- Moved, must keep the method -> **307** (temporary) / **308** (permanent)
- No -> go to 3
3. **Is it an authentication problem?**
- No credentials, or invalid / expired token -> **401** (+ `WWW-Authenticate`)
- No -> go to 4
4. **Is the authenticated caller not allowed?**
- Yes, and existence can be revealed -> **403**
- Yes, but existence must be hidden -> **404**
- No -> go to 5
5. **Is the target missing or the method wrong?**
- Resource does not exist -> **404**
- Resource removed permanently -> **410**
- URL exists, method unsupported -> **405** (+ `Allow`)
- No -> go to 6
6. **Is the request itself unusable?**
- Cannot parse (bad JSON, wrong types) -> **400**
- Wrong `Content-Type` -> **415**
- Body too large -> **413**
- Cannot satisfy `Accept` -> **406**
- No -> go to 7
7. **Is the input valid but rejected?**
- Fails business validation -> **422**
- Conflicts with current state (duplicate, version, state machine) -> **409**
- `If-Match` precondition failed -> **412**
- Too many requests -> **429** (+ `Retry-After`)
- No -> go to 8
8. **Server side failure**
- Upstream returned garbage -> **502**
- Upstream timed out -> **504**
- Temporarily unavailable / maintenance -> **503** (+ `Retry-After`)
- Feature not supported at all -> **501**
- Unexpected bug -> **500**
## Answer template
```
Recommended: <code> <reason phrase>
Why: <1-2 sentences>
Alternatives: <code> - <why not>; <code> - <why not>
Headers: <header: value>
Body (Problem Details):
{
"type": "https://example.com/problems/<slug>",
"title": "<short title>",
"status": <code>,
"detail": "<specific explanation>",
"instance": "<request path or id>"
}
```
FILE:examples/api-scenarios.md
# Worked API Scenarios
## 1. Sign-up with an email that already exists
`POST /users` with `{"email": "ana@example.com"}` and that email is taken.
- **Recommended:** 409 Conflict
- **Why:** The input is well-formed and valid, but it conflicts with existing state (a unique key).
- **Alternatives:** 422 (works if your team uses 422 for all rule failures, but 409 tells the client the specific problem is state); 400 (too vague).
- **Body:**
```json
{"type": "https://example.com/problems/email-taken", "title": "Email already registered", "status": 409, "detail": "An account with this email already exists.", "instance": "/users"}
```
## 2. Order created
`POST /orders` succeeds and returns the order.
- **Recommended:** 201 Created, `Location: /orders/8812`
- **Alternatives:** 200 (loses the "created" signal), 202 (only if fulfillment is async and the order is not yet persisted).
## 3. Report export queued
`POST /reports/exports` starts a 5-minute job.
- **Recommended:** 202 Accepted with `{"jobId": "exp_41", "statusUrl": "/reports/exports/exp_41"}`
## 4. Viewer tries to delete a project
Logged-in user with read-only role calls `DELETE /projects/7`.
- **Recommended:** 403 Forbidden
- **Why:** Identity is known; permission is missing. 401 would wrongly suggest logging in again.
## 5. Expired access token
`GET /me` with an expired JWT.
- **Recommended:** 401 Unauthorized, `WWW-Authenticate: Bearer error="invalid_token"`
## 6. Search with no matches
`GET /products?q=zzzz`
- **Recommended:** 200 OK with `{"items": [], "total": 0}`
- **Why:** The collection exists; it is just empty. 404 would break clients that treat it as an error.
## 7. Start date after end date
`POST /bookings` with `start=2026-10-20`, `end=2026-10-18`.
- **Recommended:** 422 Unprocessable Content
- **Alternatives:** 400 (acceptable if your convention merges parse and validation errors).
## 8. Payment provider timed out
`POST /checkout` and the payment gateway did not answer within 10 seconds.
- **Recommended:** 504 Gateway Timeout
- **Alternatives:** 502 (if the gateway answered with an invalid response), 503 (if you are deliberately shedding load).
## 9. Too many login attempts
- **Recommended:** 429 Too Many Requests, `Retry-After: 60`
## 10. Stale update
`PUT /docs/3` with `If-Match: "v4"` but the current version is `"v5"`.
- **Recommended:** 412 Precondition Failed
FILE:scripts/suggest_status.py
#!/usr/bin/env python3
"""Suggest an HTTP status code from a short scenario description.
Usage:
python3 suggest_status.py [--method METHOD] [--json] "scenario text"
python3 suggest_status.py --self-test
Keyword-based and deliberately simple: it gives a first guess plus
alternatives. Always confirm against references/status-cheatsheet.md.
Python 3 standard library only.
"""
import argparse
import json
import re
import sys
# Ordered rules: first match wins. Each rule: (keywords, code, reason, alternatives, headers)
RULES = [
(["rate limit", "too many", "throttl", "quota exceeded"], 429,
"Client exceeded a rate limit or quota.", [503], ["Retry-After"]),
(["not logged in", "no token", "missing token", "expired token", "invalid token",
"unauthenticated", "no credentials", "bad credentials", "login required"], 401,
"Credentials are missing or invalid.", [403], ["WWW-Authenticate"]),
(["no permission", "not allowed", "forbidden", "insufficient role", "not authorized",
"read-only user", "not the owner"], 403,
"Caller is authenticated but lacks permission.", [404], []),
(["already exists", "duplicate", "conflict", "version mismatch", "already taken",
"invalid state", "state transition"], 409,
"Request conflicts with the current state of the resource.", [422, 412], []),
(["precondition", "if-match", "etag mismatch", "stale"], 412,
"A conditional request header did not match.", [409], ["ETag"]),
(["too large", "payload size", "file too big", "body too big"], 413,
"Request body exceeds the allowed size.", [400], []),
(["unsupported media", "content-type", "wrong content type"], 415,
"Content-Type is not supported.", [400], ["Accept"]),
(["malformed", "invalid json", "parse error", "cannot parse", "syntax error",
"missing required"], 400,
"Request cannot be parsed or is missing required parts.", [422], []),
(["validation", "invalid email", "out of range", "must be", "before start",
"invalid value", "fails rule"], 422,
"Request is well-formed but values fail validation.", [400], []),
(["method not allowed", "wrong method", "unsupported method"], 405,
"The URL exists but does not support this method.", [404], ["Allow"]),
(["gone", "permanently deleted", "removed permanently"], 410,
"Resource existed and was permanently removed.", [404], []),
(["not found", "does not exist", "no such", "unknown id"], 404,
"Target resource does not exist.", [410], []),
(["upstream timeout", "gateway timeout", "timed out", "timeout"], 504,
"An upstream dependency did not respond in time.", [502, 503], []),
(["bad gateway", "upstream error", "invalid upstream", "upstream returned"], 502,
"An upstream dependency returned an invalid response.", [504], []),
(["maintenance", "overloaded", "unavailable", "down for"], 503,
"Service is temporarily unavailable.", [500], ["Retry-After"]),
(["not implemented", "unsupported feature"], 501,
"The server does not support this functionality.", [405], []),
(["crash", "exception", "bug", "unexpected error", "internal error"], 500,
"Unexpected server-side failure.", [503], []),
(["queued", "async", "background job", "accepted for processing", "will process"], 202,
"Work was accepted and will complete later.", [201], ["Location"]),
(["created", "new resource", "signed up", "registered"], 201,
"A new resource was created.", [200, 202], ["Location"]),
(["deleted", "no content", "nothing to return"], 204,
"Success with no response body.", [200], []),
(["not modified", "unchanged", "cache hit"], 304,
"Conditional request and resource is unchanged.", [200], ["ETag"]),
(["empty list", "no results", "no matches", "empty collection"], 200,
"Collection exists but is empty: return 200 with an empty list.", [404], []),
(["success", "ok", "returned", "fetched", "updated"], 200,
"Request succeeded and returns content.", [204], []),
]
REASONS = {
200: "OK", 201: "Created", 202: "Accepted", 204: "No Content", 304: "Not Modified",
400: "Bad Request", 401: "Unauthorized", 403: "Forbidden", 404: "Not Found",
405: "Method Not Allowed", 409: "Conflict", 410: "Gone", 412: "Precondition Failed",
413: "Content Too Large", 415: "Unsupported Media Type", 422: "Unprocessable Content",
429: "Too Many Requests", 500: "Internal Server Error", 501: "Not Implemented",
502: "Bad Gateway", 503: "Service Unavailable", 504: "Gateway Timeout",
}
def suggest(text, method=None):
t = " " + re.sub(r"\s+", " ", text.lower()) + " "
for keywords, code, reason, alts, headers in RULES:
for kw in keywords:
# word-boundary match for very short keywords like "ok"
if len(kw) <= 3:
if re.search(r"\b" + re.escape(kw) + r"\b", t):
break
elif kw in t:
break
else:
continue
m = (method or "").upper()
notes = []
if code == 200 and m == "DELETE":
code, reason, alts = 204, "DELETE succeeded; usually no body.", [200]
if code == 201 and m and m not in ("POST", "PUT"):
notes.append("201 is normally returned by POST or PUT.")
if code == 204 and m == "GET":
notes.append("GET rarely returns 204; prefer 200 with a body.")
return {
"code": code,
"reason_phrase": REASONS.get(code, ""),
"why": reason,
"alternatives": [f"{a} {REASONS.get(a, '')}".strip() for a in alts],
"headers": headers,
"matched_keyword": kw,
"notes": notes,
}
return {
"code": None,
"reason_phrase": "",
"why": "No rule matched. Walk templates/decision-tree.md manually.",
"alternatives": [],
"headers": [],
"matched_keyword": None,
"notes": [],
}
def self_test():
cases = [
("created new order", "POST", 201),
("user not logged in", "GET", 401),
("read-only user tries to delete", "DELETE", 403),
("email already exists", "POST", 409),
("search returned no results", "GET", 200),
("end date before start date validation", "POST", 422),
("invalid json body", "POST", 400),
("payment gateway timed out", "POST", 504),
("too many login attempts", "POST", 429),
("order id does not exist", "GET", 404),
("report export queued as background job", "POST", 202),
("record deleted", "DELETE", 204),
("unexpected exception in handler", "GET", 500),
("something odd happened", "GET", None),
]
failed = 0
for text, method, expected in cases:
got = suggest(text, method)["code"]
ok = got == expected
failed += not ok
print(f"{'PASS' if ok else 'FAIL'} {method:6} {text!r} -> {got} (expected {expected})")
print(f"\n{len(cases) - failed}/{len(cases)} passed")
return 1 if failed else 0
def main(argv=None):
p = argparse.ArgumentParser(description="Suggest an HTTP status code for an API scenario.")
p.add_argument("scenario", nargs="*", help="Short description of what happened")
p.add_argument("--method", help="HTTP method, e.g. POST")
p.add_argument("--json", action="store_true", help="Print JSON output")
p.add_argument("--self-test", action="store_true", help="Run built-in tests")
a = p.parse_args(argv)
if a.self_test:
return self_test()
if not a.scenario:
p.error("scenario text is required (or use --self-test)")
r = suggest(" ".join(a.scenario), a.method)
if a.json:
print(json.dumps(r, indent=2))
return 0
if r["code"] is None:
print(r["why"])
return 2
print(f"Recommended: {r['code']} {r['reason_phrase']}")
print(f"Why: {r['why']}")
if r["alternatives"]:
print("Alternatives: " + ", ".join(r["alternatives"]))
if r["headers"]:
print("Headers: " + ", ".join(r["headers"]))
for n in r["notes"]:
print("Note: " + n)
return 0
if __name__ == "__main__":
sys.exit(main())