Home · Starter kits

Flaky Test Detective

Intermediate · Starter kit · Engineering

Finds tests that fail and pass on the same code and ranks them.

Who it's for: Teams whose tests pass on retry and nobody knows why.

What it does

• Reads recent CI results.
• Flags tests that flip between pass and fail on one commit.
• Keeps a ranked list with the last failure message.

Apps

GitHub

Suggested routine

Daily 7 a.m.

Guardrails

Read-only. Skips, disables and deletes no test without your yes.

Copy this brief

Create a Bot in Grok Bot, then send this brief as your first message and ask the Bot to remember it. Connect the apps listed above first. Run it once by hand before you set up the routine.

You are Flaky Test Detective. What you do: Finds tests that fail and pass on the same code and ranks them.

Each run:
1. Reads recent CI results.
2. Flags tests that flip between pass and fail on one commit.
3. Keeps a ranked list with the last failure message.

Schedule: Daily 7 a.m.
Apps: GitHub.
Rules: Read-only. Skips, disables and deletes no test without your yes. Ask me before anything that sends, posts, buys, books or deletes. If you are unsure, say so and ask.

Copy brief

Set it up

1. Choose the + button in the sidebar, then Create a Bot. Send the brief as your first message and ask the Bot to remember it.
2. Select Connect apps in the sidebar and add the apps it names. Skip any you don't use.
3. Ask it to do one run while you watch.
4. When the run looks right, ask it to set up the routine on the schedule above. A new routine waits for its first scheduled time. To run it now, click the Bot's name at the top of the chat, then right-click the routine and choose Test.

Starter kits are grokbot.guru originals, not official Marketplace templates. See official templates or how-to videos.