Stepmeter for Jira

A working prototype, checked against the Usage tab

Built a command-line prototype that produces the waste report automatically. It reads each flow through the Automation Rule Management API, then counts the events that would have started it from each work item's change history since the flow was created: edits, label changes, status changes and new work items. Changes made by automation are skipped, because they don't start other flows by default. For flows that check labels, it uses the labels each work item had at the time to split runs that passed the check from runs that stopped.

Then I compared it with the Usage tab on the same afternoon. For all 7 flows that react to events, the run counts matched exactly. Steps matched for 5 of them, including the wasted steps: 6 of 15 on the broad flow and 3 of 11 on the flow that edits before checking.

The other two were both web request flows, and they explained the old mystery. A request that succeeded used 3 steps, the same as one that failed, so a web request counts as 2 steps, not 1. The model now uses that.

The remaining gap is the daily scheduled flow: the prototype assumes one run a day and counted 5, but it had run 6 times because I also ran it once by hand. Manual runs don't appear in work item history. After the web request fix, the prototype accounts for 89 of the site's 97 steps, and the 8-step gap is that one manual run.

A first demo report and a one-page brief

Built a first version of the report Stepmeter will give admins, using the test site's flows and run results: total steps, steps on runs that did nothing or failed, the cause in each flow's structure, and a suggested fix. On the test site, 12 of 57 steps (21%) went to runs that did nothing or failed, across three flows.

Wrote a one-page brief that sets out the findings and one idea for the Usage tab itself: a wasted-steps column with plain-language flags such as "edits before checking" and "fails every run".

The API returns each flow's full structure

Ran the spike script against the test site. It listed all 9 flows and fetched each one's configuration through the Automation Rule Management API, using a classic API token with basic authentication on the site address. My first run failed with HTTP 401 because of a typo in the email address, which a login check added to the script now catches.

The response includes what Stepmeter needs to estimate cost per run: each flow's state (enabled or disabled), the trigger type, and every component in order, with branches and their children nested. The trigger type distinguishes a broad "any update" trigger from a narrow field-change trigger, and a scheduled search from an event, which are the patterns behind most of the waste found so far.

Flow configuration from this API can include secrets in plain text (AUTO-2207), and the test site has a flow with a planted fake one. The script masks secret-looking values before saving, and a search of the saved files found no trace of it.

It does not include how often each flow runs. Run frequency will have to come from another source, which is the main open problem for Phase 1.

The web request flow has only one component in the API response, so its extra step on the Usage tab doesn't come from hidden structure. A follow-up test is running.

Loops are billed per work item, and the model holds

Built four more flows overnight, each covering one cost pattern: a branch over sub-tasks, a daily scheduled search over 7 work items, a web request with a fake secret in it, and a flow that edits a work item before checking whether it should. I wrote down a step prediction for every run before Atlassian reported it.

56 of 57 steps matched across 17 runs on 7 flows. The one miss was the web request flow: 3 steps where I predicted 2, on a run where the request failed. I haven't explained that yet.

Branches and scheduled searches are billed per work item. The sub-task branch used 4 steps for 2 sub-tasks, not 3. The daily search used 8 steps per run, one for the trigger and one comment for each of the 7 matching work items. The audit log groups this work into one line per action, so the per-item cost isn't visible there.

Some waste is labelled "Success". When the broad flow stopped at its condition, the audit log said "No actions performed". The flow that edits first and checks second stopped at the same condition on the same work item, but because its edit had already run, the audit log called it "Success". It used 3 steps, didn't do its job, and overwrote the work item's description on the way.

Across the test site, 6 of the broad flow's 15 steps went to runs that did nothing.

First real data from the test site

Built a test site with 30 work items and the first three flows: a simple one (created, then assign), a broad one (any update, then check for a vip label, then comment) and a narrow one (labels changed, then comment). Ran three tests that touched a vip item and a non-vip item.

The step model matched exactly. Counting one step per trigger, condition and action that ran, I predicted 11 steps across 5 runs. The Usage tab reported 11: 2, 2 and 7.

Atlassian already ranks flows by steps. Each space's Automation Usage tab has a "High usage flows" table with runs and estimated steps, sortable by steps. My assumption from the public documentation was wrong, which is what this phase was for. The tab also lags: it took about two hours to show the new runs.

The gap is waste and the fix, not the ranking. The broad flow ran 3 times for 7 steps. Two runs stopped at the condition and did nothing, yet used 4 of the 7 steps. The audit log shows those runs as "No actions performed", but the Usage tab only shows the total. Telling an admin how much of a flow's cost is wasted, and how to rewrite it, needs both the flow's structure and its run results together.

Decision: Stepmeter will focus on wasted steps and how to fix them, not on ranking flows (decision 11).

Phase 0 kit ready, building in public

Set up a public GitHub repository so the work is visible as it happens.

Wrote the Phase 0 kit: 17 test automation rules, each built to show one pattern that changes step cost (broad triggers, conditions placed last, branches over sub-tasks and search results, hourly schedules), a script that lists and reads every rule, and an interview guide for Jira admins.

Because the API returns secrets in full, the script strips URLs, headers and tokens before saving anything. I tested it offline on a sample rule with four planted secrets. All four were removed and the component counts were correct.

From a broad goal to one problem

Started with a broad goal: find a real problem on Atlassian's platform and build something useful for it.

Three early ideas were ruled out by research: two were already shipped and one had no data access. I then reviewed about 20 ideas against three checks, verified the shortlist's key facts against Atlassian's documentation, and chose automation step cost.

Two findings shape the design. The Rule Management API accepts API tokens but not Forge or OAuth apps, so the first version will be a web tool rather than a Marketplace app. And because the API returns rule configuration in full, secret handling is a core requirement.