Review policy

A review policy decides which pull requests the GitHub App reviews on its own, which ones wait until a maintainer asks, and which are never reviewed. It runs in serve when a webhook arrives. You set it per repository in the dashboard, and for new repositories as a server default.

The policy is about the App's reviews. A remote review from a laptop or from CI looks at a diff, not at a pull request, so it does not apply it.

Three actions

Templates

Rules

A policy is an ordered list of rules. For each pull request the first rule whose conditions all hold decides the action. When no rule holds, the last setting, Any other pull request, decides. A rule can test:

The author's relation is GitHub's own author_association:

A rule that asks about something GitHub did not send never matches.

Asking for a review

A pull request that waits, or one that no rule reviews, can be started three ways:

• The request label. Add co-maintainer:review to the pull request. Adding a label already needs triage rights, so nothing else is checked.
• The request comment. Comment /co-maintainer review as the first words of a comment on the pull request. Only people in the allowed set can ask, which is owners, members and collaborators unless you change it.
• Review now. On the dashboard, a pull request that is waiting shows a Review now button on the Activity page, under Pull requests we skipped. Whoever is signed in is already allowed to ask, and the button ignores the round limit.

When a label or comment request starts a review, the App reacts with an eyes emoji on the comment or the pull request, so the person who asked can see it was picked up.

A request never overrides Do not review. If a rule skips the pull request, a request is refused and the reason is recorded.

GitHub sends no fork, target branch or size with a comment, so a comment request is checked again when the review starts and the pull request has been fetched. If a rule about those three skips it, or it is over the size limit, the review does not run and no eyes reaction is added.

What a request covers

Comments and reviews on a waiting pull request do not start a review either. The second setting is the one that carries an earlier request forward.

Round limit

Reviews per pull request stops the webhook after that many reviews. A request obeys it too, and the skip reason reads Round limit reached: 2 of 2 reviews. Leave it blank for no limit.

Set it up

In the dashboard

• Repository settings, card Who gets a review. Pick a template, read the plain sentences under In short, and open Edit the rules to change them. The sentences update as you edit, and Save is disabled while the policy is not valid. Editing a template turns it into custom rules.
• Settings, Defaults for new repositories. The template a repository starts with. Custom rules are set with a file, see below.
• Add repository. After the preview, the plan step lists the templates with their descriptions. The server default is selected, and you decide.

On the command line

# The server default for new repositories
co-maintainer set --review-policy=trusted-auto

# A custom policy from a file
co-maintainer set --review-policy-file=policy.json

# Back to no default
co-maintainer set --unset=review-policy

A policy file is JSON. Everything except the rules can be left out:

{
  "rules": [
    { "name": "draft", "when": { "draft": true }, "action": "skip" },
    {
      "name": "first time contributor",
      "when": { "association": ["FIRST_TIME_CONTRIBUTOR", "FIRST_TIMER"] },
      "action": "on-request"
    },
    {
      "name": "small change by anyone we know",
      "when": { "association": ["OWNER", "MEMBER", "COLLABORATOR", "CONTRIBUTOR"], "changedLines": { "max": 300 } },
      "action": "review"
    }
  ],
  "default": "on-request",
  "requestLabel": "co-maintainer:review",
  "requestCommand": "/co-maintainer review",
  "requesters": ["OWNER", "MEMBER", "COLLABORATOR"],
  "approvalScope": "head",
  "maxRounds": 3
}

A policy that cannot be read is refused and says what is wrong, for example unknown template "x" or rules[0].action must be one of review, on-request, skip. Nothing is saved.

Existing repositories

A repository added before policies existed keeps the three switches it had: automatic review on or off, skip drafts, skip bots. They behave exactly as they always did, and the repository shows Simple switches. Choosing a template replaces them. On a repository that follows a policy, the switches are hidden and changing one through the API is refused with a message that says to change the policy instead.

A new repository with no choice follows the server default. With no server default it behaves like Everyone.

Why a pull request was skipped

Activity, Pull requests we skipped names the rule that decided, for example Rule 3 (trusted author): waiting for a maintainer request. or Default rule: waiting for a maintainer request. A rule without a name shows its position only, so name the rules you write.

Permissions

A request needs nothing new. The eyes reaction uses the Issues permission the App already has, and a label request arrives on the pull_request event it already receives. See GitHub App for the permission table.