Repository navigation
P4: properties for permission evaluate() - #27
Conversation
There was a problem hiding this comment.
I noticed you changed the timeout minutes from 10 to 15. The permission tests you wrote seem to be pure functions and should finish pretty quickly. Was the timeout bump to 15 minutes needed for this PR?
| fc.property(rulesetArb, ruleArb, patternCommandArb, (baseRules, newRule, [, command]) => { | ||
| const before = evaluate(PERMISSION, command, baseRules as any) | ||
| const withAdded = [...baseRules, newRule] | ||
| const withRemoved = withAdded.slice(0, -1) |
There was a problem hiding this comment.
From what I can tell, this should still give back baseRules without changing it, since it adds the new rule to the end of the array then slices it off. Could you remove a rule from a random index instead, and check that the result equals referenceEvaluate(command, remaining) and that the removed rule is never what gets returned?
|
nice work, the R3 test that checks evaluate() against a simpler version is a really smart approach. Your traceability file is also easy to follow, and I like that you were upfront about R7 and R8 not being implemented yet. |
|
Hi Anjali, I'm your red-teamer for P5. Here are my three mutants, written against commit RT1 (R1)--- a/packages/opencode/src/permission/index.ts
+++ b/packages/opencode/src/permission/index.ts
@@ -97,7 +97,7 @@ const layer = Layer.effect(
yield* Effect.logInfo("evaluated", { permission: request.permission, pattern, action: rule })
if (rule.action === "deny") {
return yield* new PermissionV1.DeniedError({
- ruleset: ruleset.filter((rule) => Wildcard.match(request.permission, rule.permission)),
+ ruleset: ruleset.filter((rule) => Wildcard.match(pattern, rule.permission)),
})
}
if (rule.action === "allow") continueUses RT2 (R3)--- a/packages/core/src/util/wildcard.ts
+++ b/packages/core/src/util/wildcard.ts
@@ -8,7 +8,7 @@ export function match(input: string, pattern: string) {
.replace(/\*/g, ".*")
.replace(/\?/g, ".")
- if (escaped.endsWith(" .*")) escaped = escaped.slice(0, -3) + "( .*)?"
+ if (escaped.endsWith(" .*")) escaped = escaped.slice(0, -3) + ".*"
return new RegExp("^" + escaped + "$", process.platform === "win32" ? "si" : "s").test(normalized)
}Now RT3 (R5)--- a/packages/opencode/src/permission/index.ts
+++ b/packages/opencode/src/permission/index.ts
@@ -34,7 +34,7 @@ export function evaluate(permission: string, pattern: string, ...rulesets: Permi
return (
rulesets
.flat()
- .findLast((rule) => Wildcard.match(permission, rule.permission) && Wildcard.match(pattern, rule.pattern)) ?? {
+ .findLast((rule) => Wildcard.match(pattern, rule.pattern)) ?? {
action: "ask",
permission,
pattern: "*",Drops the permission check, so a rule for a different permission (like The patches are also in |
Issue for this PR
Type of change
What does this PR do?
Adds fast-check property-based tests for 4 of my Project 3 permission-rule requirements (R3, R4, R5, R6), testing the real evaluate() function in packages/opencode/src/permission/index.ts. Covers: deny-rule consistency against an independent reference model (R3), rule removal round-tripping (R4), rule order determining precedence (R5), and a specific rm * rule overriding a broader catch-all (R6). All 4 properties pass against the current implementation, so no production code changes were needed. Also added traceability/anjalid.md mapping each requirement to its interview evidence and property.
How did you verify your code works?
Ran bun test test/permission/permission.property.test.ts locally
Screenshots / recordings
If this is a UI change, please include a screenshot or recording.
Checklist
If you do not follow this template your PR will be automatically rejected.