OPA policy testing, profiling, and benchmarks

Contents

  1. 1. Tests
  2. 2. Profiling
  3. 3. Benchmarks

Policy without tests drifts. Slow policy burns SLOs. OPA ships first-class test, profile, and bench — here’s how I use them on real Rego.

Tests

Any rule named test_* is a test case.

They’re still rules — you can query them. Naming files *_test.rego keeps intent obvious.

From the quick-start RBAC package:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
package example_rbac_test

import data.example_rbac

test_not_allow {
not example_rbac.allow with input as {}
}

test_allow {
example_rbac.allow with input as {
"action": {
"operation": "read",
"resource": "widgets",
},
"subject": {"user": "inspector-alice"},
}
}

with mocks input (and can reshape data). It doesn’t apply to functions — plan around that.

1
2
3
4
5
cd quick-start
opa test . -v
# data.example_rbac_test.test_not_allow: PASS (693.201µs)
# data.example_rbac_test.test_allow: PASS (562.02µs)
# PASS: 2/2

Coverage: opa test . -c. Gate CI: opa test . -c --threshold 100. vscode-opa paints coverage in the editor.

Run one test: opa test -r test_allow . -v

Profiling

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
cd quick-start
opa eval --profile -d example.rego -d data.json -i input.json -f pretty "data.example_rbac.allow"

true
+------------------------------+---------+
| METRIC | VALUE |
+------------------------------+---------+
| timer_rego_data_parse_ns | 18332 |
| timer_rego_load_files_ns | 5506307 |
| timer_rego_module_compile_ns | 573408 |
| timer_rego_module_parse_ns | 5226456 |
| timer_rego_query_compile_ns | 87390 |
| timer_rego_query_eval_ns | 292198 |
| timer_rego_query_parse_ns | 277932 |
+------------------------------+---------+
+----------+----------+----------+-------------------------+
| TIME | NUM EVAL | NUM REDO | LOCATION |
+----------+----------+----------+-------------------------+
| 54.121µs | 1 | 1 | data.example_rbac.allow |
| 42.776µs | 1 | 2 | example.rego:15 |
| 37.861µs | 1 | 1 | example.rego:9 |
| 35.81µs | 1 | 1 | example.rego:25 |
| 22.469µs | 1 | 1 | example.rego:10 |
| 21.353µs | 2 | 2 | example.rego:16 |
| 16.888µs | 2 | 1 | example.rego:17 |
| 15.923µs | 2 | 1 | example.rego:23 |
| 14.736µs | 1 | 2 | example.rego:22 |
| 8.798µs | 1 | 1 | example.rego:24 |
+----------+----------+----------+-------------------------+

Sort/limit: --profile-sort (see sort criteria), --profile-limit (default 10 lines).

Benchmarks

opa bench feels like Go’s testing benchmark — memory and query timers included.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
$ opa bench --count 1 -d example.rego -d data.json -i input.json -f pretty "data.example_rbac.allow"
+-------------------------------------------+------------+
| samples | 14162 |
| ns/op | 93655 |
| B/op | 15117 |
| allocs/op | 311 |
| histogram_timer_rego_query_eval_ns_75% | 89900 |
| histogram_timer_rego_query_eval_ns_90% | 112253 |
| histogram_timer_rego_query_eval_ns_95% | 125465 |
| histogram_timer_rego_query_eval_ns_99% | 222404 |
| histogram_timer_rego_query_eval_ns_99.9% | 549291 |
| histogram_timer_rego_query_eval_ns_99.99% | 550611 |
| histogram_timer_rego_query_eval_ns_count | 14162 |
| histogram_timer_rego_query_eval_ns_max | 550611 |
| histogram_timer_rego_query_eval_ns_mean | 76735 |
| histogram_timer_rego_query_eval_ns_median | 68336 |
| histogram_timer_rego_query_eval_ns_min | 38896 |
| histogram_timer_rego_query_eval_ns_stddev | 38828 |
+-------------------------------------------+------------+

In tests: opa test -v --bench example.rego example_test.rego data.json

--format gobench feeds benchstat:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
opa test -v --bench --count 5 --format gobench example.rego example_test.rego data.json | tee b.txt

BenchmarkDataExampleRbacTestTestNotAllow 15186 87349 ns/op 73644 timer_rego_query_eval_ns/op 14663 B/op 270 allocs/op
BenchmarkDataExampleRbacTestTestAllow 10000 115857 ns/op 101565 timer_rego_query_eval_ns/op 17828 B/op 367 allocs/op
--------------------------------------------------------------------------------
PASS: 2/2
...

benchstat b.txt

name time/op
DataExampleRbacTestTestNotAllow 92.9µs ±40%
DataExampleRbacTestTestAllow 123µs ± 8%

name timer_rego_query_eval_ns/op
DataExampleRbacTestTestNotAllow 79.0k ±41%
DataExampleRbacTestTestAllow 109k ± 9%

name alloc/op
DataExampleRbacTestTestNotAllow 14.7kB ± 0%
DataExampleRbacTestTestAllow 17.8kB ± 0%

name allocs/op
DataExampleRbacTestTestNotAllow 270 ± 0%
DataExampleRbacTestTestAllow 367 ± 0%

Compare revisions with benchstat old.txt new.txt.

Tools help; structure matters. Official performance takeaways:

  • Cut iteration and search in policy code
    • Prefer objects keyed by stable IDs over repeated array scans
    • Consider partial evaluation to linearize non-linear rules
  • Write for rule indexing
  • Profile before micro-optimizing
  • Benchmark on realistic inputs when latency regresses

Next: bundles — packaging policy + data for distributed OPA.

More in the OPA series.

Questions? Leave a comment below or email: newbvirgil@gmail.com