OPA Rego comprehensions — object, array, and set

Contents

  1. 1. Comprehensions
    1. 1.1. Object comprehensions
    2. 1.2. Array comprehensions
    3. 1.3. Set comprehensions
    4. 1.4. Exercise

Comprehensions build composite values from inner queries — object, array, or set — in one expression. The official definition is dry; examples stick better.

We’ll aggregate a list of file paths in different ways.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// input.json
{
"files": [
{
"type": "posix",
"path": "/Users/newbmiao/Documents/1.yaml"
},
{
"type": "posix",
"path": "/Users/newbmiao/Documents/2.yaml"
},
{
"type": "traditional-mac",
"path": "Macintosh HD:Users:newbmiao:Documents:3.yml"
},
{
"type": "traditional-mac",
"path": "Macintosh HD:Users:newbmiao:Documents:3.json"
}
]
}

Comprehensions

Syntax recap: { key: value | body }, [ value | body ], { value | body } (set).

Object comprehensions

Group paths by type:

1
2
3
4
5
6
7
8
9
10
11
import input.files

group_files_by_type := {type: paths |
file := files[_]
type := file.type
paths := [path |
tmp := files[_]
tmp.type == type
path := tmp.path
]
}

Outer loop binds type; inner comprehension collects matching paths.

1
2
3
4
5
6
7
8
9
10
11
12
13
opa eval -f values -d . -i input.json "data.example_comprehensions.group_files_by_type"
[
{
"posix": [
"/Users/newbmiao/Documents/1.yaml",
"/Users/newbmiao/Documents/2.yaml"
],
"traditional-mac": [
"Macintosh HD:Users:newbmiao:Documents:3.yml",
"Macintosh HD:Users:newbmiao:Documents:3.json"
]
}
]

Form: { <key>: <term> | <body> }. Every conjunct in <body> must succeed.

Array comprehensions

paths above is an array comprehension: [ <term> | <body> ].

Arrays use [] and allow duplicates.

Normalize everything to posix-style paths:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
convert_all_to_posix_path_array := [path |
path1 := [p |
file := files[_]
file.type == "posix"
p := file.path
]

path2 := [p |
file := files[_]
file.type == "traditional-mac"
p := replace(replace(file.path, "Macintosh HD", ""), ":", "/")
]

paths := array.concat(path1, path2)
path = paths[_]
]

Inner comprehensions filter by file.type; array.concat merges.

Set comprehensions

Same idea, set form — no duplicates:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
convert_all_to_posix_path_sets := {path |
path1 := {p |
file := files[_]
file.type == "posix"
p := file.path
}

path2 := {p |
file := files[_]
file.type == "traditional-mac"
p := replace(replace(file.path, "Macintosh HD", ""), ":", "/")
}

paths := path1 | path2
path = paths[_]
}

Syntax: { <term> | <body> }. Union |, intersection &, difference -.

Once the shapes feel familiar, a lot of aggregation code shrinks to one comprehension.

Exercise

Group by file extension. Target shape:

1
2
3
4
5
6
7
8
9
10
11
12
{
"json": [
"Macintosh HD:Users:newbmiao:Documents:3.json"
],
"yaml": [
"/Users/newbmiao/Documents/1.yaml",
"/Users/newbmiao/Documents/2.yaml"
],
"yml": [
"Macintosh HD:Users:newbmiao:Documents:3.yml"
]
}

Hints — hard-code extensions:

extSets := ["yaml", "yml", "json"]

Or regex:

1
2
3
extSets := {e |
e = regex.find_all_string_submatch_n(".*\\.(.*)$", files[_].path, -1)[0][1]
}

Answers in opa-koans.

Next: tests, profiling, and benchmarks.

Code: comprehensions.rego

Questions? Leave a comment below or email: newbvirgil@gmail.com