Appendix B: Functions as Values, in Depth
Read this workshop after the collection chapters.
Read this workshop after the collection chapters. You already know how a callback supplies one result to map. Here we examine the function value itself, the distinction between passing and calling it, and functions that produce other functions.
Passing a function and calling it
To total each order line, declare a helper in the header and try handing its name to map:
Example 250 — Returning a function value from map.
Input payload — order.json:
{
"orderId": "A-1001",
"customer": "Dana",
"items": [
{ "sku": "PEN-01", "price": 2.5, "qty": 4 },
{ "sku": "PAD-22", "price": 6.0, "qty": 2 },
{ "sku": "CLP-08", "price": 1.0, "qty": 10 }
]
}%dw 2.0
output application/json
fun lineTotal(item) = item.price * item.qty
---
payload.items map lineTotal[ERROR] Error while executing the script:
[ERROR] Cannot coerce Function to String
3| fun lineTotal(item) = item.price * item.qty
^^^^^^^^^^^^^^^^^^^^^
Trace:
at 250-returning-a-function-value-from-map::main (line: 3, column: 23) at:
3| fun lineTotal(item) = item.price * item.qty
^^^^^^^^^^^^^^^^^^^^^The arrow points at the function’s body, but the mistake is in the call. Each of the three evaluations on the right of map produced lineTotal itself, leaving three function values in the result array. The JSON writer cannot print those values, hence Cannot coerce Function to String. Adding ($) calls the function on the current item and supplies a line total instead. A function can be passed around as a value; it produces that total only when called.
The number of arguments is checked, and the message is odd
A two-parameter function requires two arguments unless one has a default. Supply just one and you get this error:
Example 251 — Call a function with too few arguments.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun discountedBy(price, rate) = price * (1 - rate)
---
discountedBy(20)[ERROR] Error while executing the script:
[ERROR] Expects `2` argument but got `1`. Expecting: (_Undefined0, _Undefined1), but got: (20).
5| discountedBy(20)
^^^^^^^^^^^^^^^^
Location:
251-arity-too-few (line: 5, column:1)_Undefined0 and _Undefined1 are the two parameters, printed under the runtime’s names for them because neither declared a type. Three arguments produce the mirror image, Expects `2` argument but got `3`, with (20, 0.1, 0.5) after the but got. Both were run. The block ends in Location: rather than Trace:, the shape chapter 14 identified as a compile-time rejection. The arity of a call is checked against the declaration before any input is read.
A parameter can carry a default, and then the shorter call is legal:
Example 252 — Use a default parameter.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun discountedBy(price, rate = 0.1) = price * (1 - rate)
---
{ withDefault: discountedBy(20), explicit: discountedBy(20, 0.3) }{
"withDefault": 18,
"explicit": 14
}Chapter 13 uses exactly this on reduce’s accumulator, (item, acc = 0).
The header is a set of declarations, not a sequence of steps
Chapter 3 introduced immutable bindings. Functions can call other functions declared later in the header:
Example 253 — Call a function declared later.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun grossLine(item) = net(item.price) * item.qty
fun net(price) = price * 0.9
---
payload.items map grossLine($)[
9,
10.8,
9
]A var may use a fun declared after it too. This forward reference works for function declarations. This behavior does not make declarations interchangeable with sequential assignments; variable dependency rules remain distinct from function declarations.
One thing the header does not check for you is the same function declared twice with the same shape. This script runs, and prints 18:
Example 254 — Declare the same function signature twice.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun net(price) = price * 0.9
fun net(price) = price * 0.8
---
net(20)18The first declaration won and the second was never called. fun supports overloading, which the next section comes to, and two identical signatures form an overload set in which the first match always wins. So a stale copy near the bottom of a long header can go unnoticed: the script keeps running the first declaration, and nothing warns you. A var declared twice is an error, Duplicated variable — the next section shows it.
A function without a name
A lambda is a function written inline: parameters in parentheses, an arrow, an expression.
Syntax fragment.
(price) -> price * 0.9On its own it does nothing, and the CLI is blunt about a script whose entire body is a lambda:
Example 255 — Try writing a function value as JSON.
Use order.json as payload, as above.
%dw 2.0
output application/json
---
(price) -> price * 0.9[ERROR] Error while executing the script:
[ERROR] Cannot coerce Function to String
4| (price) -> price * 0.9
^^^^^^^^^^^^^^^^^^^^^^
Trace:
at 255-lambda-as-output-fails::main (line: 4, column: 1) at:
4| (price) -> price * 0.9
^^^^^^^^^^^^^^^^^^^^^^The body produced a function value. That value can be stored, passed and called, but the JSON writer cannot serialize it.
A lambda can be bound to a name with var, and its parameters and result can be typed the way a fun’s can:
Example 256 — Annotate a lambda parameter.
Use order.json as payload, as above.
%dw 2.0
output application/json
var lineTotal = (item: Object): Number -> item.price * item.qty
---
{ totals: payload.items map lineTotal($), typeName: typeOf(lineTotal) }{
"totals": [
10,
12,
10
],
"typeName": "Function"
}typeOf(lineTotal) is "Function", confirming that the declaration produced a value that can be passed to another function. Call the typed lambda with the whole array instead of one item, lineTotal(payload.items), and the rejection reads 1: Array against 1: Object, in the You called the function shape from above.
fun or var?
A named fun and a lambda in a var share most calling behavior. Both can be called infix: 20 discountedBy 0.1 printed 18 with discountedBy declared either way. Both take default parameters; the var form printed the same 18 and 14. Both are values: typeOf says "Function" of either, and a fun can be aliased with var alias = lineTotal and called through that alias. Choosing between them means considering overloading, the form used for recursion, and whether the function was declared directly or produced by another call.
fun can be overloaded and var cannot:
Example 257 — Select a function overload.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun describe(n: Number) = "a number: " ++ n
fun describe(s: String) = "a string: " ++ s
fun describe(a: Array) = "an array of " ++ sizeOf(a)
---
[ describe(2.5), describe(payload.customer), describe(payload.items) ][
"a number: 2.5",
"a string: Dana",
"an array of 3"
]Three declarations share a name and the runtime picks by argument type. The same lines written as var describe = ... fail to compile:
[ERROR] Error while executing the script:
[ERROR] Duplicated variable: `describe`.
3| var describe = (n: Number) -> "a number: " ++ n
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Location:
34_var_overload_fails (line: 3, column:1)Now drop the Array case and call describe(true). There is no Boolean overload, so a reasonable guess is an error. The CLI printed "a string: true", because overload resolution goes through chapter 3’s coercion rule. The String parameter accepted the boolean by coercion and that overload was chosen — overloading dispatches on what the argument can become, not only on what it is.
A recursive calculation needs a name it can call again. A named function can use its own name in its body:
Example 258 — Calculate with a recursive function.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun countdown(n) = if (n == 0) [0] else [n] ++ countdown(n - 1)
fun total(items) = if (isEmpty(items)) 0 else items[0].price * items[0].qty + total(items[1 to -1] default [])
---
{ countdown: countdown(3), orderTotal: total(payload.items) }{
"countdown": [
3,
2,
1,
0
],
"orderTotal": 32
}total expresses a fold through recursion: its body calls itself on the remaining items. Chapter 13 shows why reduce, and then sum, are clearer for a plain total. On this runtime a lambda in a var that names itself, var countdown = (n) -> ... countdown(n - 1), also ran and printed the same [3, 2, 1, 0]. I would still write a recursive function as a fun; it is the form the documentation shows and the form a reader expects.
The third difference is a habit rather than a capability. I use fun for a function declared directly in the header. A function produced by calling another function belongs in a var, as the currying example below shows.
Higher-order functions: functions that take functions
map walks an array and applies a function to each element. It does not know or care what the function does; you supply that:
Example 259 — Call a lambda for each line.
Use order.json as payload, as above.
%dw 2.0
output application/json
---
payload.items map (item) -> item.price * item.qty[
10,
12,
10
]A function that takes a function is a higher-order function. map, filter and reduce all qualify; chapters 6, 7 and 13 introduce them separately. For map and filter, the second parameter is the index; for reduce, it is the accumulator.
Handing a named function to map: three ways that work and the one that does not
In the opening payload.items map lineTotal, the infix form evaluates the right-hand expression once per element, with the element and index bound to $ and $$. That expression must produce the result you want for the current element. item.price * item.qty does so when item names the parameter, and lineTotal($) does so by calling the helper. Bare lineTotal evaluates to the function value each time, which leaves the writer with functions it cannot serialise. The working call is:
Example 260 — Call a named function inside map.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun lineTotal(item) = item.price * item.qty
---
payload.items map lineTotal($)[
10,
12,
10
]The prefix form is different, and there the bare name is right:
Example 261 — Pass a function to prefix map.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun lineTotal(item) = item.price * item.qty
---
map(payload.items, lineTotal)[
10,
12,
10
]map(array, function) takes the function as an argument. Passing lineTotal hands over the value, and map does the calling. Although lineTotal declares one parameter, map calls it with two: the element and the index. The extra argument was ignored and the call succeeded.
The same trap applies to a lambda in a var. payload.items map lineTotal with var lineTotal = (item) -> ... fails with the identical Cannot coerce Function to String. This is about infix versus prefix, not about fun versus var. So the rule: infix, call it; prefix, pass it. A bare name on the right of an infix map is the one mistake in this chapter with a misleading message. The arrow points at the function’s body, not at the call that misused it.
Writing your own higher-order function
A parameter can hold a function, and a body can call it:
Example 262 — Declare a function-valued parameter.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun applyTo(items: Array, f: (Any) -> Any): Array = items map f($)
fun lineTotal(item) = item.price * item.qty
---
{ totals: applyTo(payload.items, lineTotal), skus: applyTo(payload.items, (item) -> item.sku) }{
"totals": [
10,
12,
10
],
"skus": [
"PEN-01",
"PAD-22",
"CLP-08"
]
}(Any) -> Any is a function type: one parameter of any type, any result. Inside applyTo, f is called the infix way, f($), because items map f would hit the opening error from inside your own function. The annotation earns its keep when someone passes the wrong thing:
Example 263 — Reject an incompatible function argument.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun applyTo(items: Array, f: (Any) -> Any): Array = items map f($)
---
applyTo(payload.items, 3)[ERROR] Error while executing the script:
[ERROR] You called the function 'applyTo' with these arguments:
1: Array ([{sku: "PEN-01",price: 2.5,qty: 4}, {sku: "PAD-22",price: 6,qty: 2}, {sku: "C...)
2: Number (3)
But it expects arguments of these types:
1: Array
2: Function
5| applyTo(payload.items, 3)
^^^^^^^
Trace:
at 263-function-typed-param-rejects::main (line: 5, column: 1) at:
5| applyTo(payload.items, 3)
^^^^^^^2: Function in the expected list. Without the annotation the failure would have happened inside the body, when 3($) was attempted, with a message about 3 rather than about applyTo.
The positional shorthands: $, $$, $$$
For a one-line lambda, a parameter name often adds nothing. Inside an implicit callback for a higher-order function, DataWeave binds the callback arguments to positional names: $ is the first, $$ the second, $$$ the third.
Example 264 — Inspect map shorthand parameters.
Use order.json as payload, as above.
%dw 2.0
output application/json
---
{
totals: payload.items map $.price * $.qty,
numbered: payload.items map { line: $$ + 1, sku: $.sku },
afterFirst: payload.items filter ($$ > 0) map $.sku
}{
"totals": [
10,
12,
10
],
"numbered": [
{
"line": 1,
"sku": "PEN-01"
},
{
"line": 2,
"sku": "PAD-22"
},
{
"line": 3,
"sku": "CLP-08"
}
],
"afterFirst": [
"PAD-22",
"CLP-08"
]
}For map and filter that is element and index. What $$ and $$$ mean depends entirely on what the function passes, and the object functions pass three things:
Example 265 — Inspect object callback shorthand.
Use order.json as payload, as above.
%dw 2.0
output application/json
---
{
renamed: payload.items[0] mapObject { (upper($$)): $ },
positions: payload.items[0] mapObject { ($$ ++ "_" ++ $$$): $ },
plucked: payload.items[0] pluck { key: $$, value: $, index: $$$ }
}{
"renamed": {
"SKU": "PEN-01",
"PRICE": 2.5,
"QTY": 4
},
"positions": {
"sku_0": "PEN-01",
"price_1": 2.5,
"qty_2": 4
},
"plucked": [
{
"key": "sku",
"value": "PEN-01",
"index": 0
},
{
"key": "price",
"value": 2.5,
"index": 1
},
{
"key": "qty",
"value": 4,
"index": 2
}
]
}In mapObject and pluck the order is value, key, index: $, $$, $$$. The parentheses around upper($$) make it a dynamic key; chapter 10 explains those. Ask map for a third argument it does not pass and the name simply does not exist:
Example 266 — Reject a third map parameter.
Use order.json as payload, as above.
%dw 2.0
output application/json
---
payload.items map { sku: $.sku, index: $$, third: $$$ }[ERROR] Error while executing the script:
[ERROR] Unable to resolve reference of: `$$$`.
4| payload.items map { sku: $.sku, index: $$, third: $$$ }
^^^
Location:
266-reject-a-third-map-parameter (line: 4, column:51)Where $ does not exist
The examples here bind the shorthands inside implicit callbacks. They are not ambient variables available anywhere in a script. Three places people reach for $ and find nothing, all run:
- With named parameters.
payload.items map (item) -> $.skufails withUnable to resolve reference of: `$`. Once you name the parameters the positional names are gone — it is one or the other. - In a
funbody.fun lineTotal(item) = $.price * $.qtyfails the same way, twice, once per$. A function body is not an infix right-hand side. - In a prefix call.
map(payload.items, $.price * $.qty)fails identically, and so doesmap(payload.items, lineTotal($)). The prefix form takes a function argument, and there is nothing to bind$to.
Chapter 9’s Nesting rebinds $, silently section shows why nested callbacks should use named parameters. The same scope rule applies to function values here.
Infix or prefix: the same call, read two ways
payload.items map lineTotal($) reads as a sentence because map is being called infix: left operand, function name, right operand. Every two-parameter function can be called that way, including your own, and every infix call has a prefix spelling:
Example 267 — Compare prefix and infix calls.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun lineTotal(item) = item.price * item.qty
fun discountedBy(price, rate) = price * (1 - rate)
---
{
infix: payload.items map lineTotal($),
prefix: map(payload.items, (item) -> lineTotal(item)),
ownInfix: 20 discountedBy 0.1,
ownPrefix: discountedBy(20, 0.1)
}{
"infix": [
10,
12,
10
],
"prefix": [
10,
12,
10
],
"ownInfix": 18,
"ownPrefix": 18
}Infix is for exactly two parameters. A three-parameter function has no infix form, and trying one is an arity error rather than a syntax error:
Example 268 — Reject a three-argument infix call.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun clamp(x, lo, hi) = max([lo, min([x, hi])])
---
5 clamp 1[ERROR] Error while executing the script:
[ERROR] Expects `3` argument but got `2`. Expecting: (_Undefined0, _Undefined1, _Undefined2), but got: (5, 1).
5| 5 clamp 1
^^^^^^^^^
Location:
268-reject-a-three-argument-infix-call (line: 5, column:1)The parser accepted 5 clamp 1 as a two-argument call and the arity check rejected it. The choice between the two forms is about reading. A chain of transforms reads top to bottom as infix — payload.items filter ($.qty >= 4) map $.sku ran and printed ["PEN-01", "CLP-08"] — while the prefix spelling of the same chain reads inside out. Prefix earns its place when the right-hand side is a function value you already hold rather than an expression.
Currying: functions that return functions
A discount calculation can fix the rate now and accept the price later. Its body is a lambda, so calling it produces a new function that retains the chosen rate:
Example 269 — Return a function with a bound discount.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun discount(rate) = (price) -> price * (1 - rate)
var tenOff = discount(0.1)
---
{ single: tenOff(20), direct: discount(0.1)(20), typeName: typeOf(tenOff) }{
"single": 18,
"direct": 18,
"typeName": "Function"
}discount(0.1) hands back a function that remembers the rate and still wants a price. tenOff holds it and tenOff(20) finishes the job; discount(0.1)(20) does both in one expression. This is partial application, and it is the case where a lambda in a var is the natural form — nobody wrote tenOff out, discount produced it.
The produced function goes anywhere a function goes, including into map:
Example 270 — Map with a curried function.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun discount(rate) = (price) -> price * (1 - rate)
var clearance = discount(0.3)
---
payload.items map clearance($.price)[
1.75,
4.2,
0.7
]Make discount(0.3) the script’s output and the CLI prints Cannot coerce Function to String, with the arrow under the lambda in discount’s body. The same message has now appeared in three settings because each sent a function value to the JSON writer; the particular route through a lambda or map did not change that failure.
Composition, and why the order is not decoration
Chaining two transforms is nesting two calls: withTax(net(20)) runs net first and withTax on its result. When the pairing repeats, capture it as a function that takes two functions and returns their composition:
Example 271 — Compose two calculations.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun net(p) = p * 0.9
fun withTax(p) = p * 1.2
fun compose(f, g) = (x) -> f(g(x))
var netThenTax = compose(withTax, net)
---
{
nested: withTax(net(20)),
composed: netThenTax(20),
lines: payload.items map netThenTax($.price * $.qty)
}{
"nested": 21.6,
"composed": 21.6,
"lines": [
10.8,
12.96,
10.8
]
}compose takes two functions whose types line up and returns a third, without mentioning a price. Read compose(f, g) from the inner call outward: g runs first, then f, just as in f(g(x)). netThenTax is a var because another function produced it. I name composed functions in execution order like this, so a caller can read the order directly without reversing the arguments in their head.
Whether the order matters depends on the functions. net and withTax both multiply, and multiplication commutes, so compose(net, withTax)(20) is the same 21.6. A test built from those two would pass whichever way round you wrote it. Replace one with a subtraction and the order shows:
Example 272 — Change the order of composition.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun lessFive(p) = p - 5
fun tax(rate) = (p) -> p * (1 + rate)
fun compose(f, g) = (x) -> f(g(x))
var discountThenTax = compose(tax(0.2), lessFive)
var taxThenDiscount = compose(lessFive, tax(0.2))
---
{ discountThenTax: discountThenTax(20), taxThenDiscount: taxThenDiscount(20) }{
"discountThenTax": 18,
"taxThenDiscount": 19
}A flat discount before tax is (20 - 5) * 1.2; after tax it is 20 * 1.2 - 5. One unit of difference on a twenty-unit order, no error, and a tax authority with a view on which is correct. tax(0.2) is the curried function from the previous section handed straight to compose. The resulting transform combines small functions that can be produced and passed as values. When a function body needs a local name along the way, do gives it a header of its own; chapter 5 introduced that scope.
Exercises
Your own map. Write applyTo(items, f) that applies f to every element of items, without a type annotation. Use it twice on Dana’s items: once with a named fun that returns the line total, once with a lambda that returns the SKU. Run it, then explain why the body has to be items map f($) and not items map f.
Show answer
Example 273 — Apply a function argument.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun applyTo(items, f) = items map f($)
fun lineTotal(item) = item.price * item.qty
---
{
totals: applyTo(payload.items, lineTotal),
skus: applyTo(payload.items, (item) -> item.sku)
}{
"totals": [
10,
12,
10
],
"skus": [
"PEN-01",
"PAD-22",
"CLP-08"
]
}Inside applyTo, f is a function value. items map f would evaluate f once per element and collect three function values, which is the chapter’s opening error; f($) calls it on the element. At the call site, applyTo(payload.items, lineTotal) is prefix, so the bare name is right. Pass it in, call it inside.
Order of composition. Using net(p) = p * 0.9 and tax(rate) = (p) -> p * (1 + rate), build netThenTax and taxThenNet with compose and run both on 20. Then say, before running, what happens if net becomes lessFive(p) = p - 5.
Show answer
Example 274 — Compose discount and tax.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun net(p) = p * 0.9
fun tax(rate) = (p) -> p * (1 + rate)
fun compose(f, g) = (x) -> f(g(x))
var netThenTax = compose(tax(0.2), net)
var taxThenNet = compose(net, tax(0.2))
---
{ netThenTax: netThenTax(20), taxThenNet: taxThenNet(20) }{
"netThenTax": 21.6,
"taxThenNet": 21.6
}Both are 21.6, because 0.9 * 1.2 * 20 is the same number in either order. With lessFive in place of net the two diverge to 18 and 19, as the composition section showed. A test of composition order needs a pair of functions that do not commute, or it tests nothing.
The missing overload. Declare describe for Number and for String only, and call describe(true). Predict the result before running, then explain it with chapter 5’s scalar-parameter coercion rule.
Show answer
Example 275 — Call an overload with a Boolean.
Use order.json as payload, as above.
%dw 2.0
output application/json
fun describe(n: Number) = "a number: " ++ n
fun describe(s: String) = "a string: " ++ s
---
describe(true)"a string: true"No error. A typed parameter accepts any argument that can be coerced to its type, and a Boolean coerces to a String, so the String overload matched and ran. If the intent was to reject anything that is not a number or a string, overloading will not do it; an explicit is check will.
These techniques are available when they clarify a reusable calculation. A direct named function and explicit parameters remain good choices when the extra abstraction would make the caller harder to follow.
Comments