Mastodon
Go back

Increasing Output Buffer Size in Apache Drill UDFs Custom (Simple) Functions

Putting this here to make it easier for others who try to Google this topic to find it w/o having to find and tediously search through other UDFs (user-defined functions).

I was/am making a custom UDF for base64 decoding/encoding and ran into:

SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: #### (expected: range(0, 256))

It’s incredibly easy to “fix” (and, if my Java weren’t so rusty I’d likely have seen it sooner) but I found this idiom in the spatial UDFs for Drill that enables increasing the default buffer size:

buffer = out.buffer = buffer.reallocIfNeeded(outputSize);

Hopefully this will prevent someone else from spinning a few minutes trying to tackle this use-case. I even had looked at the source for the DrillBuf class and did not manage to put 2 + 2 together for some reason.


Join the conversation:follow@hrbrmstr@rud.ison the fediverse to read and reply to this post from your own instance.

No comments yet.


Previous Post
Revisiting Readability With RStudio
Next Post
Teasing Out Top Daily Topics with GDELT's Television Explorer