Increasing Output Buffer Size in Apache Drill UDFs Custom (Simple) Functions

Putting this here to make it easier for others who try to Google this topic to find it w/o having to find and tediously search through other UDFs (user-defined functions).

I was/am making a custom UDF for base64 decoding/encoding and ran into:

SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: #### (expected: range(0, 256))

It’s incredibly easy to “fix” (and, if my Java weren’t so rusty I’d likely have seen it sooner) but I found this idiom in the spatial UDFs for Drill that enables increasing the default buffer size:

buffer = out.buffer = buffer.reallocIfNeeded(outputSize);

Hopefully this will prevent someone else from spinning a few minutes trying to tackle this use-case. I even had looked at the source for the DrillBuf class and did not manage to put 2 + 2 together for some reason.

Cover image from Data-Driven Security
Amazon Author Page

1 Comment Increasing Output Buffer Size in Apache Drill UDFs Custom (Simple) Functions

  1. Irfan

    I was stuck with limited buffer size when developing a UDF in apache drill and this article helped me to resolve it. Thanks for sharing this

    Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.