make_udxf

xorq.flight.exchanger.make_udxf(
    process_df,
    maybe_schema_in,
    maybe_schema_out,
    name=None,
    description=None,
    command=None,
    do_wraps=True,
)

Create a User-Defined Exchange Function (UDXF) for Apache Arrow Flight.

Creates a UDXF class that can process pandas DataFrames via Arrow Flight protocol. The returned class can be used with flight_udxf() to create distributed data processing operations.

Parameters

Name Type Description Default
process_df callable Function that takes a pandas DataFrame and returns a transformed DataFrame. required
maybe_schema_in Schema | callable Input schema specification. Can be an Ibis Schema or a callable that validates the input schema. required
maybe_schema_out Schema | callable Output schema specification. Can be an Ibis Schema or a callable that computes the output schema from the input schema. required
name str | None Name for the UDXF. If None, uses the function name from process_df. None
description str | None Description of the UDXF. If None, uses the name. None
command str | None Unique command identifier. If None, generates from function tokenization. None
do_wraps bool If True, wraps the function with error handling and streaming support. True

Returns

Name Type Description
type A UDXF class (subclass of AbstractExchanger) that can be instantiated and used with flight_udxf().

Examples

from xorq.flight.exchanger import make_udxf
import xorq.api as xo

# Define processing function
def transform_data(df):
    return df.assign(new_col=df['old_col'] * 2)

# Create UDXF
udxf_class = make_udxf(
    process_df=transform_data,
    maybe_schema_in=xo.schema({"old_col": "int64"}),
    maybe_schema_out=xo.schema({"old_col": "int64", "new_col": "int64"}),
    name="DoubleColumn"
)

# Use with flight_udxf
from xorq.expr.relations import flight_udxf
udxf = flight_udxf(
    expr=input_expr,
    process_df=transform_data,
    maybe_schema_in=xo.schema({"old_col": "int64"}).to_pyarrow(),
    maybe_schema_out=xo.schema({"old_col": "int64", "new_col": "int64"}).to_pyarrow()
)