make_udxf
xorq.flight.exchanger.make_udxf(
process_df,
maybe_schema_in,
maybe_schema_out,
name=None,
description=None,
command=None,
do_wraps=True,
)Create a User-Defined Exchange Function (UDXF) for Apache Arrow Flight.
Creates a UDXF class that can process pandas DataFrames via Arrow Flight protocol. The returned class can be used with flight_udxf() to create distributed data processing operations.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| process_df | callable | Function that takes a pandas DataFrame and returns a transformed DataFrame. | required |
| maybe_schema_in | Schema | callable | Input schema specification. Can be an Ibis Schema or a callable that validates the input schema. | required |
| maybe_schema_out | Schema | callable | Output schema specification. Can be an Ibis Schema or a callable that computes the output schema from the input schema. | required |
| name | str | None | Name for the UDXF. If None, uses the function name from process_df. |
None |
| description | str | None | Description of the UDXF. If None, uses the name. | None |
| command | str | None | Unique command identifier. If None, generates from function tokenization. | None |
| do_wraps | bool | If True, wraps the function with error handling and streaming support. | True |
Returns
| Name | Type | Description |
|---|---|---|
| type | A UDXF class (subclass of AbstractExchanger) that can be instantiated and used with flight_udxf(). |
Examples
from xorq.flight.exchanger import make_udxf
import xorq.api as xo
# Define processing function
def transform_data(df):
return df.assign(new_col=df['old_col'] * 2)
# Create UDXF
udxf_class = make_udxf(
process_df=transform_data,
maybe_schema_in=xo.schema({"old_col": "int64"}),
maybe_schema_out=xo.schema({"old_col": "int64", "new_col": "int64"}),
name="DoubleColumn"
)
# Use with flight_udxf
from xorq.expr.relations import flight_udxf
udxf = flight_udxf(
expr=input_expr,
process_df=transform_data,
maybe_schema_in=xo.schema({"old_col": "int64"}).to_pyarrow(),
maybe_schema_out=xo.schema({"old_col": "int64", "new_col": "int64"}).to_pyarrow()
)