Backend connection issues
This page covers connection timeouts, authentication errors, and file or path not found when using backends. By the end, you will know how to verify credentials and fix common connection failures.
Verifying credentials
Connection failures usually come from missing credentials, wrong environment variables, or network or firewall issues.
What to check:
- AWS: Check
AWS_ACCESS_KEY_IDand related environment variables, or IAM roles if you run on EC2 or ECS. - Database backends: See the backend-specific sections below and the Environment variables reference.
Troubleshooting
Issue: PostgreSQL connection fails
Cause: Connection-related environment variables are not set or are wrong.
Solution: Check that the variables are set:
echo $POSTGRES_HOST
echo $POSTGRES_DATABASEIf they are empty, set them or use a profile before running your code. See Environment variables for the full list.
Issue: Snowflake or Trino connection fails
Cause: Wrong account, user, or password, or network blocked by firewall or VPN.
Solution: Confirm account, user, and password (and warehouse, database, and schema for Snowflake). Verify network access to the service.
Issue: Connection timeout
Error: ArrowIOError: Connection timeout after 30s
Cause: The operation took too long or the network is unreachable.
Solution:
- Use streaming to reduce load:
data.to_pyarrow_batches(chunk_size=100000). - Check network connectivity (for example,
ping s3.amazonaws.comfor S3). - For local work, try the embedded or DuckDB backend:
con = xo.duckdb.connect().
Issue: File not found, for example S3
Error: FileNotFoundError: s3://bucket/path/file.csv not found
Cause: Wrong path, typo in bucket or key, or missing IAM permissions.
Solution:
- Confirm the path:
aws s3 ls s3://bucket/path/. - Check for typos in the bucket or key.
- Check that IAM permissions include
s3:GetObject(ands3:ListBucketif you list objects).
Issue: Authentication failed
Error: ClientError: Access Denied (or similar).
Cause: Credentials are missing, wrong, or lack permission for the operation.
Solution:
- Verify credentials (environment variables or profile). For AWS:
print(os.environ.get("AWS_ACCESS_KEY_ID")). - Check IAM permissions for the operation (read, write, or list as needed).
- On EC2 or ECS, prefer IAM roles instead of long-lived keys.
Next steps
- Environment variables for backend configuration
- Overview for all topics