Replace per-request subprocess spawning with a single long-lived duckdb Python connection (in-memory + ATTACH read-only). LOAD httpfs and S3 auth are paid once at startup; object cache accumulates across requests. Benchmarked improvement on remote: Q1 10x, Q2 3x, Q3 9x, Q4 22x faster. Add duckdb==1.5.1 Python package to Dockerfile.
3.2 KiB
3.2 KiB