Replace per-request subprocess spawning with a single long-lived duckdb Python connection (in-memory + ATTACH read-only). LOAD httpfs and S3 auth are paid once at startup; object cache accumulates across requests. Benchmarked improvement on remote: Q1 10x, Q2 3x, Q3 9x, Q4 22x faster. Add duckdb==1.5.1 Python package to Dockerfile.
5.5 KiB
5.5 KiB