Performance

Fixing Memory Leaks When Scraping Millions of URLs with aiohttp in Python

by

When running an async web scraper across 500,000 product pages, my server crashed after 45 minutes due to an Out Of Memory (OOM) error. Python RAM usage ballooned from 150 MB to over 7 GB. The Culprit: Unclosed ClientSessions and Unlimited Concurrency Creating a new ClientSession() inside every async task prevents connection reuse and leaks … Read more

Handling Webhook Timeouts in n8n When Processing Heavy AI Workflows

by

When triggering an n8n workflow via an incoming webhook that runs an LLM chain, the HTTP request frequently timed out with a 504 Gateway Timeout because the AI analysis took 20 to 45 seconds. The Architectural Fix: Decoupling Response from Processing Instead of making the client wait for the final AI output, I changed the … Read more