To obtain the RENTRI technical documentation, we developed a custom script named rentri_downloader. The traditional HTML scraping approach was not sufficient as the portal uses a modern asynchronous loading architecture (AJAX/Fetch).

The Problem

By downloading the site's raw HTML, the documentation section appeared empty. By analyzing the network traffic (Network Tab), we identified a direct GET request to internal API endpoints that return data in a clean Markdown format.

The Solution: Rust and Reqwest

We bypassed the graphical interface (and the need for heavy tools like Selenium) by intercepting the API endpoint. Using the language Rust and the library reqwest, we created a fast, resilient, and secure scraper. The script downloads the entire official documentation (including the API Reference) by mimicking browser headers, inserting slight delays to avoid server overload on the government site, and saving the material locally while maintaining the original directory structure.