Updates
๐ V0.2 - September 1st, 2023
This release includes the following updates to Scrubr:
- Updated scraping engine - In this version we moved the web scraping from HTTParty to Selenium. Instead of straight HTTP requests, each web request is now run through a headless Chrome browser, allowing better scraping of dynamic content that is loaded via Javascript.
- User accounts - You can now create a free Scrubr account! With an account, you are able to save, edit and share your scrubbed pages.
- Enhanced parsing - This release includes an updated parsing engine that has a higher success rate of identifying and removing page menus, footers, navigation elements and other non-text components. ๐งผ
- Chrome extension - The Scrubr Chrome extension is now available! ๐งฉ The extension allows you to right-click on any webpage to send it to Scrubr for cleaning.
๐ฌ In development:
- Tagging - Add tags to categorize and organize your scrubbed pages.
- Scrubbing options - Customize how much scrubbing is done and how elements like headers and links behave.