Bumblebee: browser automation

BUMBLEBEE: BROWSER AUTOMATION

02 APRIL 2025

-
-

Bumblebee is a tool I built for one of my employers to automate the generation -of web scraping scripts.

- -

In 2024, we were tasked with collecting market data using various methods, -including scraping data from authorized websites for traders’ use.

- -

Manual authoring of such scripts took time. The scripts were often brittle due -to the complexity of the modern web, and they lacked optimizations such as -bypassing the UI and retrieving the data files directly when possible, which -would have significantly reduced our compute costs.

- -

To alleviate these challenges, I, with the help of a colleague, Andy Zhang, -built Bumblebee: a web browser powered by C# Windows Forms, Microsoft Edge WebView2, and -the Scintilla.NET text editor.

- -

Bumblebee works by injecting a custom JavaScript program that intercepts -client-side events and sends them to Bumblebee for analysis. In addition to -front-end events, Bumblebee also captures internal browser events, which it -then interprets to generate code in real time. Note that we developed Bumblebee -before the advent of now-popular LLMs. Bumblebee supports dynamic websites, -pop-ups, developer tools, live manual override, event debouncing, and filtering -hidden elements and scripts.

- -

Before settling on a desktop application, we contemplated designing Bumblebee -as a browser extension. We chose the desktop app because extensions don’t offer -the deep, event-based control we needed. Besides, the company’s security -policy, which prohibited browser extensions, would have complicated the -deployment of an extension-based solution. My first prototype used a C# binding -of the Chromium project. WebView’s more intuitive API and its seamless -integration with Windows Forms led us to choose it over the Chromium wrapper.

- -

What began as a personal side project to improve my own workflow enabled us to -collectively improve the quality of our web scripts at a much larger scale. -Bumblebee predictably reduced the time we spent on authoring scripts from hours -to a few minutes.

- -

by W. D. Sadeep Madurange