From e15f1076b59e997108914f6a5b9b28652d323268 Mon Sep 17 00:00:00 2001 From: Sadeep Madurange Date: Sat, 20 Dec 2025 21:36:45 +0800 Subject: Change website structure to a log. --- _log/bumblebee.md | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 _log/bumblebee.md (limited to '_log/bumblebee.md') diff --git a/_log/bumblebee.md b/_log/bumblebee.md new file mode 100644 index 0000000..c26fa0d --- /dev/null +++ b/_log/bumblebee.md @@ -0,0 +1,51 @@ +--- +title: "Bumblebee: browser automation" +date: 2025-04-02 +layout: post +project: true +thumbnail: thumb_sm.png +--- + +Bumblebee is a tool I built for one of my employers to automate the generation +of web scraping scripts. + + + +In 2024, we were tasked with collecting market data using various methods, +including scraping data from authorized websites for traders' use. + +Manual authoring of such scripts took time. The scripts were often brittle due +to the complexity of the modern web, and they lacked optimizations such as +bypassing the UI and retrieving the data files directly when possible, which +would have significantly reduced our compute costs. + +To alleviate these challenges, I, with the help of a colleague, Andy Zhang, +built Bumblebee: a web browser powered by C# Windows Forms, Microsoft Edge WebView2, and +the Scintilla.NET text editor. + +Bumblebee works by injecting a custom JavaScript program that intercepts +client-side events and sends them to Bumblebee for analysis. In addition to +front-end events, Bumblebee also captures internal browser events, which it +then interprets to generate code in real time. Note that we developed Bumblebee +before the advent of now-popular LLMs. Bumblebee supports dynamic websites, +pop-ups, developer tools, live manual override, event debouncing, and filtering +hidden elements and scripts. + +Before settling on a desktop application, we contemplated designing Bumblebee +as a browser extension. We chose the desktop app because extensions don't offer +the deep, event-based control we needed. Besides, the company's security +policy, which prohibited browser extensions, would have complicated the +deployment of an extension-based solution. My first prototype used a C# binding +of the Chromium project. WebView's more intuitive API and its seamless +integration with Windows Forms led us to choose it over the Chromium wrapper. + +What began as a personal side project to improve my own workflow enabled us to +collectively improve the quality of our web scripts at a much larger scale. +Bumblebee predictably reduced the time we spent on authoring scripts from hours +to a few minutes. + -- cgit v1.2.3