From 75aa500d7644f15f4a45fe2f64de1c526b87cf95 Mon Sep 17 00:00:00 2001 From: Sadeep Madurange Date: Sun, 4 Jan 2026 18:17:43 +0800 Subject: Neo4J post update. --- README.txt | 5 ++--- _log/e-reader.md | 36 ++++++++++++++++++-------------- _log/neo4j-a-star-search.md | 19 +++++++++-------- _site/feed.xml | 2 +- _site/log/e-reader/index.html | 36 ++++++++++++++++++-------------- _site/log/neo4j-a-star-search/index.html | 17 ++++++++------- _site/posts.xml | 2 +- 7 files changed, 63 insertions(+), 54 deletions(-) diff --git a/README.txt b/README.txt index b744290..32de791 100644 --- a/README.txt +++ b/README.txt @@ -8,9 +8,8 @@ $ cd /var/www/htdocs # chown -R www:www www.asciimx.com # chmod -R 444 www.asciimx.com/ # chmod u+x,g+x www.asciimx.com/ -# cd www.asciimx.com/ -# find . -type d -exec chmod 554 {} + -# chmod 554 cgi-bin/find.cgi +# find www.asciimx.com/ -type d -exec chmod 554 {} + +# chmod 554 www.asciimx.com/cgi-bin/find.cgi Search lock setup in chroot diff --git a/_log/e-reader.md b/_log/e-reader.md index 2edd5dd..fbc7b94 100644 --- a/_log/e-reader.md +++ b/_log/e-reader.md @@ -12,25 +12,26 @@ First project with e-paper displays and ESP32. -ESP-WROOM-32, 7.5" Waveshare e-paper display, three buttons (prev/next/sleep). +System: ESP-WROOM-32, 7.5" Waveshare e-paper display, three buttons +(prev/next/sleep). -No local storage—streams books over HTTP. RTC memory tracks reading progress -between sessions. +2023-09-23: 512KB SRAM, 4MB flash (shared with FreeRTOS and ESP-IDF). Not +enough storage for books. Using ESP32's built-in WiFi to stream them over HTTP. +Recording the reading progress in RTC memory. -ESP32: 512KB SRAM, 4MB flash (shared with FreeRTOS, ESP-IDF). Not enough to -store books. Stream from webserver instead. +Rasterized pages encoded as tightly packed bitmaps (1 byte => 8 pixels), no +headers, arranged along 48 KB (480x800) boundaries; Reader can stream pages +using HTTP Range requests with zero server-side logic. Keeps both ends lean. -Custom EBM file format. Rasterized monochrome bitmaps, one per page (480x800). -One byte = 8 pixels. Pages on 48KB boundaries—HTTP Range requests work without -server logic. pdftoebm.py converts PDFs. +Page table stores 3 pages (prev/current/next) in a circular buffer. When the +user requests a page, program cycles through the table, updates the screen, and +downloads the next page. -Circular buffer holds 3 pages (prev/current/next)--page table. Single-threaded -approach too slow—user input, SPI, and HTTP on one core causes lag. +Responsiveness is inadequate. Scheduling GPIO (user input), SPI, and HTTP on +one core causes input lag. Pinned the GPIO and SPI tasks to one core, the HTTP +task to the other core. -Optimizations: pin the GPIO task (responding to user input and updating -display) to one core, pin the HTTP task to the other core. - -Better, but system unresponsive during screen updates. Made SPI transfer async: +Better, but screen updates are blocking user input; Made SPI transfer async: ``` void epd_draw_async(const uint8_t *buf, size_t n) @@ -59,11 +60,14 @@ void epd_draw_async(const uint8_t *buf, size_t n) } ``` -Much better. Squeeze a few more cycles by moving SPI buffer to DMA. +Much better. + +2023-10-06: Moved the page table to DMA-capable memory; Reclaimed a few more +CPU cycles from the SPI transfers. Can't think of anything else. -Outcome: Works but limited. Led to [Etlas](../etlas/). +Verdict: Works but limited. Led to [Etlas](../etlas/). Commit: [7f691c4](https://git.asciimx.com/esp32-e-reader/commit/?id=7f691c46093933b67aab466c0ca582ace8ab73d4) diff --git a/_log/neo4j-a-star-search.md b/_log/neo4j-a-star-search.md index 3233428..fe6f33f 100644 --- a/_log/neo4j-a-star-search.md +++ b/_log/neo4j-a-star-search.md @@ -4,13 +4,11 @@ date: 2018-03-06 layout: post --- -Replaced Dijkstra's search for vessel route tracking in Neo4J. +Work project. Marine vessel tracking with Neo4J hit a limit. Need to store +13,000 route points; Dijkstra's shortest path search slows after 4,000. -Tracking 13,000 marine vessel route points. Needed shortest paths between ports -for arrival prediction. Neo4j's Dijkstra's algorithm slows after 4,000 route -points. - -Implemented A* search using haversine function as heuristic: +Replaced Dijkstra's algorithm with A* search using haversine function as +heuristic: ``` private double computeHeuristic( @@ -50,7 +48,10 @@ private void updateCosts( } ``` -Outcome: 300x speedup. Scaled to 13,000 route points. Upstreamed changes. Neo4J v3.4.0. -[Full source](https://github.com/neo4j-contrib/neo4j-graph-algorithms/blob/bd9732d9a690319552e134708692acb5a0d6b37c/algo/src/main/java/org/neo4j/graphalgo/impl/ShortestPathAStar.java) +class="external" target="_blank" rel="noopener noreferrer">Neo4J v3.4.0 | +[Full +source](https://github.com/neo4j-contrib/neo4j-graph-algorithms/blob/bd9732d9a690319552e134708692acb5a0d6b37c/algo/src/main/java/org/neo4j/graphalgo/impl/ShortestPathAStar.java) diff --git a/_site/feed.xml b/_site/feed.xml index cb9e91e..989f4b2 100644 --- a/_site/feed.xml +++ b/_site/feed.xml @@ -1 +1 @@ -Jekyll2026-01-04T17:31:30+08:00/feed.xmlASCIIMX | LogW. D. Sadeep MadurangeSearch engine (Perl + FastCGI + SA)2026-01-03T00:00:00+08:002026-01-03T00:00:00+08:00/log/site-searchW. D. Sadeep MadurangeMatrix Rain: 2025 refactor2025-12-21T00:00:00+08:002025-12-21T00:00:00+08:00/log/matrix-digital-rainW. D. Sadeep MadurangeFingerprint door lock (LP)2025-08-18T00:00:00+08:002025-08-18T00:00:00+08:00/log/fpm-door-lock-lpW. D. Sadeep MadurangeHigh-side MOSFET switching2025-06-22T00:00:00+08:002025-06-22T00:00:00+08:00/log/mosfet-switchesW. D. Sadeep MadurangeATmega328P at 3.3V and 5V2025-06-10T00:00:00+08:002025-06-10T00:00:00+08:00/log/arduino-unoW. D. Sadeep MadurangeFingerprint door lock (RF)2025-06-05T00:00:00+08:002025-06-05T00:00:00+08:00/log/fpm-door-lock-rfW. D. Sadeep MadurangeBumblebee: browser automation2025-04-02T00:00:00+08:002025-04-02T00:00:00+08:00/log/bumblebeeW. D. Sadeep MadurangeBare-metal ATSAM3X8E2024-09-16T00:00:00+08:002024-09-16T00:00:00+08:00/log/arduino-dueW. D. Sadeep MadurangeEtlas: e-paper dashboard2024-09-05T00:00:00+08:002024-09-05T00:00:00+08:00/log/etlasW. D. Sadeep MadurangeESP32 e-reader prototype2023-10-24T00:00:00+08:002023-10-24T00:00:00+08:00/log/e-readerW. D. Sadeep Madurange \ No newline at end of file +Jekyll2026-01-04T22:36:11+08:00/feed.xmlASCIIMX | LogW. D. Sadeep MadurangeSearch engine (Perl + FastCGI + SA)2026-01-03T00:00:00+08:002026-01-03T00:00:00+08:00/log/site-searchW. D. Sadeep MadurangeMatrix Rain: 2025 refactor2025-12-21T00:00:00+08:002025-12-21T00:00:00+08:00/log/matrix-digital-rainW. D. Sadeep MadurangeFingerprint door lock (LP)2025-08-18T00:00:00+08:002025-08-18T00:00:00+08:00/log/fpm-door-lock-lpW. D. Sadeep MadurangeHigh-side MOSFET switching2025-06-22T00:00:00+08:002025-06-22T00:00:00+08:00/log/mosfet-switchesW. D. Sadeep MadurangeATmega328P at 3.3V and 5V2025-06-10T00:00:00+08:002025-06-10T00:00:00+08:00/log/arduino-unoW. D. Sadeep MadurangeFingerprint door lock (RF)2025-06-05T00:00:00+08:002025-06-05T00:00:00+08:00/log/fpm-door-lock-rfW. D. Sadeep MadurangeBumblebee: browser automation2025-04-02T00:00:00+08:002025-04-02T00:00:00+08:00/log/bumblebeeW. D. Sadeep MadurangeBare-metal ATSAM3X8E2024-09-16T00:00:00+08:002024-09-16T00:00:00+08:00/log/arduino-dueW. D. Sadeep MadurangeEtlas: e-paper dashboard2024-09-05T00:00:00+08:002024-09-05T00:00:00+08:00/log/etlasW. D. Sadeep MadurangeESP32 e-reader prototype2023-10-24T00:00:00+08:002023-10-24T00:00:00+08:00/log/e-readerW. D. Sadeep Madurange \ No newline at end of file diff --git a/_site/log/e-reader/index.html b/_site/log/e-reader/index.html index 23d409b..a5dbc01 100644 --- a/_site/log/e-reader/index.html +++ b/_site/log/e-reader/index.html @@ -46,25 +46,26 @@ -

ESP-WROOM-32, 7.5” Waveshare e-paper display, three buttons (prev/next/sleep).

+

System: ESP-WROOM-32, 7.5” Waveshare e-paper display, three buttons +(prev/next/sleep).

-

No local storage—streams books over HTTP. RTC memory tracks reading progress -between sessions.

+

2023-09-23: 512KB SRAM, 4MB flash (shared with FreeRTOS and ESP-IDF). Not +enough storage for books. Using ESP32’s built-in WiFi to stream them over HTTP. +Recording the reading progress in RTC memory.

-

ESP32: 512KB SRAM, 4MB flash (shared with FreeRTOS, ESP-IDF). Not enough to -store books. Stream from webserver instead.

+

Rasterized pages encoded as tightly packed bitmaps (1 byte => 8 pixels), no +headers, arranged along 48 KB (480x800) boundaries; Reader can stream pages +using HTTP Range requests with zero server-side logic. Keeps both ends lean.

-

Custom EBM file format. Rasterized monochrome bitmaps, one per page (480x800). -One byte = 8 pixels. Pages on 48KB boundaries—HTTP Range requests work without -server logic. pdftoebm.py converts PDFs.

+

Page table stores 3 pages (prev/current/next) in a circular buffer. When the +user requests a page, program cycles through the table, updates the screen, and +downloads the next page.

-

Circular buffer holds 3 pages (prev/current/next)–page table. Single-threaded -approach too slow—user input, SPI, and HTTP on one core causes lag.

+

Responsiveness is inadequate. Scheduling GPIO (user input), SPI, and HTTP on +one core causes input lag. Pinned the GPIO and SPI tasks to one core, the HTTP +task to the other core.

-

Optimizations: pin the GPIO task (responding to user input and updating -display) to one core, pin the HTTP task to the other core.

- -

Better, but system unresponsive during screen updates. Made SPI transfer async:

+

Better, but screen updates are blocking user input; Made SPI transfer async:

void epd_draw_async(const uint8_t *buf, size_t n)
 {
@@ -92,11 +93,14 @@ display) to one core, pin the HTTP task to the other core.

 }
-

Much better. Squeeze a few more cycles by moving SPI buffer to DMA.

+

Much better.

+ +

2023-10-06: Moved the page table to DMA-capable memory; Reclaimed a few more +CPU cycles from the SPI transfers.

Can’t think of anything else.

-

Outcome: Works but limited. Led to Etlas.

+

Verdict: Works but limited. Led to Etlas.

Commit: 7f691c4

diff --git a/_site/log/neo4j-a-star-search/index.html b/_site/log/neo4j-a-star-search/index.html index 00f7886..c8c4ce6 100644 --- a/_site/log/neo4j-a-star-search/index.html +++ b/_site/log/neo4j-a-star-search/index.html @@ -40,13 +40,11 @@

NEO4J SHORTEST PATH OPTIMIZATION

06 MARCH 2018

-

Replaced Dijkstra’s search for vessel route tracking in Neo4J.

+

Work project. Marine vessel tracking with Neo4J hit a limit. Need to store +13,000 route points; Dijkstra’s shortest path search slows after 4,000.

-

Tracking 13,000 marine vessel route points. Needed shortest paths between ports -for arrival prediction. Neo4j’s Dijkstra’s algorithm slows after 4,000 route -points.

- -

Implemented A* search using haversine function as heuristic:

+

Replaced Dijkstra’s algorithm with A* search using haversine function as +heuristic:

private double computeHeuristic(
     final double lat1, final double lon1,
@@ -84,8 +82,11 @@ points.

 }
-

Outcome: 300x speedup. Scaled to 13,000 route points. Upstreamed changes. Neo4J v3.4.0. -Full source

+

Outcome: 300x speedup. Scaled to 13,000 route points.

+ +

Upstreamed changes: Neo4J v3.4.0 | +Full +source

by W. D. Sadeep Madurange

diff --git a/_site/posts.xml b/_site/posts.xml index b7913a1..47d8e4d 100644 --- a/_site/posts.xml +++ b/_site/posts.xml @@ -1 +1 @@ -Jekyll2026-01-04T17:31:30+08:00/posts.xmlASCIIMXW. D. Sadeep Madurange \ No newline at end of file +Jekyll2026-01-04T22:36:11+08:00/posts.xmlASCIIMXW. D. Sadeep Madurange \ No newline at end of file -- cgit v1.2.3