diff options
| author | Sadeep Madurange <sadeep@asciimx.com> | 2025-12-24 16:29:32 +0800 |
|---|---|---|
| committer | Sadeep Madurange <sadeep@asciimx.com> | 2025-12-24 16:29:32 +0800 |
| commit | 1b991a54cc834e8ef9ccc8bb15dce7ff70cdf8a3 (patch) | |
| tree | a9efb89b4f08341ca6ce22fcff4ede20ef72931a /_site | |
| parent | 08e594268ed20c5c2355a249ac691c007e38aed9 (diff) | |
| download | www-1b991a54cc834e8ef9ccc8bb15dce7ff70cdf8a3.tar.gz | |
Matrix post.
Diffstat (limited to '_site')
| -rw-r--r-- | _site/feed.xml | 2 | ||||
| -rw-r--r-- | _site/log/index.html | 13 | ||||
| -rw-r--r-- | _site/log/matrix-digital-rain/index.html | 125 | ||||
| -rw-r--r-- | _site/log/neo4j-a-star-search/index.html | 370 | ||||
| -rw-r--r-- | _site/posts.xml | 2 | ||||
| -rw-r--r-- | _site/robots.txt | 2 | ||||
| -rw-r--r-- | _site/sitemap.xml | 32 |
7 files changed, 467 insertions, 79 deletions
diff --git a/_site/feed.xml b/_site/feed.xml index 4d5229e..c7b21b4 100644 --- a/_site/feed.xml +++ b/_site/feed.xml @@ -1 +1 @@ -<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="http://localhost:4000/feed.xml" rel="self" type="application/atom+xml" /><link href="http://localhost:4000/" rel="alternate" type="text/html" /><updated>2025-12-22T23:37:38+08:00</updated><id>http://localhost:4000/feed.xml</id><title type="html">ASCIIMX | Log</title><author><name>W. D. Sadeep Madurange</name></author><entry><title type="html">Recreating the Matrix rain with ANSI escape sequences</title><link href="http://localhost:4000/log/matrix-digital-rain/" rel="alternate" type="text/html" title="Recreating the Matrix rain with ANSI escape sequences" /><published>2025-12-21T00:00:00+08:00</published><updated>2025-12-21T00:00:00+08:00</updated><id>http://localhost:4000/log/matrix-digital-rain</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[The Matrix digital rain implemented in raw C using ANSI escape sequences with zero dependencies—not even ncurses.]]></summary></entry><entry><title type="html">How to manage Suckless software installations</title><link href="http://localhost:4000/log/suckless-software/" rel="alternate" type="text/html" title="How to manage Suckless software installations" /><published>2025-11-30T00:00:00+08:00</published><updated>2025-11-30T00:00:00+08:00</updated><id>http://localhost:4000/log/suckless-software</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Since suckless software requires users to modify the source code and recompile to customize, I need a way to maintain patches over the long term while retaining the ability to upgrade the software as new versions are released.]]></summary></entry><entry><title type="html">Fingerprint door lock</title><link href="http://localhost:4000/log/fpm-door-lock/" rel="alternate" type="text/html" title="Fingerprint door lock" /><published>2025-08-18T00:00:00+08:00</published><updated>2025-08-18T00:00:00+08:00</updated><id>http://localhost:4000/log/fpm-door-lock</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[This project features a fingerprint door lock powered by an ATmega328P microcontroller.]]></summary></entry><entry><title type="html">On the use of MOSFETs as electronic switches</title><link href="http://localhost:4000/log/mosfet-switches/" rel="alternate" type="text/html" title="On the use of MOSFETs as electronic switches" /><published>2025-06-22T00:00:00+08:00</published><updated>2025-06-22T00:00:00+08:00</updated><id>http://localhost:4000/log/mosfet-switches</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Recently, I needed a low-power circuit for one of my battery-operated projects. Much of the system’s power savings depended on its ability to electronically switch off components, such as servos, that draw high levels of quiescent currents. My search for a solution led me to MOSFETs, transistors capable of controlling circuits operating at voltages far above their own.]]></summary></entry><entry><title type="html">How to configure ATmega328P microcontrollers to run at 3.3V and 5V</title><link href="http://localhost:4000/log/arduino-uno/" rel="alternate" type="text/html" title="How to configure ATmega328P microcontrollers to run at 3.3V and 5V" /><published>2025-06-10T00:00:00+08:00</published><updated>2025-06-10T00:00:00+08:00</updated><id>http://localhost:4000/log/arduino-uno</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[This is a quick reference for wiring up ATmega328P ICs to run at 5V and 3.3V. While the 5V configuration is common, the 3.3V configuration can be useful in low-power applications and when interfacing with parts that themselves run at 3.3V. In this guide, the 5V setup is configured with a 16MHz crystal oscillator, while the 3.3V configuration makes use of an 8MHz crystal oscillator.]]></summary></entry><entry><title type="html">My first PCB</title><link href="http://localhost:4000/log/my-first-pcb/" rel="alternate" type="text/html" title="My first PCB" /><published>2025-04-26T00:00:00+08:00</published><updated>2025-04-26T00:00:00+08:00</updated><id>http://localhost:4000/log/my-first-pcb</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[In 2023, I started tinkering with DIY electronics as a hobby. Until now, I’ve been using development boards like the Arduino Uno and ESP-32-WROOM so that I can focus on the software. Recently, I decided to step outside of my comfort zone and design a PCB from scratch for a door lock I’m working on.]]></summary></entry><entry><title type="html">Bumblebee: browser automation</title><link href="http://localhost:4000/log/bumblebee/" rel="alternate" type="text/html" title="Bumblebee: browser automation" /><published>2025-04-02T00:00:00+08:00</published><updated>2025-04-02T00:00:00+08:00</updated><id>http://localhost:4000/log/bumblebee</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Bumblebee is a tool I built for one of my employers to automate the generation of web scraping scripts.]]></summary></entry><entry><title type="html">How to set up ATSAM3X8E microcontrollers for bare-metal programming in C</title><link href="http://localhost:4000/log/arduino-due/" rel="alternate" type="text/html" title="How to set up ATSAM3X8E microcontrollers for bare-metal programming in C" /><published>2024-09-16T00:00:00+08:00</published><updated>2024-09-16T00:00:00+08:00</updated><id>http://localhost:4000/log/arduino-due</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[This article is a step-by-step guide for programming bare-metal ATSAM3X8E chips found on Arduino Due boards. It also includes notes on the chip’s memory layout relevant for writing linker scripts. The steps described in this article were tested on an OpenBSD workstation.]]></summary></entry><entry><title type="html">Etlas: e-paper dashboard</title><link href="http://localhost:4000/log/etlas/" rel="alternate" type="text/html" title="Etlas: e-paper dashboard" /><published>2024-09-05T00:00:00+08:00</published><updated>2024-09-05T00:00:00+08:00</updated><id>http://localhost:4000/log/etlas</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Etlas is a news, stock market, and weather tracker powered by an ESP32 NodeMCU D1, featuring a 7.5-inch Waveshare e-paper display and a DHT22 sensor module.]]></summary></entry><entry><title type="html">Experimental e-reader</title><link href="http://localhost:4000/log/e-reader/" rel="alternate" type="text/html" title="Experimental e-reader" /><published>2023-10-24T00:00:00+08:00</published><updated>2023-10-24T00:00:00+08:00</updated><id>http://localhost:4000/log/e-reader</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[This project features an experimental e-reader powered by an ESP-WROOM-32 development board and a 7.5-inch Waveshare e-paper display built with the intention of learning about e-paper displays.]]></summary></entry></feed>
\ No newline at end of file +<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="/feed.xml" rel="self" type="application/atom+xml" /><link href="/" rel="alternate" type="text/html" /><updated>2025-12-24T16:28:25+08:00</updated><id>/feed.xml</id><title type="html">ASCIIMX | Log</title><author><name>W. D. Sadeep Madurange</name></author><entry><title type="html">Recreating the Matrix rain with ANSI escape sequences</title><link href="/log/matrix-digital-rain/" rel="alternate" type="text/html" title="Recreating the Matrix rain with ANSI escape sequences" /><published>2025-12-21T00:00:00+08:00</published><updated>2025-12-21T00:00:00+08:00</updated><id>/log/matrix-digital-rain</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[My 2022 implementation of the Matrix rain had too many loose ends. Unicode support was inflexible: the charset had to be a single contiguous block with no way to mix ASCII with something like Katakana; Phosphor decay level was stored in a dedicated array–still don’t understand why I did that when I had already used bit-packing for the RGB channels; The algorithm was difficult to decipher. The 2022 version worked, but that’s not the same thing as being correct.]]></summary></entry><entry><title type="html">How to manage Suckless software installations</title><link href="/log/suckless-software/" rel="alternate" type="text/html" title="How to manage Suckless software installations" /><published>2025-11-30T00:00:00+08:00</published><updated>2025-11-30T00:00:00+08:00</updated><id>/log/suckless-software</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Since suckless software requires users to modify the source code and recompile to customize, I need a way to maintain patches over the long term while retaining the ability to upgrade the software as new versions are released.]]></summary></entry><entry><title type="html">Fingerprint door lock</title><link href="/log/fpm-door-lock/" rel="alternate" type="text/html" title="Fingerprint door lock" /><published>2025-08-18T00:00:00+08:00</published><updated>2025-08-18T00:00:00+08:00</updated><id>/log/fpm-door-lock</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[This project features a fingerprint door lock powered by an ATmega328P microcontroller.]]></summary></entry><entry><title type="html">On the use of MOSFETs as electronic switches</title><link href="/log/mosfet-switches/" rel="alternate" type="text/html" title="On the use of MOSFETs as electronic switches" /><published>2025-06-22T00:00:00+08:00</published><updated>2025-06-22T00:00:00+08:00</updated><id>/log/mosfet-switches</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Recently, I needed a low-power circuit for one of my battery-operated projects. Much of the system’s power savings depended on its ability to electronically switch off components, such as servos, that draw high levels of quiescent currents. My search for a solution led me to MOSFETs, transistors capable of controlling circuits operating at voltages far above their own.]]></summary></entry><entry><title type="html">How to configure ATmega328P microcontrollers to run at 3.3V and 5V</title><link href="/log/arduino-uno/" rel="alternate" type="text/html" title="How to configure ATmega328P microcontrollers to run at 3.3V and 5V" /><published>2025-06-10T00:00:00+08:00</published><updated>2025-06-10T00:00:00+08:00</updated><id>/log/arduino-uno</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[This is a quick reference for wiring up ATmega328P ICs to run at 5V and 3.3V. While the 5V configuration is common, the 3.3V configuration can be useful in low-power applications and when interfacing with parts that themselves run at 3.3V. In this guide, the 5V setup is configured with a 16MHz crystal oscillator, while the 3.3V configuration makes use of an 8MHz crystal oscillator.]]></summary></entry><entry><title type="html">My first PCB</title><link href="/log/my-first-pcb/" rel="alternate" type="text/html" title="My first PCB" /><published>2025-04-26T00:00:00+08:00</published><updated>2025-04-26T00:00:00+08:00</updated><id>/log/my-first-pcb</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[In 2023, I started tinkering with DIY electronics as a hobby. Until now, I’ve been using development boards like the Arduino Uno and ESP-32-WROOM so that I can focus on the software. Recently, I decided to step outside of my comfort zone and design a PCB from scratch for a door lock I’m working on.]]></summary></entry><entry><title type="html">Bumblebee: browser automation</title><link href="/log/bumblebee/" rel="alternate" type="text/html" title="Bumblebee: browser automation" /><published>2025-04-02T00:00:00+08:00</published><updated>2025-04-02T00:00:00+08:00</updated><id>/log/bumblebee</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Bumblebee is a tool I built for one of my employers to automate the generation of web scraping scripts.]]></summary></entry><entry><title type="html">How to set up ATSAM3X8E microcontrollers for bare-metal programming in C</title><link href="/log/arduino-due/" rel="alternate" type="text/html" title="How to set up ATSAM3X8E microcontrollers for bare-metal programming in C" /><published>2024-09-16T00:00:00+08:00</published><updated>2024-09-16T00:00:00+08:00</updated><id>/log/arduino-due</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[This article is a step-by-step guide for programming bare-metal ATSAM3X8E chips found on Arduino Due boards. It also includes notes on the chip’s memory layout relevant for writing linker scripts. The steps described in this article were tested on an OpenBSD workstation.]]></summary></entry><entry><title type="html">Etlas: e-paper dashboard</title><link href="/log/etlas/" rel="alternate" type="text/html" title="Etlas: e-paper dashboard" /><published>2024-09-05T00:00:00+08:00</published><updated>2024-09-05T00:00:00+08:00</updated><id>/log/etlas</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Etlas is a news, stock market, and weather tracker powered by an ESP32 NodeMCU D1, featuring a 7.5-inch Waveshare e-paper display and a DHT22 sensor module.]]></summary></entry><entry><title type="html">Experimental e-reader</title><link href="/log/e-reader/" rel="alternate" type="text/html" title="Experimental e-reader" /><published>2023-10-24T00:00:00+08:00</published><updated>2023-10-24T00:00:00+08:00</updated><id>/log/e-reader</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[This project features an experimental e-reader powered by an ESP-WROOM-32 development board and a 7.5-inch Waveshare e-paper display built with the intention of learning about e-paper displays.]]></summary></entry></feed>
\ No newline at end of file diff --git a/_site/log/index.html b/_site/log/index.html index 32f0ff2..4296fb7 100644 --- a/_site/log/index.html +++ b/_site/log/index.html @@ -174,6 +174,19 @@ + <tr> + <td class="posts-td posts-td-link"> + <a href="/log/neo4j-a-star-search/" class="link-decor-none">Neo4J A* search</a> + </td> + <td class="posts-td posts-td-time"> + <span class="post-meta"> + <time datetime="2018-03-06 00:00:00 +0800">2018-03-06</time> + </span> + </td> + </tr> + + + </table> </div> diff --git a/_site/log/matrix-digital-rain/index.html b/_site/log/matrix-digital-rain/index.html index 6007da5..4a23d29 100644 --- a/_site/log/matrix-digital-rain/index.html +++ b/_site/log/matrix-digital-rain/index.html @@ -44,40 +44,60 @@ <h2 class="center" id="title">RECREATING THE MATRIX RAIN WITH ANSI ESCAPE SEQUENCES</h2> <h6 class="center">21 DECEMBER 2025</h5> <br> - <div class="twocol justify"><p>The Matrix digital rain implemented in raw C using ANSI escape sequences with -zero dependencies—not even ncurses.</p> - -<video style="max-width:100%;" controls="" poster="poster.png"> - <source src="matrix.mp4" type="video/mp4" /> -</video> + <div class="twocol justify"><p>My 2022 implementation of the Matrix rain had too many loose ends. Unicode +support was inflexible: the charset had to be a single contiguous block with no +way to mix ASCII with something like Katakana; Phosphor decay level was stored +in a dedicated array–still don’t understand why I did that when I had already +used bit-packing for the RGB channels; The algorithm was difficult to decipher. +The 2022 version worked, but that’s not the same thing as being correct.</p> + +<p>I began by placing the decay factor in the LSB of the 4-byte RGB value. Let’s +call that RGB-PD. PD plays a somewhat analogous role to an alpha channel; I +avoided labelling it A so as not to cause confusion:</p> + +<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>enum { + R, /* Red */ + G, /* Green */ + B, /* Blue */ + PD /* Phosphor decay level */ +}; -<p>This is a fork of Domsson’s unique rendition of the Matrix rain: <a href="https://github.com/domsson/fakesteak" class="external" target="_blank" rel="noopener noreferrer">Fakesteak</a>. Three years ago, I forked his project -and added truecolor and Unicode support. I also drastically modified the -algorithm to produce a rain that resembled the original aesthetic with high -visual fidelity.</p> +typedef union color_tag { + uint32_t value; + unsigned char color[4]; +} color; +</code></pre></div></div> -<h2 id="unicode-support">Unicode support</h2> +<p>The decision to use union over more portable bit twiddling was made three years +ago, as I recall, for readability. Seeing as all my systems are little-endian, +this is unlikely to cause me trouble. Besides, if union is never to be used, +why is it in the language anyway?</p> -<p>Unicode support in the 2022 version lacked flexibility. The charset used in the -rain had to be a single contiguous block defined by <code class="language-plaintext highlighter-rouge">UNICODE_MIN</code> and -<code class="language-plaintext highlighter-rouge">UNICODE_MAX</code> settings:</p> +<p>The blend() function, which emulates the dim afterglow of Phosphor by eroding +the RGB channels towards the background, remains as elegant as it did three +years ago:</p> -<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#define UNICODE_MIN 0x0021 -#define UNICODE_MAX 0x007E +<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#define DECAY_MPLIER 2 -static inline void insert_code(matrix *mat, - size_t row, size_t col) +static inline void blend(matrix *mat, + size_t row, size_t col) { - mat->code[index(mat, row, col)] = rand() - % (UNICODE_MAX - UNICODE_MIN) - + UNICODE_MIN; + unsigned char *color; + + color = mat->rgb[index(mat, row, col)].color; + color[R] = color[R] - (color[R] - RGB_BG_RED) / DECAY_MPLIER; + color[G] = color[G] - (color[G] - RGB_BG_GRN) / DECAY_MPLIER; + color[B] = color[B] - (color[B] - RGB_BG_BLU) / DECAY_MPLIER; } </code></pre></div></div> -<p>There was no way, for instance, to use both ASCII and Katakana at the same -time. The user had to pick one. In the new version, the user can use any number -of Unicode blocks using <code class="language-plaintext highlighter-rouge">glyphs</code> array. In fact, the default rain now includes -both ASCII and half-width Katakana characters:</p> +<p>While the memory inefficiency of Phosphor decay tracking was a technical +oversight I hadn’t noticed, the limitation around mixing nonadjacent Unicode +blocks was a nagging concern even three years ago. So, a fix was long overdue.</p> + +<p>In the new version, I introduced a glyphs array that enables a user to add as +many Unicode blocks as they want. The insert_code() function picks a block +from the array at random, and then picks a character from that block at random:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#define UNICODE(min, max) (((uint64_t)max << 32) | min) @@ -104,51 +124,32 @@ static inline void insert_code(matrix *mat, } </code></pre></div></div> -<p>Entries in the <code class="language-plaintext highlighter-rouge">glyphs</code> array are Unicode blocks bit-packed in an 8-byte -container: the four low bytes forms the first codepoint and the four high bytes -the last.</p> - -<h2 id="phosphor-decay">Phosphor decay</h2> +<p>The Unicode blocks are stored in 8-byte containers: the low four bytes form the +first codepoint and the high four bytes the last. Here, I chose bitwise +operations over unions because, first and foremost, the operations themselves +are trivial and idiomatic, and the UNICODE() macro simplifies the management of +charsets. The insert_code() function is now ready to take its rightful place +next to blend().</p> -<p>The dim afterglow of monochrome CRT displays is achieved by carefully scaling -the RGB channels individually and mixing them:</p> - -<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#define DECAY_MPLIER 2 - -static inline void blend(matrix *mat, - size_t row, size_t col) -{ - unsigned char *color; - - color = mat->rgb[index(mat, row, col)].color; - color[R] = color[R] - (color[R] - RGB_BG_RED) / DECAY_MPLIER; - color[G] = color[G] - (color[G] - RGB_BG_GRN) / DECAY_MPLIER; - color[B] = color[B] - (color[B] - RGB_BG_BLU) / DECAY_MPLIER; -} -</code></pre></div></div> - -<p>The blending function emulates the phosphor decay by gradually transitioning -each raindrop’s color towards the background color. The multiplier is the -number of passes over the rain track needed before the afterglow disappears.</p> - -<h2 id="the-algorithm">The algorithm</h2> - -<p>Nonetheless, the rain resembles the original with high visual fidelity. It’s -highly customizable and gentle on the CPU. On my 14” ThinkPad T490, which has a -resolution of 1920x1080 and 4GHz CPU, it uses 2-3% of the CPU with occasional -jumps of up to about 8%. Not too bad for a weekend project. The program has -been tested with xterm and urxvt terminal emulators on OpenBSD and Arch Linux -systems. Someone has managed to get it moving on a Raspberry Pi as well.</p> - -<p>Lastly, to compile and run:</p> +<p>The result is a digital rain that captures the original Matrix aesthetic with +high visual fidelity:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ cc -O3 main.c -o matrix $ ./matrix </code></pre></div></div> -<p>“All I see is blonde, brunette, red head.”</p> +<video style="max-width:100%;" controls="" poster="poster.png"> + <source src="matrix.mp4" type="video/mp4" /> +</video> + +<p>There was no cause to measure the program’s performance characteristics +precisely; it’s gentle on the CPU. On my ThinkPad T490 running OpenBSD, which +has a resolution of 1920x1080, it uses about 2-3% of the CPU, with occasional +jumps of up to about 8%; the cores remain silent, the fans don’t whir, the rain +falls in quiet.</p> <p>Files: <a href="source.tar.gz">source.tar.gz</a></p> + </div> <p class="post-author right">by W. D. Sadeep Madurange</p> </div> diff --git a/_site/log/neo4j-a-star-search/index.html b/_site/log/neo4j-a-star-search/index.html new file mode 100644 index 0000000..9473dc6 --- /dev/null +++ b/_site/log/neo4j-a-star-search/index.html @@ -0,0 +1,370 @@ +<!DOCTYPE html> +<html> + <head> + <meta charset="utf-8"> + <title>Neo4J A* search</title> + + <head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Neo4J A* search</title> + <link rel="stylesheet" href="/assets/css/main.css"> + <link rel="stylesheet" href="/assets/css/skeleton.css"> +</head> + + + + </head> + <body> + + <div id="nav-container" class="container"> + <ul id="navlist" class="left"> + + <li > + <a href="/" class="link-decor-none">hme</a> + </li> + <li class="active"> + <a href="/log/" class="link-decor-none">log</a> + </li> + <li > + <a href="/projects/" class="link-decor-none">poc</a> + </li> + <li > + <a href="/about/" class="link-decor-none">abt</a> + </li> + <li><a href="/feed.xml" class="link-decor-none">rss</a></li> + </ul> +</div> + + + + <main> + <div class="container"> + <div class="container-2"> + <h2 class="center" id="title">NEO4J A* SEARCH</h2> + <h6 class="center">06 MARCH 2018</h5> + <br> + <div class="twocol justify"><p>Back in 2018, we used <a href="https://neo4j.com/" class="external" target="_blank" rel="noopener noreferrer">Neo4J</a> graph database to track the +movement of marine vessels. We were interested in the shortest path a ship +could take through a network of about 13,000 route points. Graph theoretic +algorithms provide optimal solutions to such problems, and the set of route +points lends itself well to graph-based modelling.</p> + +<p>A graph is a finite set of vertices, and a subset of vertex pairs (edges). +Edges can have weights. In the case of vessel tracking, the route points form +the vertices of a graph; the routes between them the edges; and the distances +between them the weights. For various reasons, people are interested in +minimizing (or maximizing) the weight of a path through a set of vertices, such +as the shortest path between two ports to predict a vessel’s arrival time.</p> + +<p>Given a graph, an algorithm like Dijkstra’s search could compute the shortest +path between two vertices. In fact, this was the algorithm Neo4J shipped with +at the time. One drawback of Dijkstra’s algorithm is that it computes all the +shortest paths from the source to all other vertices before terminating at the +destination vertex. The time complexity of this exhaustive search prevented our +database from scaling beyond 4,000 route points.</p> + +<p>The following enhancement to Dijkstra’s search, also known as the A* search, +employs a heuristic to steer the search in the direction of the destination +more quickly. In the case of our network of vessels, which are on the earth’s +surface, spherical distance is a good candidate for a heuristic:</p> + +<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>package org.neo4j.graphalgo.impl; + +import java.util.stream.Stream; +import java.util.stream.StreamSupport; + +import org.neo4j.graphalgo.api.Graph; +import org.neo4j.graphalgo.core.utils.ProgressLogger; +import org.neo4j.graphalgo.core.utils.queue.IntPriorityQueue; +import org.neo4j.graphalgo.core.utils.queue.SharedIntPriorityQueue; +import org.neo4j.graphalgo.core.utils.traverse.SimpleBitSet; +import org.neo4j.graphdb.Direction; +import org.neo4j.graphdb.Node; +import org.neo4j.kernel.internal.GraphDatabaseAPI; + +import com.carrotsearch.hppc.IntArrayDeque; +import com.carrotsearch.hppc.IntDoubleMap; +import com.carrotsearch.hppc.IntDoubleScatterMap; +import com.carrotsearch.hppc.IntIntMap; +import com.carrotsearch.hppc.IntIntScatterMap; + +public class ShortestPathAStar extends Algorithm<ShortestPathAStar> { + + private final GraphDatabaseAPI dbService; + private static final int PATH_END = -1; + + private Graph graph; + private final int nodeCount; + private IntDoubleMap gCosts; + private IntDoubleMap fCosts; + private double totalCost; + private IntPriorityQueue openNodes; + private IntIntMap path; + private IntArrayDeque shortestPath; + private SimpleBitSet closedNodes; + private final ProgressLogger progressLogger; + + public static final double NO_PATH_FOUND = -1.0; + + public ShortestPathAStar( + final Graph graph, + final GraphDatabaseAPI dbService) { + + this.graph = graph; + this.dbService = dbService; + + nodeCount = Math.toIntExact(graph.nodeCount()); + gCosts = new IntDoubleScatterMap(nodeCount); + fCosts = new IntDoubleScatterMap(nodeCount); + openNodes = SharedIntPriorityQueue.min( + nodeCount, + fCosts, + Double.MAX_VALUE); + path = new IntIntScatterMap(nodeCount); + closedNodes = new SimpleBitSet(nodeCount); + shortestPath = new IntArrayDeque(); + progressLogger = getProgressLogger(); + } + + public ShortestPathAStar compute( + final long startNode, + final long goalNode, + final String propertyKeyLat, + final String propertyKeyLon, + final Direction direction) { + + reset(); + + final int startNodeInternal = + graph.toMappedNodeId(startNode); + final double startNodeLat = + getNodeCoordinate(startNodeInternal, propertyKeyLat); + final double startNodeLon = + getNodeCoordinate(startNodeInternal, propertyKeyLon); + + final int goalNodeInternal = + graph.toMappedNodeId(goalNode); + final double goalNodeLat = + getNodeCoordinate(goalNodeInternal, propertyKeyLat); + final double goalNodeLon = + getNodeCoordinate(goalNodeInternal, propertyKeyLon); + + final double initialHeuristic = + computeHeuristic(startNodeLat, + startNodeLon, + goalNodeLat, + goalNodeLon); + + gCosts.put(startNodeInternal, 0.0); + fCosts.put(startNodeInternal, initialHeuristic); + openNodes.add(startNodeInternal, 0.0); + + run(goalNodeInternal, + propertyKeyLat, + propertyKeyLon, + direction); + + if (path.containsKey(goalNodeInternal)) { + totalCost = gCosts.get(goalNodeInternal); + int node = goalNodeInternal; + while (node != PATH_END) { + shortestPath.addFirst(node); + node = path.getOrDefault(node, PATH_END); + } + } + return this; + } + + private void run( + final int goalNodeId, + final String propertyKeyLat, + final String propertyKeyLon, + final Direction direction) { + + final double goalLat = + getNodeCoordinate(goalNodeId, propertyKeyLat); + final double goalLon = + getNodeCoordinate(goalNodeId, propertyKeyLon); + + while (!openNodes.isEmpty() && running()) { + int currentNodeId = openNodes.pop(); + if (currentNodeId == goalNodeId) { + return; + } + + closedNodes.put(currentNodeId); + + double currentNodeCost = + this.gCosts.getOrDefault( + currentNodeId, + Double.MAX_VALUE); + + graph.forEachRelationship( + currentNodeId, + direction, + (source, target, relationshipId, weight) -> { + double neighbourLat = + getNodeCoordinate(target, propertyKeyLat); + double neighbourLon = + getNodeCoordinate(target, propertyKeyLon); + double heuristic = + computeHeuristic( + neighbourLat, + neighbourLon, + goalLat, + goalLon); + + updateCosts( + source, + target, + weight + currentNodeCost, + heuristic); + + if (!closedNodes.contains(target)) { + openNodes.add(target, 0); + } + return true; + }); + + progressLogger.logProgress( + (double) currentNodeId / (nodeCount - 1)); + } + } + + private double computeHeuristic( + final double lat1, + final double lon1, + final double lat2, + final double lon2) { + + final int earthRadius = 6371; + final double kmToNM = 0.539957; + final double latDistance = Math.toRadians(lat2 - lat1); + final double lonDistance = Math.toRadians(lon2 - lon1); + final double a = Math.sin(latDistance / 2) + * Math.sin(latDistance / 2) + + Math.cos(Math.toRadians(lat1)) + * Math.cos(Math.toRadians(lat2)) + * Math.sin(lonDistance / 2) + * Math.sin(lonDistance / 2); + final double c = 2 + * Math.atan2(Math.sqrt(a), Math.sqrt(1 - a)); + final double distance = earthRadius * c * kmToNM; + return distance; + } + + private double getNodeCoordinate( + final int nodeId, + final String coordinateType) { + + final long neo4jId = graph.toOriginalNodeId(nodeId); + final Node node = dbService.getNodeById(neo4jId); + return (double) node.getProperty(coordinateType); + } + + private void updateCosts( + final int source, + final int target, + final double newCost, + final double heuristic) { + + final double oldCost = + gCosts.getOrDefault(target, Double.MAX_VALUE); + + if (newCost < oldCost) { + gCosts.put(target, newCost); + fCosts.put(target, newCost + heuristic); + path.put(target, source); + } + } + + private void reset() { + closedNodes.clear(); + openNodes.clear(); + gCosts.clear(); + fCosts.clear(); + path.clear(); + shortestPath.clear(); + totalCost = NO_PATH_FOUND; + } + + public Stream<Result> resultStream() { + return StreamSupport.stream( + shortestPath.spliterator(), false) + .map(cursor -> new Result( + graph.toOriginalNodeId(cursor.value), + gCosts.get(cursor.value))); + } + + public IntArrayDeque getFinalPath() { + return shortestPath; + } + + public double getTotalCost() { + return totalCost; + } + + public int getPathLength() { + return shortestPath.size(); + } + + @Override + public ShortestPathAStar me() { + return this; + } + + @Override + public ShortestPathAStar release() { + graph = null; + gCosts = null; + fCosts = null; + openNodes = null; + path = null; + shortestPath = null; + closedNodes = null; + return this; + } + + public static class Result { + + /** + * the neo4j node id + */ + public final Long nodeId; + + /** + * cost to reach the node from startNode + */ + public final Double cost; + + public Result(Long nodeId, Double cost) { + this.nodeId = nodeId; + this.cost = cost; + } + } +} +</code></pre></div></div> + +<p>The heuristic function is domain-specific. If chosen wisely, it can +significantly speed up the search. In our case, we achieved a 300x speedup, +enabling us to expand our search from 4,000 to 13,000 route points. The <a href="https://github.com/neo4j-contrib/neo4j-graph-algorithms/releases/tag/3.4.0.0" class="external" target="_blank" rel="noopener noreferrer">v3.4.0</a> of the +Neo4J graph algorithms shipped with our A* search algorithm.</p> + +</div> + <p class="post-author right">by W. D. Sadeep Madurange</p> + </div> + </div> + </main> + + <div class="footer"> + <div class="container"> + <div class="twelve columns right container-2"> + <p id="footer-text">© ASCIIMX - 2025</p> + </div> + </div> +</div> + + + </body> +</html> diff --git a/_site/posts.xml b/_site/posts.xml index 1d3402d..37970da 100644 --- a/_site/posts.xml +++ b/_site/posts.xml @@ -1 +1 @@ -<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="http://localhost:4000/posts.xml" rel="self" type="application/atom+xml" /><link href="http://localhost:4000/" rel="alternate" type="text/html" /><updated>2025-12-22T23:37:38+08:00</updated><id>http://localhost:4000/posts.xml</id><title type="html">ASCIIMX</title><author><name>W. D. Sadeep Madurange</name></author></feed>
\ No newline at end of file +<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="/posts.xml" rel="self" type="application/atom+xml" /><link href="/" rel="alternate" type="text/html" /><updated>2025-12-24T16:28:25+08:00</updated><id>/posts.xml</id><title type="html">ASCIIMX</title><author><name>W. D. Sadeep Madurange</name></author></feed>
\ No newline at end of file diff --git a/_site/robots.txt b/_site/robots.txt index d297064..e087884 100644 --- a/_site/robots.txt +++ b/_site/robots.txt @@ -1 +1 @@ -Sitemap: http://localhost:4000/sitemap.xml +Sitemap: /sitemap.xml diff --git a/_site/sitemap.xml b/_site/sitemap.xml index 7a775fd..e8e9d7b 100644 --- a/_site/sitemap.xml +++ b/_site/sitemap.xml @@ -1,55 +1,59 @@ <?xml version="1.0" encoding="UTF-8"?> <urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <url> -<loc>http://localhost:4000/log/e-reader/</loc> +<loc>/log/neo4j-a-star-search/</loc> +<lastmod>2018-03-06T00:00:00+08:00</lastmod> +</url> +<url> +<loc>/log/e-reader/</loc> <lastmod>2023-10-24T00:00:00+08:00</lastmod> </url> <url> -<loc>http://localhost:4000/log/etlas/</loc> +<loc>/log/etlas/</loc> <lastmod>2024-09-05T00:00:00+08:00</lastmod> </url> <url> -<loc>http://localhost:4000/log/arduino-due/</loc> +<loc>/log/arduino-due/</loc> <lastmod>2024-09-16T00:00:00+08:00</lastmod> </url> <url> -<loc>http://localhost:4000/log/bumblebee/</loc> +<loc>/log/bumblebee/</loc> <lastmod>2025-04-02T00:00:00+08:00</lastmod> </url> <url> -<loc>http://localhost:4000/log/my-first-pcb/</loc> +<loc>/log/my-first-pcb/</loc> <lastmod>2025-04-26T00:00:00+08:00</lastmod> </url> <url> -<loc>http://localhost:4000/log/arduino-uno/</loc> +<loc>/log/arduino-uno/</loc> <lastmod>2025-06-10T00:00:00+08:00</lastmod> </url> <url> -<loc>http://localhost:4000/log/mosfet-switches/</loc> +<loc>/log/mosfet-switches/</loc> <lastmod>2025-06-22T00:00:00+08:00</lastmod> </url> <url> -<loc>http://localhost:4000/log/fpm-door-lock/</loc> +<loc>/log/fpm-door-lock/</loc> <lastmod>2025-08-18T00:00:00+08:00</lastmod> </url> <url> -<loc>http://localhost:4000/log/suckless-software/</loc> +<loc>/log/suckless-software/</loc> <lastmod>2025-11-30T00:00:00+08:00</lastmod> </url> <url> -<loc>http://localhost:4000/log/matrix-digital-rain/</loc> +<loc>/log/matrix-digital-rain/</loc> <lastmod>2025-12-21T00:00:00+08:00</lastmod> </url> <url> -<loc>http://localhost:4000/about/</loc> +<loc>/about/</loc> </url> <url> -<loc>http://localhost:4000/</loc> +<loc>/</loc> </url> <url> -<loc>http://localhost:4000/log/</loc> +<loc>/log/</loc> </url> <url> -<loc>http://localhost:4000/projects/</loc> +<loc>/projects/</loc> </url> </urlset> |
