diff options
| -rw-r--r-- | README.txt | 9 | ||||
| -rw-r--r-- | _log/search-with-cgi.md | 98 | ||||
| -rw-r--r-- | _site/feed.xml | 2 | ||||
| -rw-r--r-- | _site/log/search-with-cgi/index.html | 103 | ||||
| -rw-r--r-- | _site/posts.xml | 2 |
5 files changed, 89 insertions, 125 deletions
@@ -2,9 +2,14 @@ bundle exec jekyll serve JEKYLL_ENV=production bundle exec jekyll build -CGI scripts: +Permissions -# chown -R www:www cgi-bin/ +$ cd /var/www/htdocs +# chown -R www:www www.asciimx.com +# chmod -R 444 www.asciimx.com/ +# chmod u+x,g+x www.asciimx.com/ +# cd www.asciimx.com/ +# find . -type d -exec chmod 554 {} + # chmod 554 cgi-bin/find.cgi Checking CGI script errors in chroot: diff --git a/_log/search-with-cgi.md b/_log/search-with-cgi.md index 2578878..0109294 100644 --- a/_log/search-with-cgi.md +++ b/_log/search-with-cgi.md @@ -4,47 +4,42 @@ date: 2025-12-29 layout: post --- -Need a way to search site--number of articles are growing. +Number of articles on the site are growing. Need a way to search site. -Searching site client-side using the RSS feed and JavaScript is not an option-- -bloats the feed and breaks the site for Lynx and other text browsers. +Searching the RSS feed client-side using JavaScript is not an option. That +would make the feed much heavier and break the site for text-based web browsers +like Lynx. -Perl's great for text processing--especially regex work. Few lines of Perl -could do a regex search and send the result back via CGI. OpenBSD httpd speaks -CGI, Perl and slowcgi are in the base systems. No dependencies. Works on every -conceivable browser. +Not gonna use an inverted index--More than an evening's effort, especially if I +want partial matching. I want partial matching. -Perl: traverse the directory with File::Find recursively. If search text is -found grab the file name, title and up to 50 chars of the first paragraph to -include in the search result. +Few lines of Perl could do a regex search and send the result back via CGI. +OpenBSD httpd speaks CGI. Perl and slowcgi are in the base system. No +dependencies. + +Perl: traverse directory with File::Find. If search text is found grab the file +name, title and up to 50 chars from the first paragraph to include in the +search result. ``` -find(sub { - if (open my $fh, '<', $_) { - my $content = do { local $/; <$fh> }; - close $fh; - - if ($content =~ /\Q$search_text\E/i) { - my ($title) = $content =~ /<title>(.*?)<\/title>/is; - $title ||= $File::Find::name; - my ($p_content) = $content =~ /<p[^>]*>(.*?)<\/p>/is; - my $snippet = $p_content || ""; - $snippet =~ s/<[^>]*>//g; - $snippet =~ s/\s+/ /g; - $snippet = substr($snippet, 0, 50); - $snippet .= "..." if length($p_content || "") > 50; - - push @results, { - path => $File::Find::name, - title => $title, - snippet => $snippet - }; - } - } +find({ + wanted => sub { + return unless -f $_ && $_ eq 'index.html'; + # ... file reading ... + if ($content =~ /\Q$search_text\E/i) { + # Extract title, snippet + push @results, { + path => $File::Find::name, + title => $title, + snippet => $snippet + }; + } + }, + follow => 0, }, $dir); ``` -Don't need the Perl CGI module, httpd sets QUERY_STRING for the slowcgi script: +httpd sets the search text in QUERY_STRING env. Don't need Perl's CGI module. ``` my %params; @@ -58,38 +53,21 @@ if ($ENV{QUERY_STRING}) { } ``` -Run the script as www user. Permissions: 554 (read + execute). +Security. -Running in OpenBSD chroot: Check Perl's dynamic object dependencies: +ReDOS, XSS, command injection, symlink attacks. Did I miss anything? Probably. -``` -$ ldd $(which perl) -/usr/bin/perl: - Start End Type Open Ref GrpRef Name - 000008797e8e6000 000008797e8eb000 exe 1 0 0 /usr/bin/perl - 0000087c1ffe5000 0000087c20396000 rlib 0 1 0 /usr/lib/libperl.so.26.0 - 0000087bf4508000 0000087bf4539000 rlib 0 2 0 /usr/lib/libm.so.10.1 - 0000087b9e801000 0000087b9e907000 rlib 0 2 0 /usr/lib/libc.so.102.0 - 0000087bba182000 0000087bba182000 ld.so 0 1 0 /usr/libexec/ld.so -``` +ReDOS: sanitized user input, length-limit search text, quote metacharacters +with `\Q$search_text\E`. -Copy them over to chroot. Now should have /var/www/usr/bin/perl, -/usr/lib/libperl.so.26.0, and so on. +XSS: sanitized user input. Escaped HTML. -Troubleshooting: look for issues in logs or try executing the script in chroot: +Command injection: no exec()/system() calls. Non-privileged user (www). -``` -$ cat /var/log/messages | grep slowcgi -# chroot /var/www/ htdocs/path/to/script/script.cgi -``` -The last command exposes any missing Perl modules in chroot and where to find -them. Copy them over as well. +Symlink attacks: File::Find don't follow symlinks (follow => 0). chroot. -``` -location "/cgi-bin/*" { - fastcgi socket "/run/slowcgi.sock" -} -``` +Access controls: files (444), directories and CGI script: 554. -in httpd.conf routes queries to slowcgi. +Verdict: O(n) speed. Works on every conceivable browser. Good enough. +Commit: [9fec793](https://git.asciimx.com/www/commit/?h=term&id=9fec793abe0a73e5cd502a1d1e935e2413b85079) diff --git a/_site/feed.xml b/_site/feed.xml index bc8a117..6b0e884 100644 --- a/_site/feed.xml +++ b/_site/feed.xml @@ -1 +1 @@ -<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="/feed.xml" rel="self" type="application/atom+xml" /><link href="/" rel="alternate" type="text/html" /><updated>2025-12-29T21:56:36+08:00</updated><id>/feed.xml</id><title type="html">ASCIIMX | Log</title><author><name>W. D. Sadeep Madurange</name></author><entry><title type="html">Site search using Perl + CGI</title><link href="/log/search-with-cgi/" rel="alternate" type="text/html" title="Site search using Perl + CGI" /><published>2025-12-29T00:00:00+08:00</published><updated>2025-12-29T00:00:00+08:00</updated><id>/log/search-with-cgi</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Need a way to search site–number of articles are growing.]]></summary></entry><entry><title type="html">Matrix Rain: 2025 refactor</title><link href="/log/matrix-digital-rain/" rel="alternate" type="text/html" title="Matrix Rain: 2025 refactor" /><published>2025-12-21T00:00:00+08:00</published><updated>2025-12-21T00:00:00+08:00</updated><id>/log/matrix-digital-rain</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[The 2022 version worked but had some loose ends. Unicode support was inflexible–couldn’t mix ASCII with Katakana; Phosphor decay was stored in a separate array when it should’ve been packed with RGB; Code was harder to read than it needed to be.]]></summary></entry><entry><title type="html">Fingerprint door lock (LP)</title><link href="/log/fpm-door-lock-lp/" rel="alternate" type="text/html" title="Fingerprint door lock (LP)" /><published>2025-08-18T00:00:00+08:00</published><updated>2025-08-18T00:00:00+08:00</updated><id>/log/fpm-door-lock-lp</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Second iteration of the RF door lock. Old version worked but drew too much quiescent current. Sensor and servo pulled 13.8mA and 4.6mA idle. Linear regulators were a disaster. Battery didn’t last 24 hours.]]></summary></entry><entry><title type="html">High-side MOSFET switching</title><link href="/log/mosfet-switches/" rel="alternate" type="text/html" title="High-side MOSFET switching" /><published>2025-06-22T00:00:00+08:00</published><updated>2025-06-22T00:00:00+08:00</updated><id>/log/mosfet-switches</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Needed low-power switching for the fingerprint door lock. Servo and FPM draw high quiescent current–had to cut power electronically during sleep. MOSFETs can do this.]]></summary></entry><entry><title type="html">ATmega328P at 3.3V and 5V</title><link href="/log/arduino-uno/" rel="alternate" type="text/html" title="ATmega328P at 3.3V and 5V" /><published>2025-06-10T00:00:00+08:00</published><updated>2025-06-10T00:00:00+08:00</updated><id>/log/arduino-uno</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Quick reference for wiring ATmega328P ICs at 5V and 3.3V. 5V uses 16MHz crystal, 3.3V uses 8MHz.]]></summary></entry><entry><title type="html">Fingerprint door lock (RF)</title><link href="/log/fpm-door-lock-rf/" rel="alternate" type="text/html" title="Fingerprint door lock (RF)" /><published>2025-06-05T00:00:00+08:00</published><updated>2025-06-05T00:00:00+08:00</updated><id>/log/fpm-door-lock-rf</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Wanted to unlock door with fingerprint, wirelessly to avoid drilling.]]></summary></entry><entry><title type="html">Bumblebee: browser automation</title><link href="/log/bumblebee/" rel="alternate" type="text/html" title="Bumblebee: browser automation" /><published>2025-04-02T00:00:00+08:00</published><updated>2025-04-02T00:00:00+08:00</updated><id>/log/bumblebee</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Built with Andy Zhang for an employer. Tool to automate web scraping script generation.]]></summary></entry><entry><title type="html">ATSAM3X8E bare-metal programming</title><link href="/log/arduino-due/" rel="alternate" type="text/html" title="ATSAM3X8E bare-metal programming" /><published>2024-09-16T00:00:00+08:00</published><updated>2024-09-16T00:00:00+08:00</updated><id>/log/arduino-due</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Notes on programming ATSAM3X8E chips (Arduino Due) without bootloader. Tested on OpenBSD.]]></summary></entry><entry><title type="html">Etlas: e-paper dashboard</title><link href="/log/etlas/" rel="alternate" type="text/html" title="Etlas: e-paper dashboard" /><published>2024-09-05T00:00:00+08:00</published><updated>2024-09-05T00:00:00+08:00</updated><id>/log/etlas</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Repurposed e-reader prototype into something for regular use. News, stocks, weather dashboard. ESP32 NodeMCU D1 + 7.5” Waveshare e-paper + DHT22 sensor.]]></summary></entry><entry><title type="html">ESP32 e-reader prototype</title><link href="/log/e-reader/" rel="alternate" type="text/html" title="ESP32 e-reader prototype" /><published>2023-10-24T00:00:00+08:00</published><updated>2023-10-24T00:00:00+08:00</updated><id>/log/e-reader</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[First project with e-paper displays and ESP32.]]></summary></entry></feed>
\ No newline at end of file +<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="/feed.xml" rel="self" type="application/atom+xml" /><link href="/" rel="alternate" type="text/html" /><updated>2025-12-30T22:31:10+08:00</updated><id>/feed.xml</id><title type="html">ASCIIMX | Log</title><author><name>W. D. Sadeep Madurange</name></author><entry><title type="html">Site search using Perl + CGI</title><link href="/log/search-with-cgi/" rel="alternate" type="text/html" title="Site search using Perl + CGI" /><published>2025-12-29T00:00:00+08:00</published><updated>2025-12-29T00:00:00+08:00</updated><id>/log/search-with-cgi</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Number of articles on the site are growing. Need a way to search site.]]></summary></entry><entry><title type="html">Matrix Rain: 2025 refactor</title><link href="/log/matrix-digital-rain/" rel="alternate" type="text/html" title="Matrix Rain: 2025 refactor" /><published>2025-12-21T00:00:00+08:00</published><updated>2025-12-21T00:00:00+08:00</updated><id>/log/matrix-digital-rain</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[The 2022 version worked but had some loose ends. Unicode support was inflexible–couldn’t mix ASCII with Katakana; Phosphor decay was stored in a separate array when it should’ve been packed with RGB; Code was harder to read than it needed to be.]]></summary></entry><entry><title type="html">Fingerprint door lock (LP)</title><link href="/log/fpm-door-lock-lp/" rel="alternate" type="text/html" title="Fingerprint door lock (LP)" /><published>2025-08-18T00:00:00+08:00</published><updated>2025-08-18T00:00:00+08:00</updated><id>/log/fpm-door-lock-lp</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Second iteration of the RF door lock. Old version worked but drew too much quiescent current. Sensor and servo pulled 13.8mA and 4.6mA idle. Linear regulators were a disaster. Battery didn’t last 24 hours.]]></summary></entry><entry><title type="html">High-side MOSFET switching</title><link href="/log/mosfet-switches/" rel="alternate" type="text/html" title="High-side MOSFET switching" /><published>2025-06-22T00:00:00+08:00</published><updated>2025-06-22T00:00:00+08:00</updated><id>/log/mosfet-switches</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Needed low-power switching for the fingerprint door lock. Servo and FPM draw high quiescent current–had to cut power electronically during sleep. MOSFETs can do this.]]></summary></entry><entry><title type="html">ATmega328P at 3.3V and 5V</title><link href="/log/arduino-uno/" rel="alternate" type="text/html" title="ATmega328P at 3.3V and 5V" /><published>2025-06-10T00:00:00+08:00</published><updated>2025-06-10T00:00:00+08:00</updated><id>/log/arduino-uno</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Quick reference for wiring ATmega328P ICs at 5V and 3.3V. 5V uses 16MHz crystal, 3.3V uses 8MHz.]]></summary></entry><entry><title type="html">Fingerprint door lock (RF)</title><link href="/log/fpm-door-lock-rf/" rel="alternate" type="text/html" title="Fingerprint door lock (RF)" /><published>2025-06-05T00:00:00+08:00</published><updated>2025-06-05T00:00:00+08:00</updated><id>/log/fpm-door-lock-rf</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Wanted to unlock door with fingerprint, wirelessly to avoid drilling.]]></summary></entry><entry><title type="html">Bumblebee: browser automation</title><link href="/log/bumblebee/" rel="alternate" type="text/html" title="Bumblebee: browser automation" /><published>2025-04-02T00:00:00+08:00</published><updated>2025-04-02T00:00:00+08:00</updated><id>/log/bumblebee</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Built with Andy Zhang for an employer. Tool to automate web scraping script generation.]]></summary></entry><entry><title type="html">ATSAM3X8E bare-metal programming</title><link href="/log/arduino-due/" rel="alternate" type="text/html" title="ATSAM3X8E bare-metal programming" /><published>2024-09-16T00:00:00+08:00</published><updated>2024-09-16T00:00:00+08:00</updated><id>/log/arduino-due</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Notes on programming ATSAM3X8E chips (Arduino Due) without bootloader. Tested on OpenBSD.]]></summary></entry><entry><title type="html">Etlas: e-paper dashboard</title><link href="/log/etlas/" rel="alternate" type="text/html" title="Etlas: e-paper dashboard" /><published>2024-09-05T00:00:00+08:00</published><updated>2024-09-05T00:00:00+08:00</updated><id>/log/etlas</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[Repurposed e-reader prototype into something for regular use. News, stocks, weather dashboard. ESP32 NodeMCU D1 + 7.5” Waveshare e-paper + DHT22 sensor.]]></summary></entry><entry><title type="html">ESP32 e-reader prototype</title><link href="/log/e-reader/" rel="alternate" type="text/html" title="ESP32 e-reader prototype" /><published>2023-10-24T00:00:00+08:00</published><updated>2023-10-24T00:00:00+08:00</updated><id>/log/e-reader</id><author><name>W. D. Sadeep Madurange</name></author><summary type="html"><![CDATA[First project with e-paper displays and ESP32.]]></summary></entry></feed>
\ No newline at end of file diff --git a/_site/log/search-with-cgi/index.html b/_site/log/search-with-cgi/index.html index 060a282..71b9f23 100644 --- a/_site/log/search-with-cgi/index.html +++ b/_site/log/search-with-cgi/index.html @@ -49,46 +49,41 @@ <h2 class="center" id="title">SITE SEARCH USING PERL + CGI</h2> <h6 class="center">29 DECEMBER 2025</h5> <br> - <div class="twocol justify"><p>Need a way to search site–number of articles are growing.</p> - -<p>Searching site client-side using the RSS feed and JavaScript is not an option– -bloats the feed and breaks the site for Lynx and other text browsers.</p> - -<p>Perl’s great for text processing–especially regex work. Few lines of Perl -could do a regex search and send the result back via CGI. OpenBSD httpd speaks -CGI, Perl and slowcgi are in the base systems. No dependencies. Works on every -conceivable browser.</p> - -<p>Perl: traverse the directory with File::Find recursively. If search text is -found grab the file name, title and up to 50 chars of the first paragraph to -include in the search result.</p> - -<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>find(sub { - if (open my $fh, '<', $_) { - my $content = do { local $/; <$fh> }; - close $fh; - - if ($content =~ /\Q$search_text\E/i) { - my ($title) = $content =~ /<title>(.*?)<\/title>/is; - $title ||= $File::Find::name; - my ($p_content) = $content =~ /<p[^>]*>(.*?)<\/p>/is; - my $snippet = $p_content || ""; - $snippet =~ s/<[^>]*>//g; - $snippet =~ s/\s+/ /g; - $snippet = substr($snippet, 0, 50); - $snippet .= "..." if length($p_content || "") > 50; - - push @results, { - path => $File::Find::name, - title => $title, - snippet => $snippet - }; - } - } + <div class="twocol justify"><p>Number of articles on the site are growing. Need a way to search site.</p> + +<p>Searching the RSS feed client-side using JavaScript is not an option. That +would make the feed much heavier and break the site for text-based web browsers +like Lynx.</p> + +<p>Not gonna use an inverted index–More than an evening’s effort, especially if I +want partial matching. I want partial matching.</p> + +<p>Few lines of Perl could do a regex search and send the result back via CGI. +OpenBSD httpd speaks CGI. Perl and slowcgi are in the base system. No +dependencies.</p> + +<p>Perl: traverse directory with File::Find. If search text is found grab the file +name, title and up to 50 chars from the first paragraph to include in the +search result.</p> + +<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>find({ + wanted => sub { + return unless -f $_ && $_ eq 'index.html'; + # ... file reading ... + if ($content =~ /\Q$search_text\E/i) { + # Extract title, snippet + push @results, { + path => $File::Find::name, + title => $title, + snippet => $snippet + }; + } + }, + follow => 0, }, $dir); </code></pre></div></div> -<p>Don’t need the Perl CGI module, httpd sets QUERY_STRING for the slowcgi script:</p> +<p>httpd sets the search text in QUERY_STRING env. Don’t need Perl’s CGI module.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my %params; if ($ENV{QUERY_STRING}) { @@ -101,38 +96,24 @@ if ($ENV{QUERY_STRING}) { } </code></pre></div></div> -<p>Run the script as www user. Permissions: 554 (read + execute).</p> +<p>Security.</p> -<p>Running in OpenBSD chroot: Check Perl’s dynamic object dependencies:</p> +<p>ReDOS, XSS, command injection, symlink attacks. Did I miss anything? Probably.</p> -<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ ldd $(which perl) -/usr/bin/perl: - Start End Type Open Ref GrpRef Name - 000008797e8e6000 000008797e8eb000 exe 1 0 0 /usr/bin/perl - 0000087c1ffe5000 0000087c20396000 rlib 0 1 0 /usr/lib/libperl.so.26.0 - 0000087bf4508000 0000087bf4539000 rlib 0 2 0 /usr/lib/libm.so.10.1 - 0000087b9e801000 0000087b9e907000 rlib 0 2 0 /usr/lib/libc.so.102.0 - 0000087bba182000 0000087bba182000 ld.so 0 1 0 /usr/libexec/ld.so -</code></pre></div></div> +<p>ReDOS: sanitized user input, length-limit search text, quote metacharacters +with <code class="language-plaintext highlighter-rouge">\Q$search_text\E</code>.</p> -<p>Copy them over to chroot. Now should have /var/www/usr/bin/perl, -/usr/lib/libperl.so.26.0, and so on.</p> +<p>XSS: sanitized user input. Escaped HTML.</p> -<p>Troubleshooting: look for issues in logs or try executing the script in chroot:</p> +<p>Command injection: no exec()/system() calls. Non-privileged user (www).</p> -<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ cat /var/log/messages | grep slowcgi -# chroot /var/www/ htdocs/path/to/script/script.cgi -</code></pre></div></div> -<p>The last command exposes any missing Perl modules in chroot and where to find -them. Copy them over as well.</p> +<p>Symlink attacks: File::Find don’t follow symlinks (follow => 0). chroot.</p> -<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>location "/cgi-bin/*" { - fastcgi socket "/run/slowcgi.sock" -} -</code></pre></div></div> +<p>Access controls: files (444), directories and CGI script: 554.</p> -<p>in httpd.conf routes queries to slowcgi.</p> +<p>Verdict: O(n) speed. Works on every conceivable browser. Good enough.</p> +<p>Commit: <a href="https://git.asciimx.com/www/commit/?h=term&id=9fec793abe0a73e5cd502a1d1e935e2413b85079">9fec793</a></p> </div> <p class="post-author right">by W. D. Sadeep Madurange</p> </div> diff --git a/_site/posts.xml b/_site/posts.xml index 66974e6..6f08cc2 100644 --- a/_site/posts.xml +++ b/_site/posts.xml @@ -1 +1 @@ -<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="/posts.xml" rel="self" type="application/atom+xml" /><link href="/" rel="alternate" type="text/html" /><updated>2025-12-29T21:56:36+08:00</updated><id>/posts.xml</id><title type="html">ASCIIMX</title><author><name>W. D. Sadeep Madurange</name></author></feed>
\ No newline at end of file +<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="/posts.xml" rel="self" type="application/atom+xml" /><link href="/" rel="alternate" type="text/html" /><updated>2025-12-30T22:31:10+08:00</updated><id>/posts.xml</id><title type="html">ASCIIMX</title><author><name>W. D. Sadeep Madurange</name></author></feed>
\ No newline at end of file |