Archive for the ‘WoT’ Category

Lekythion search update: additional blogs and other things

Sunday, May 31st, 2020

I’ve added several new blogs to lekythion’s search feature, including:

trinque.org
loper-os.org
thimbron.com
ossasepia.com
fixpoint.welshcomputing.com
billymg.com
ztkfg.com
qntra.net

I intend to add more as I have time. I am also open to suggestions for additional sites and blogs to index.

The crawler still needs some tuning for many of the sites listed above. For example there are still instances where the bot will return multiple identical results for a search term due to different urls displaying the same content.

Also ranking is still very basic and doesn’t incorporate anything like pagerank, although it can search using l-distance.

In addition to adding the encyclopedia (work on which is underway), I’m considering adding Bitcoin transaction search. I wrote my own app for such purposes last year and have found it occasionally useful. I don’t know how much demand there is for publicly tracking transactions, but it wouldn’t be a big deal to set something up and try it out.

In terms of infrastructure, the bot is now running on its own vps. The next step is automatic index updates.

Exploring WoT Search Project

Sunday, May 24th, 2020

I am in the process of exploring working on a WoT search bot. At the moment I don’t know how exactly to make a “WoT” search bot. I did a little searching of the logs, and this line from trinque aligns most with my own itch. Some possibilities:

  1. Everyone has their own search bot that searches sites they specify (presumably in their WoT).

  2. Anyone can register their list of sites they want indexed/searched1. Registered lists are associated with a nick. The search bot interfaces with deedbot to get WoT for your nick when you register and ranks search results originating from registered sites according to your WoT rankings when available.

  3. ??

I am not sure how to best display search results. I think I will start with maybe displaying the top 5 in chan/dm. I don’t know if I have the stomach for putting up a search webshit.

At the moment I am indexing Trilema. As soon as that finishes (if ever - currently going for about 10 hours) I’ll try to get lekythion (a bot I wrote a while ago for checking prices and logging #exusiae) serving up search results and see how that works out and gauge interest.

  1. It may be that at some point someone develops a WP plugin that the search bot can interface with that would provide “live” results and obviate the need for indexing mp-wp blogs []