July Search Update

July 18th, 2020

Work is progressing on the search bot.

I now have a configuration that will index the btcbase.org logs. The resulting index is not perfect - anything from reddit is excluded due to my ip being blocked, and many archive.is pages are not successfully indexed due to archive.is periodically going offline. There is an error when attempting to index any Bitcointalk link that I haven’t been able to resolve. Also, due to the timespan involved, many links have rotted and are lost forever. Most links provided as “shortened” links also no longer work. The results for this crawl should show up in the bot’s index about one week from now.

Work on the encyclopedia crawl progresses as well. Apify delivered a half-functional crawling script that works with their platform. At the moment I don’t have a configuration which allows the crawl to get all volumes of the encyclopedia. I am currently working with support to get this resolved.

So I'm getting divorced

June 28th, 2020

So I’m getting divorced. Not because I had the courage, like I should have had, to have broken it off years ago, but because of a condom wrapper my wife found in my back pocket on the morning of Father’s day. My wife wants to try to fix things, but I don’t, and I’ve said that. I welcome the shift from feeling anger, resentment, and fear daily to mostly sorrow and relief. She’s been very apologetic, and I should have given her the chance to respond to how I felt, like I said above, many years ago, before it was way too late. In any case I have my freedom now, like I could have had all along if I had any courage at all.

Crawling btcbase.org/log

June 18th, 2020

The results for lekythion’s first crawl of btcbase.org are in. The index created is not of much use due to the crawler being blocked by many sites including archive.is, reddit and tardstalk for not complying with the robots.txt files.

Nevertheless I now have a comprehensive list of links from the #trilema logs. I think the index should include the logs themselves, but it might be convenient to be able to compartmentalize crawling external links into a separate task/configuration.

The Voice of Trilema

June 8th, 2020

Now that I’ve been reading Trilema (successfully or not) for about six years, I thought I’d take some time to comment on the style and voice of the English portion of the the blog.

His voice is distinctive in many ways, but some of the distinctive features that most stand out to me are mircea_popescu’s frequent use of the 2nd person and the the use of graphic sexual metaphor to illustrate many, many concepts.

Given the prominence of this person in his writing, it wouldn’t be a waste of time to think a little bit about who exactly is “you.”

Is it really me, the person reading? In many cases yes, if I go by his qualifications - those usually being something along the lines of either not having read some text or not having some particular thought on my own by my late teens. The interesting thing is, after time, if I come back and read an article again, the “you” is less and less me, and more someone else, and the article begins to feel off - because I’ve likely read the text or internalized the thought by that point.

Whoever the “you” is, mircea_popescu in this voice at least plainly holds the reader in contempt. I think this is intended as a form of rhetoric. In other words he’s using this voice to evoke particular emotions in the reader for persuasive purposes. Whoever the reader is he is in a miserable state and is absolutely responsible for being in this state and is responsible as well for his parents and grandparents sucking and so forth. There is no quarter given, without exception, except perhaps to those who realize their worthlessness and die either by their own hand or in some highly risky endeavor1. Now I have to be careful here because I know it’s impossible to know what someone else is thinking, and I certainly don’t want to appear to be making that claim. I am only speculating. In any case, I speculate that the intended effect of this use of rhetoric is to arouse a feeling of shame. Shame is painful. “You” are to be spurred on by this pain. “You” are to do whatever is necessary to deal with the pain. In my observations of some others reading Trilema, this pain turns them immediately away and they never come back. Others, such as myself, somehow either become addicted to the pain, or perhaps keep reading on to find some sort of salvation from it2.

As for the frequent use of graphic sexual metaphor mircea_popescu himself explains why he prefers this style. I suspect this rhetoric also is meant to instill pain in many readers, and perhaps levity in others. Either way it is memorable, and I can’t think of any other writer using sexual metaphor to this extent.

Another aspect of his style that I’d like to comment on is the … finality, the unforgivingness. Read any article and there is a very good chance he’ll make you aware of a mistake you’ve made that can never, ever be corrected, only regretted. It is interesting to contrast this with other writers in the Anglosphere today, where there is endless redemption to be found.

And while there is doubtless much more to be said, this is all my cup of chai will allow me to put forth today.

  1. Not that it matters but I don’t particularly disagree with this attitude []
  2. Pain itself in various forms is a major theme in Trilema articles and it makes sense that given the significance of pain’s role in mp’s thoughts on education, reading Trilema itself would be painful []

Lekythion Search Bot Source

June 7th, 2020

I am publishing the current source for the lekythion search bot here for anyone who may be interested. If anyone should later be interested in publishing a patch, I will go to the further trouble of creating a V genesis. I suspect the widespread allergy to Ruby will prevent that from ever happening, but the offer stands.

Also, lekythion now gets an updated index every 2 days.

Lekythion search update: additional blogs and other things

May 31st, 2020

I’ve added several new blogs to lekythion’s search feature, including:

trinque.org
loper-os.org
thimbron.com
ossasepia.com
fixpoint.welshcomputing.com
billymg.com
ztkfg.com
qntra.net

I intend to add more as I have time. I am also open to suggestions for additional sites and blogs to index.

The crawler still needs some tuning for many of the sites listed above. For example there are still instances where the bot will return multiple identical results for a search term due to different urls displaying the same content.

Also ranking is still very basic and doesn’t incorporate anything like pagerank, although it can search using l-distance.

In addition to adding the encyclopedia (work on which is underway), I’m considering adding Bitcoin transaction search. I wrote my own app for such purposes last year and have found it occasionally useful. I don’t know how much demand there is for publicly tracking transactions, but it wouldn’t be a big deal to set something up and try it out.

In terms of infrastructure, the bot is now running on its own vps. The next step is automatic index updates.

Rule Britannica

May 27th, 2020

Absolutely Nobody:

Me: Let’s scrape the 1910 Encyclopedia Britannica and load it up into an mp-wp instance.

When writing qntra articles I often found myself depending on Pediwiki to find original sources when doing background checks. This was a revolting experience to say the least.

To address this, I am standing up an mp-wp instance containing the the entire 11th edition of Encyclopedia Britannica, which is available in unusable form only from the Internet Archive and Project Gutenberg. It goes without saying that this will be added to lekythion’s search index.

Obviously it won’t contain bios of more recent public figures, but it is a start, and would perhaps have come in handy when writing the unpersoning piece.

I look forward to being able to reference specific phrases/lines in the best encyclopedia ever.

Search Prototype

May 26th, 2020

So the search project produced a prototype, which is available in #exusiae.

It works like this:

18:00:07 thimbronion !s Bitcoin
18:00:08 lekythion 10 results
18:00:08 lekythion ³http://trilema.com/2013/bitcoin-prices-bitcoin-inflexibility/
18:00:08 lekythion Bitcoin prices, Bitcoin inflexibility on Trilema - A blog by Mircea Popescu.
18:00:08 lekythion …keeping the Bitcoin). Other than this ~4% of the Bitcoin monetary…
18:00:08 lekythion …Bitcoin. Will people stop throwing dollars at Bitcoin because Bitcoin
18:00:09 lekythion …Will people start throwing Bitcoin at dollars because Bitcoin prices…
18:00:10 lekythion ²http://trilema.com/2015/introducing-the-bitcoin-isp/
18:00:11 lekythion Introducing the Bitcoin ISP on Trilema - A blog by Mircea Popescu.
18:00:12 lekythion …Bitcoin, The Most Serene Republic Of ~. In any case, Bitcoin ISP will…
18:00:13 lekythion …Bitcoin ISP, your only avenue is to voice your concerns in #bitcoin
18:00:14 lekythion …Soon to become a Bitcoin registered company, trading as S.BISP…
18:00:15 lekythion All results can be found at ¹http://paste.deedbot.org/?id=ZwnE.

The bot currently only searches an index of trilema.com. The !s command accepts Apache Lucene queries.

I now confront some problems.

  1. Fine tuning was required. I had to tune the indexer to extract certain elements from Trilema to get the quality of the results somewhere near acceptable. This means every site is going to need tuning. For example, the good stuff is all the div.entry class in mp-wp, while trinque.org has it somewhere else.

  2. I don’t yet know how to let others add sites they want to search. This is partially due to the first issue because if I just take lists of sites from people and don’t customize the indexer, the results won’t be great. It’s also due to not knowing the best way to allow users to configure their lists of sites to index. The first thing that popped into my head was to allow users to sign a text file that includes a list of all the sites they want to index and provide that to me. I would then do the configuration on the server manually and associate that index with their nick such that it would be the default index searched whenever they search. Perhaps at some point users could also specify by WoT identity others’ indexes they’d like to search.

One positive result is that after futzing around trying to use Google to find particular Trilema articles, I find using my own index to be much more productive.

Sites and documents I personally want to index:

trilema.com
loper-os.org
the blog of everyone from #ossasepia
thebitcoin.foundation
the naggum archive
Encyclopedia Britannica, 11th Edition
bitcointalk.org

Exploring WoT Search Project

May 24th, 2020

I am in the process of exploring working on a WoT search bot. At the moment I don’t know how exactly to make a “WoT” search bot. I did a little searching of the logs, and this line from trinque aligns most with my own itch. Some possibilities:

  1. Everyone has their own search bot that searches sites they specify (presumably in their WoT).

  2. Anyone can register their list of sites they want indexed/searched1. Registered lists are associated with a nick. The search bot interfaces with deedbot to get WoT for your nick when you register and ranks search results originating from registered sites according to your WoT rankings when available.

  3. ??

I am not sure how to best display search results. I think I will start with maybe displaying the top 5 in chan/dm. I don’t know if I have the stomach for putting up a search webshit.

At the moment I am indexing Trilema. As soon as that finishes (if ever - currently going for about 10 hours) I’ll try to get lekythion (a bot I wrote a while ago for checking prices and logging #exusiae) serving up search results and see how that works out and gauge interest.

  1. It may be that at some point someone develops a WP plugin that the search bot can interface with that would provide “live” results and obviate the need for indexing mp-wp blogs []

Costa Rica Servers

May 5th, 2020

I have found a hosting service in Costa Rica called Costa Rica Servers that accepts Bitcoin as payment. I can’t find any references to this company in the logs1.

In their favor they accept Bitcoin payment via BTCPay as opposed to Bitpay. A strike against them is that they also accept payment in two shitcoins not worth mentioning.

When I asked them about colocation and payment, I got a reply from Edgar Fernandez who seems unfortunately to be involved with EOS.

Hi Adam,

Thanks for your inquiry. The best way to bring the server is to use a Customs Agent / Courier service and have them handle the local customs. We use Star Box (http://starboxcr.com/)

As for the BTC payments, we accept BTC/LTC/Dash through a BTCPay service we run in our own server where the USD invoice is converted at the time of payment at the rate published by Kraken exchange at the time. We welcome BTC payments and provide discount codes for new clients paying in BTC!

Hope you decide on CRServers.com!

Please let us know if we can help with the introduction to Star Box.

Thanks

Edgar Fernandez
EDENiA
edgar@edenia.com

In terms of their network connectivity, they’ve provided a little bit of info on their datacenter connectivity page. I’m not in a position to compare this to Latechco’s connectivity and would appreciate any pointers as to how to do so.

I am planning a trip to Costa Rica in June to fulfill my visa requirement that I physically be in Costa Rica at least one day per year. I may take this opportunity to colocate a box at Costa Rica Servers, but I haven’t decided yet.

  1. I can only assume CR was specifically excluded in the Pizarro ISP search based on some criteria - but I don’t know what that would be []