Blatta 9986: Rebroadcast simple hearsay, embargo simple hearsay, more...

December 1st, 2021

Version 9986 includes:

- rebroadcast simple hearsay messages
- embargo simple hearsay messages
- more granular db write locking that should eliminate occasional slqite3.Programming error exceptions
- Pass broadcast message bytes straight through instead of recompiling them before rebroadcast
- Better handling of AT entries with empty addresses
- Fix erroneous references to 'logger' that should have been 'logging'

9986-rebroadcast-simple-hearsay-and-more.vpatch

9986-rebroadcast-simple-hearsay-and-more.vpatch.thimbronion.sig

Blatta 9987: hearsay embargo

November 29th, 2021

Version 9987 includes:

- embargoing of in-wot messages
- hearsay message labeling
- removed IRC message formatting from pest message body
- some removal of dead/unnecessary irc related code
- limit of one irc client connection per station
- fix for crash when messages sent to unknown handle
- proper padding
- message bytes complied one time only
- a small number of unit tests

Notes:

This version introduces threads in order to implement hearsay embargoing. This opens the possibility of a whole new world of bugs. The pest portion of Blatta has been refactored for ease of unit testing. I added unit tests to avoid recreating the actual network conditions that would produce the hearsay messages that need to be tested. I am considering adding unit tests for the message packing portion of the code. This could come in handy for others when building their own pestrons.

9987-embargoing.vpatch

9987-embargoing.vpatch.thimbronion.sig

Blatta 9988: deduplication by message hash

November 17th, 2021

Version 9988 includes:

- Overhaul of deduplication algo: Previously timestamps were used to determine whether or not a message was unique. Now a hash of the message is being used. WARNING: Messages from stations that have not yet applied 9988 may not be detected as dupes by 9988 due to the switch from timestamps to hashes. There were multiple issues in previous versions causing messages with identical irc message payloads to appear to be different when hashed.

- Fix for overzealous rubbish message broadcasts.

Additionally, I have started publishing my station's debug log in an effort to aid Blatta testers.

9988-hash-dedup.vpatch

9988-hash-dedup.vpatch.thimbronion.sig

Blatta 9989: presence for wot

November 14th, 2021

Version 9989 includes:

- joins automatically sent for peers in WOT.
- join/part sent for peer/unpeer commands.
- AT timestamp now updated on receipt of rubbish packets
- --channel-name command line option for specifying what chan your Pest network expects everyone to be on.
- Don't include hostname in irc messages from clients.

9989-show-wot-nicks.vpatch

9989-show-wot-nicks.vpatch.thimbronion.sig

Blatta 9990: rubbish messages

November 13th, 2021

Blatta 9990 implements rubbish messages in order to attempt to keep ephemeral ports open.

Also included is a fix to prevent Blatta from crashing on bad input to the AT command and some refactoring wrt message handling that doesn't affect behavior.

9990-keep-ephemeral-ports-open.vpatch

9990-keep-ephemeral-ports-open.vpatch.thimbronion.sig

Blatta 9991: Better logging and miscellaneous fixes

November 11th, 2021

Improved logging with abbreviated message hashes associated with addresses should help track down missing messages.

Improved logging of various things that can go wrong with a message.

Fixes a crash caused by attempting to add a duplicate handle.

AT timestamps are now only set when a message has been received from a peer.

Removed some options that shouldn't be used. Renamed --ports to --irc-ports for clarity

Poll the socket only once per second instead of continuously

Renamed default database file to blatta.db

9991-improved-logging.vpatch

9991-improved-logging.vpatch.thimbronion.sig

Blatta 9992: Better feedback and user input edge case handling

November 6th, 2021

Version 9992 consists entirely of bugfixes related to poor user feedback and poor handling of command input edge cases (errors when the wot is empty, there is no key for a handle, etc.)

9992-handle-edge-cases-add-feedback.vpatch

9992-handle-edge-cases-add-feedback.vpatch.thimbronion.sig

New Alcuin Release and Rename to Blatta

November 1st, 2021

As you may know, "blatta" is Latin for "cockroach." This name seems more aligned with the Pest theme, so I'm going with that.

I'm attempting to release this as a signed genesis. I haven't been able to figure out how to press it myself yet using vk.pl, but it *should* work for those who know how to use their v implementation.

This version implements several Pest-specific commands: AT, WOT, KEY, UNKEY, PEER, UNPEER, and GENKEY. In addition, this version requires only one key to be shared between a pair of peers, and also uses base64 format to store and display keys.

This is the first version that doesn't require a station to be restarted to add new peers, so it may be a bit easier at this point to stand up a longer running test net.

genesis.vdiff genesis.vdiff.thimbronion.sig

Alcuin 9994

October 11th, 2021

From the updated README:

Alcuin implements a subset of the Pest spec, which can be found
here: http://www.loper-os.org/pub/pest/pest_draft.html                                                    

Although it is a work in progress and much is yet to be completed, Alcuin
*should* be able to interact with another Pest client
successfully.

Notably missing:                                                                                          

- Pest specific client commands
- Pest specific warning/informational output for incoming/outgoing messages
- GetData message support
- Key Offer message support
- Key Slice message support                                                                               

Under the hood there is a dynamic AT and WOT, but there is no way to update
them via the client.

For the moment the client must still be configured via the config.py file,
although this is now passed in via the --address-table-path command line
option.  For now, the config file is really a combination WOT/AT.

I am unable to provide a genesis using vdiff at this time because I can't get vdiff to exclude certain ephemeral files and directories.

alcuin-9994.tar.gz alcuin-9994.tar.gz.asc

Alcuin 9995: propagate field

August 25th, 2021
  • add propagate field
  • don't propagate PMs
  • filter commands sent to client via udp by nick/channel

alcuin-9995.diff alcuin-9995.tar.gz

1
2 diff --git a/Makefile b/Makefile
3 index 7ab6ef1..5b21c74 100644
4 --- a/Makefile
5 +++ b/Makefile
6 @@ -12,9 +12,8 @@ dist:
7 rm -rf alcuin-$(VERSION)
8
9 clean:
10 - rm -rf genesis.diff alcuin-$(VERSION).diff.escaped alcuin-$(VERSION)
11 + rm -rf alcuin-$(VERSION).diff alcuin-$(VERSION).diff.escaped alcuin-$(VERSION)
12 rm *.tar.gz
13 - rm *.diff.escaped
14 find . -name "*.swp" -delete
15 find . -name "*.pyc" -delete
16
17 diff --git a/lib/client.py b/lib/client.py
18 index f3dc6ab..25ec4ab 100644
19 --- a/lib/client.py
20 +++ b/lib/client.py
21 @@ -30,6 +30,16 @@ class Client(object):
22 else:
23 self.__handle_command = self.__registration_handler
24
25 + def is_addressed_to_me(self, message):
26 + command = self.__parse_udp_message(message)
27 + if command[0] == 'PRIVMSG':
28 + if command[1][0][0] == '#' or command[1][0] == self.nickname:
29 + return True
30 + else:
31 + return False
32 + else:
33 + return True
34 +
35 def get_prefix(self):
36 return "%s!%s@%s" % (self.nickname, self.user, self.host)
37 prefix = property(get_prefix)
38 @@ -51,6 +61,30 @@ class Client(object):
39 def write_queue_size(self):
40 return len(self.__writebuffer)
41
42 + def __parse_udp_message(self, message):
43 + data = " ".join(message.split()[1:]) + "\r\n"
44 + lines = self.__linesep_regexp.split(data)
45 + lines = lines[:-1]
46 + commands = []
47 + for line in lines:
48 + if not line:
49 + # Empty line. Ignore.
50 + continue
51 + x = line.split(" ", 1)
52 + command = x[0].upper()
53 + if len(x) == 1:
54 + arguments = []
55 + else:
56 + if len(x[1]) > 0 and x[1][0] == ":":
57 + arguments = [x[1][1:]]
58 + else:
59 + y = string.split(x[1], " :", 1)
60 + arguments = string.split(y[0])
61 + if len(y) == 2:
62 + arguments.append(y[1])
63 + commands.append([command, arguments])
64 + return commands[0]
65 +
66 def __parse_read_buffer(self):
67 lines = self.__linesep_regexp.split(self.__readbuffer)
68 self.__readbuffer = lines[-1]
69 @@ -294,21 +328,18 @@ class Client(object):
70 targetname = arguments[0]
71 message = arguments[1]
72 client = server.get_client(targetname)
73 + self.server.print_debug(self.server.nicknames)
74
75 - if client:
76 - formatted_message = ":%s %s %s :%s" % (self.prefix, command, targetname, message)
77 - client.message(formatted_message)
78 - self.server.peer_broadcast(formatted_message)
79 - elif server.has_channel(targetname):
80 + if server.has_channel(targetname):
81 channel = server.get_channel(targetname)
82 self.message_channel(
83 channel, command, "%s :%s" % (channel.name, message))
84 self.channel_log(channel, message)
85 else:
86 - # this isn't reliably true so let's not send the error
87 - return
88 - self.reply("401 %s %s :No such nick/channel"
89 - % (self.nickname, targetname))
90 + formatted_message = ":%s %s %s :%s" % (self.prefix, command, targetname, message)
91 + if(client):
92 + client.message(formatted_message)
93 + self.server.peer_broadcast(formatted_message, False)
94
95 def part_handler():
96 if len(arguments) < 1:
97 diff --git a/lib/infosec.py b/lib/infosec.py
98 index a431c2f..b0c9c86 100644
99 --- a/lib/infosec.py
100 +++ b/lib/infosec.py
101 @@ -7,7 +7,7 @@ import time
102 import struct
103 import sys
104
105 -PACKET_SIZE = 580
106 +PACKET_SIZE = 581
107 MAX_MESSAGE_SIZE = 512
108 MAX_SECRET_SIZE = 24
109 TS_ACCEPTABLE_SKEW = 60 * 15
110 @@ -16,7 +16,7 @@ class Infosec(object):
111 def __init__(self, server=None):
112 self.server = server
113
114 - def pack(self, peer, message, timestamp=None):
115 + def pack(self, peer, message, propagate, timestamp=None):
116 # if we are rebroadcasting we need to use the original timestamp
117 if(timestamp == None):
118 int_ts = int(time.time())
119 @@ -34,39 +34,40 @@ class Infosec(object):
120 self.server.print_debug("added %d to recent" % int_ts)
121
122 # build the packet and return it
123 - packet = struct.pack("!L64s512s", int_ts, digest_bytes, ciphertext_bytes)
124 + packet = struct.pack("!L64s?512s", int_ts, digest_bytes, propagate, ciphertext_bytes)
125 return packet
126
127 def unpack(self, peer, packet):
128 try:
129 - int_ts, digest, ciphertext_bytes = struct.unpack("!L64s512s", packet)
130 + int_ts, digest, propagate, ciphertext_bytes = struct.unpack("!L64s?512s", packet)
131 except:
132 self.server.print_error("Discarding malformed packet?")
133 - return None, None
134 + return None, None, None
135
136 # Check the timestamp and digest
137 digest_check = hashlib.sha512(peer.remote_secret + ciphertext_bytes).digest()
138
139 if(int_ts not in self._ts_range()):
140 self.server.print_debug("rejected message with timestamp out of range")
141 - return None, None
142 + return None, None, None
143 elif(self.server.recent.has(int_ts)):
144 self.server.print_debug("rejected known message: %d" % int_ts)
145 - return None, None
146 + return None, None, None
147 elif(digest_check != digest):
148 self.server.print_debug("name: %s" % peer.name)
149 self.server.print_debug("remote_secret: %s" % peer.remote_secret)
150 self.server.print_debug("ciphertext_bytes: %s" % binascii.hexlify(ciphertext_bytes))
151 self.server.print_debug("digest_check: %s" % binascii.hexlify(digest_check))
152 self.server.print_debug("rejected bad digest: %s" % binascii.hexlify(digest))
153 - return None, None
154 + return None, None, None
155 else:
156 # Return the cleartext
157 serpent = Serpent(self._pad(peer.remote_secret, MAX_SECRET_SIZE))
158 cleartext = serpent.decrypt(ciphertext_bytes).rstrip()
159 self.server.print_debug("received, validated, and decrypted udp packet: %s" % cleartext)
160 self.server.recent.insert(int_ts)
161 - return cleartext, int_ts
162 + # self.server.print_debug("%s %d %s") % cleartext, int_ts, propagate
163 + return cleartext, int_ts, propagate
164
165 def _pad(self, text, size):
166 return text.ljust(size)
167 diff --git a/lib/peer.py b/lib/peer.py
168 index e108328..5c25d25 100644
169 --- a/lib/peer.py
170 +++ b/lib/peer.py
171 @@ -13,10 +13,10 @@ class Peer(object):
172 self.socket = socket.socket(family=socket.AF_INET, type=socket.SOCK_DGRAM)
173 self.infosec = Infosec(server)
174
175 - def send(self, msg, timestamp=None):
176 + def send(self, msg, propagate, timestamp=None):
177 try:
178 full_message = str.encode(msg)
179 self.server.print_debug("sending message: %s" % full_message)
180 - self.socket.sendto(self.infosec.pack(self, full_message, timestamp), (self.address, self.port))
181 + self.socket.sendto(self.infosec.pack(self, full_message, propagate, timestamp), (self.address, self.port))
182 except Exception as ex:
183 print("Exception while attempting to encode message: %s" % ex)
184 diff --git a/lib/server.py b/lib/server.py
185 index 9e8cbe5..bdd44d5 100644
186 --- a/lib/server.py
187 +++ b/lib/server.py
188 @@ -1,4 +1,4 @@
189 -VERSION = "9996"
190 +VERSION = "9995"
191
192 import os
193 import select
194 @@ -141,20 +141,23 @@ class Server(object):
195
196 def handle_udp_data(self, data):
197 for peer in self.peers:
198 - message, timestamp = self.infosec.unpack(peer, data)
199 + message, timestamp, propagate = self.infosec.unpack(peer, data)
200 if(message != None):
201 self.print_debug("valid message from peer: %s" % peer.name)
202 # send the message to all clients
203 for c in self.clients:
204 - self.clients[c].message(message)
205 - # send the message to all other peers
206 - self.rebroadcast(peer, message, timestamp)
207 + # self.clients[c].udp_socket_readable_notification(message)
208 + if (self.clients[c].is_addressed_to_me(message)):
209 + self.clients[c].message(message)
210 + # send the message to all other peers if it should be propagated
211 + if(propagate == True):
212 + self.rebroadcast(peer, message, timestamp)
213 else:
214 self.print_debug("Unknown peer address: %s" % peer.address)
215
216 - def peer_broadcast(self, message):
217 + def peer_broadcast(self, message, propagate=True):
218 for peer in self.peers:
219 - peer.send(message)
220 + peer.send(message, propagate)
221
222 def rebroadcast(self, source_peer, message, timestamp):
223 for peer in self.peers:
224