This document describes the plexnet data layer, especially Units and Entities. This is not intended as a general overview of the whole system, just the layer where the storage and transfer of information happens. This document is based on discussion with tav, the designer of the plexnet.
One of the things the plexnet needs to do is store data. In this document we'll concentrate on the simple use case of sending a message, such as saying that the West Indies beat Australia at cricket.
The most fundamental kind of thing in the plexnet data layer is the Unit. A Unit is roughly speaking a kind of hashtable, though more exactly it's a field tree. We can make more complex datatypes out of Units. According to the espra bootstrap document, there are nine predefined complex datatypes: Items: (1) the Message Item that we're concerned about here, (2) Action, (3) Question, (4) Answer, (5) Pecu Allocation, (6) Transaction, (7) Requirement, (8) Shaila, (9) Text. Each of these kinds of Item are implemented as kinds of Units, with specific fields and values and other constraints.
So, a Message might look like this, using Python syntax for a hashtable, and remembering that actually we're using field trees:
{'content': "The West Indies beat Australia",
'__id__': 'ahwouaeaeaeaooa2180',
'context': ['sport', 'cricket']}
How do we know that this is a Message Item? Because Message Items always conform to a Schema. At this point we must introduce the simple builtin datatypes, such as Text, Binary, Set, Dict, List, Sequence, Matrix, Integer, Float, Number, and so on. One of these simple datatypes is the Function: a Schema is a native Function. Everything that is used in a Unit must be one of these simple datatypes. Since Schemas are Functions, they are used inside Units themselves: no separation of data and metadata.
One of the primitive datatypes is Binary, which represents of course binary data. But binary data is large and generally obtrustive, so we store it in a separate Storage facility. This is a separate hashtable for storing large binary objects.
So again, we have a Unit which is a kind of hashtable. These Units store data, and if this data conforms in a loose duck typing sense to a special kind of native function called a Schema, then it's said to be a kind of that particular Schema; and of course it may be a kind of more than one Schema simultaneously. There are nine kinds of complex datatypes which of which has a predefined Schema, and each of these complex datatypes is called an Item. One of these Items is the Message item.
The rôle of an Entity is to store all of the data and metadata for a particular user, such as public and private key pairs, references to the EntitySpaces, and so on. Part of what an Entity does is to act as a kind of storage mechanism for Units. Units do not, however, get stored directly in Entities. In data modelling terms, the references to the EntitySpaces are actually sets of tuples, in the form (EntitySpace, IndexStore, EventStore). An EntitySpace is a set of Units, so this is where Units such as Messages, Text, or Actions are held, plus any other Units which might not conform to one of the Item Schemas.
An IndexStore is a way of storing the data in an EntitySpace so that it can be efficiently queried. So for each {UnitID: {prop: value}} in the EntitySpace, the IndexStore will have a corresponding {prop: {value: {UnitID1, UnitID2}}}.
In networking terms, the EntitySpace and IndexStore in a particular tuple may be held in the current Node, the client on a computer. But the EventStore need not be: it can be held anywhere. An EventStore is simple a bag of Sensors. A Sensor is a kind of native function, a bit like Schema, but a Sensor basically consists of a pattern to look out for, and then the function sends a Unit with references to the created, modified, or deleted Units as a notification to a preset EntitySpace.
So for example, the sensor Sensor({content: /West Indies/i, callback=['someentitystore']}) would match our Message example where the West Indies best Australia. It would then be able to perform some action to the named callback EntityStores; that action could be specified in further keyword arguments to the Sensor. Sensor shouldn't be misunderstood as traditional RPC: Sensors simply relay that something has happened, whereas Services, introduced in the next section, have the side effects.
So say that we put our Message into a particular EntitySpace. That means that a Unit is created in that EntitySpace. At this point, all appropriate native functions called Services will be updated. The concept of a Service is a bit like the functions in Onward09.pdf. The Services in the EntitySpace can have side effects when they're activated: for example they should set off a foghorn, or they could notify the user about something. Then if the transaction goes through, i.e. the Unit is added successfully and the Services are updated, then the IndexStore is updated with the relevant entries as a changeset. At that point, the EventStore Sensors are run against the changeset too, and they're activated if there's anything that corresponds to them in that changeset.
Since EventStores can be distributed, people can have an EventStore locally for some remote events (i.e. changes to Units in EntitySpaces) that they're monitoring. Using this mechanism, one can watch for messages being sent elsewhere. When a Sensor detects some message, it can update an EntitySpace in such a way that a Service is invoked and that can notify the user for example.