Feb 2022; Copyright MyCHIPs.org
A MyCHIPs network potentially consists of many different processes running on many different machines. A single site is modeled as shown here:
The different components of the system will communicate with each other by passing messages encoded as JSON packets. The common cases, covered in previous sections, are:
- User application <-> User service / Database; (more detail)
- Database <-> Agent service
- Peer Agent service <-> Peer Agent service (peer-to-peer)
In general, the user application will interact with the system by doing one of the following:
- Sending an SQL command, encoded in a JSON record, to the database;
- Invoking a control layer action handler, which will likely further interact with the database.
The mobile application will generally not be able to interact with a peer agent module as this may be an entirely separate process from the User service.
Communications related to (or destined for) peer agent services are handled by one of four different subsystems as follows:
- tally
- chit
- route
- lift
Most messages in the following cases will have a property called "target" containing the name of the applicable subsystem. The primary exception is in communication from the agent process to the database since the subsystem is already inherently known by the database procedure being called.
MyCHIPs is structured as a model-view-controller pattern. The database is the authoritative repository of the system state (the model). Processes in the control layer (user and agent processes) and or in the mobile app or other view applications should consider their data transitory and should rely on the model as much as necessary.
Each object (such as a tally) in each subsystem should contain a status field. The object's status (possibly combined with other key fields) will indicate the state of the object. The state will be used to make key decisions about how/when to transition to a new state. Hopefully this will allow erronious or late messages to simply be ignored.
A subsystem may receive asynchronous requests from either a local user or a remote peer agent.
A user request is typically generated by writing a non-null value into the object's DB request field. This implies the need for action by the applicable agent process in order to actually accomplish the state change. The value in the request field typically indicates the next value intended to be written into the object's status field. The database will detect the request and generate a notice to the agent process on the following channel:
ma_AGENT_PUBLIC_KEY
where AGENT_KEY_PUBLIC key indicates the agent address for the process responsible for traffic for the applicable user. Usually the agent process will need to communicate something to an external peer process and then issue a message back to the database indicating it is appropriate now to transition to the requested state (the new status value).
Similarly, an agent process may receive a message from a peer agent at any time. These messages also request a change in state of the targeted object. A corresponding state change command can be issued directly to the database but it is contingent upon the existing state. In this way, commands will only be considered valid if the object is still in the intended state (context) at the instant they are interpreted.
Previous to 2022, the states of subsystem objects (tallies, chits, routes, lifts) were arbitrarily named. Moving to protocol >= 1.0 they utilize a state format encoded as follows:
- type.status.request (tallies, chits)
- status.freshness (routes)
- status.request (lifts)
where "status" indicates the current status of the object record. The state might also start with a "type" which is simply there to differentiate states that may be distinct, but share a common status. For example a chit might be pending in several different states, but we need to know the difference between a chit from our user and a chit to our user.
The optional "request" part similarly indicates the value of the request field in the object. Having a request component indicates this is a state that needs attention from the agent process.
States requiring action from the user should have a null request in the object, and there is no direct indication from the state code. However, the DB view should provide a boolean "action" field indicating the record is in one of those states.
Messages from one agent to another peer agent are sent over noise protocol
Messages are currently encoded as JSON structures. Future implementations might move to a more compressed binary form (or at least a form where properties are much more abbreviated.) But the current form is quite human-readable and so may be used for some time--particularly in testing/validation phases.
Messages destined for another user's agent will typically have the following basic properties:
- target: What kind of object (subsystem) the message applies to. For example, tally, chit, route, lift.
- action: A code representing what step or function is being requested or communicated via the packet. This may well be the name of state that is being suggested as the next step in a negotiation.
- object: The contents of the object (tally, chit, route, lift) itself.
- try: An integer starting at 1 indicating how many times the sender has tried to send this message.
- last: A timestamp for when the message was generated.
- to: The CHIP Address of the recipient, containing some or all of the following, as applicable:
- cid: The CHIP ID of the sender.
- agent: Encoded agent public key
- host: Hostname or IP number
- port: Port to connect on
- from: The CHIP Address of the sender, containing some or all of the following, as applicable:
- cid: The CHIP ID of the sender.
- agent: Encoded agent public key
- host: Hostname or IP number
- port: Port to connect on
Once an agent receives a message, it may trim the message to only those properties that are required before forwarding the message on to the database. Some fields not ever required by the database include: target, action, from, last, try.
Some messages from the agent to the database are intended to insert or update the content of the object (tally, chit, route, lift) itself. These messages may include the object property in its full JSON exchange format. In simpler cases, a message may only need to change the status of the object (for example, voiding or closing) so the whole object may not be necessary. However, at a minimum the object will have to contain any identifiers (uuid, for example) needed to uniquely locate the object in the database.
Agent actions sent to the database also include the following recipe object containing:
- context: The object must be in one of the specified states in this array before the contained state transition command will be allowed to be processed.
- update: An object describing DB fields to update. Typically this is to move the object from one state to another by updating the status field but can also include other DB fields.
- upsert: The object in the DB should be updated according to the attached object. If it doesn't exist, it should be created.
- query: Special directive for route/lift subsystems to query for upstream routes/lifts.
Property: target: tally
The object property for the tally is defined as follows:
- version: Version (1) of the tally format.
- foil: The CHIP ID of the customer/employer/client.
- stock: the CHIP ID of the provider/employee/vendor.
- uuid: A Universally Unique Identifier for this particular tally.
- comment: Any comments to the tally the parties have agreed to include.
- created: Date/time the tally was created
- contract: A reference to a digital contract containing the general terms and conditions for the tally. Can be the document digest, or an object containing:
- host: Web address of Issuer/author guaranteed to have authoritative copy of the contract
- digest: Base 64 hash of the contract
- terms: An object containing credit variables
- signed:
- digest: A string hash of the rest of the tally in a standard serialized format
- foil: The digital signature of the hash by the foil holder (client)
- stock: The digital signature of the hash by the stock holder (vendor)
Tally state transition messages are as follows:
-
DB->Agent, Agent->Agent; Initiate Tally Connection; The process is responding to a tally invitation by requesting that a connection be opened to the site containing the tally and asking that the actual tally draft be sent back.
- action: ticket;
- ticket: a connection ticket (i.e. not a tally yet!) containing:
- token: connection code
- cert: CHIP certificate of the subject peer who will receive the tally
-
DB->Agent: Request Agent to Send a Tally: action: pend
The DB requests the agent to send the contained tally proposal to the requester or intended partner. -
Agent->Agent: Sending a Tally Offer: action: offer
A peer agent is sending the contained tally intending to execute on the proposed terms. -
DB->Agent: Request Agent to Refuse Tally: action: void
The DB requests the agent to tell the prospective partner of the tally "no thanks." -
Agent->Agent: Refusing a Tally Offer: action: void
A peer agent is indicating that the referenced tally has been refused by its user. -
DB->Agent: Request Agent to Accept Tally: action: open
The DB requests the agent to tell the prospective partner of the contained, signed tally "I accept." -
Agent->Agent: Accepting a Draft Tally Offer: action: open
A peer agent is transmitting a tally that has been accepted and signed by its user. -
DB->Agent: Request Agent to Notify of Tally Close Request: action: close
The DB requests the agent to tell the partner of the current tally that it has been marked by our user to be closed upon attaining a zero balance. -
Agent->Agent: Marking a Tally for Closing: action: close
A peer agent is indicating that the referenced tally has been marked for closure by its user.
Property: target: chit
The object property for the chit is defined as follows:
- uuid: A Universally Unique Identifier(UUID) for this particular chit.
- tally: The UUID of the tally this chit belongs to.
- issue: The value 'stock' or 'foil' to indicate which holder issued the chit pledge (or setting).
- units: The number of milli-CHIPs on this chit. This should always be a positive number.
- date: Date/time the chit was became/becomes effective.
- type: The value 'tran' (transaction), 'set' (setting) or 'lift' (linear or circular)
- ref: A JSON data structure containing invoice, order, or other references material to the transaction.
For setting chits, this contains the values of the settings:
- For trading variables, use the property values explained here (target,bound,reward,clutch).
- To mark a tally for closing, specify the property 'close' as true.
- To specify a tally call date, use the property 'call'.
- To specify a tally call date, use the property 'call'.
- memo: A human-readable description or comment about the transaction.
- digest: A string hash of the rest of the chit in a standard serialized format
- signed: The digital signature of the hash by the grantor, whether Client or Vendor
- digest: the new endHash computed as a result of adding the present valid, signed chit (if applicable). See below for more information about use of this property.
Chit state transition messages are as follows:
-
DB->Agent: Request Agent to Invoice: action: pend The DB requests the agent to send the payment request to the indicated peer.
-
DB->Agent: Request Agent to Cancel: action: void The DB requests the agent to notify the peer that the referenced payment is refused.
-
DB->Agent: Request Agent: action: good The DB requests the agent to notify the peer that the referenced payment is approved.
-
Agent->Agent: Request Payment: action: pend A peer is requesting payment from our user on the indicated tally.
-
Agent->Agent: Refuse Payment: action: void A peer is refusing payment to our user on the indicated tally.
-
Agent->Agent: Approve Payment: action: good A peer is confirming/approving payment to our user on the indicated tally.
In addition to the four main subsystems mentioned above, there is a sub-protocol for maintaining consensus between the stock and foil about the order and integrity of chits recorded on the tally (the chit chain).
This can be thought of as a set of sub-states a tally can be in while its main state is open (and/or closing). For example, a stock can be said to be consensed or not yet consensed with its foil counterpart. But we can also think of consensus as an extension of the chit protocol because individual chits can either be linked into the chit chain or not yet linked.
This makes it difficult to attribute a single, granular consensus state to tally. A tally can have some chits linked, others in the process of being linked and still others needing to be linked.
So for purposes of negotiating consensus, we will define a substate that is applicable to individual chits. A foil will be considered in consensus when every good chit has been linked into its chain. A stock will be considered in consensus when every good chit has been linked into its chain and the resulting end hash has been confirmed with the foil.
Since attaining consensus is the result of hash-chaining chits, we will utilize the chit message handling module (i.e. property target = chit). Consensus messages specify the property action = 'chain' which means "Hand this message to the consensus processing module."
The consensus state machine references the following message sub-commands:
- new: (virtual) Foil sends valid chit, accompanied by new endHash
- new: (virtual) Stock sends valid chit, accompanied by proposed endHash (propHash)
- req: Stock requests chits since last acknowledged endHash (ackHash) or some other starting point in the chain.
- upd: Foil sends latest ending hash (ackHash) as well as an optional list of potentially unknown chits. If the chit list is empty, this implies an ACK (acknowledge) of some prior provisional endHash the stock may have sent.
- err: An error occurred while attempting to reconcile with a specified update packet.
The first two actions are directly correlated with the sending of a chit--a message already occuring within the chit subsystem. So these actions are accomplished simply by populating the additional chain property in the applicable chit message. The latter three will encode the desired command into the property sub.
Chain messages then contain an object with some or all of the following properties:
- action: = chain
- sub: The chaining subcommand
- tally: The uuid of the tally the message pertains to
- hash: The hash at the end of the chain
- index: The index of the last chit in the chain
- chits: An array of chits from a section of the chain
Property: target: route
The object property always contains:
- find: A CHIP address for the destination entity of this route. Must minimally contain a cid and agent. Host and port components may provide hints to intelligent algorithms about where to best find successful routes.
- tally: The UUID of the tally this route query is being sent over (and expected to be returned on).
When a route query is being sent upstream, the object property also includes:
- step: Tracks how many hops away from the original inquiry this query has come (0 on first query).
When a route response is being propagated downstream, the object property also includes:
- lading: Expresses the approximate capacity/cost of the route for carrying lift value.
- min: The amount of value that can be transmitted without incurring extra cost
- max: The greatest amount of value that can be transmitted under any cost scenario
- margin: The base cost applied to a lift of any value
- reward: An extra cost applied to any lift amount above min (and below max)
Route state transition messages are as follows:
-
DB->Agent: Request Agent to Query: action: draft The DB requests the agent to send the route request upstream to the indicated partner.
-
DB->Agent: Notification of Good: action: good A relay route has just been marked good locally. The DB requests the agent to send the good status downstream to the partner who requested it.
-
Agent->Agent: Route Query: action: query A downstream peer agent is asking us for possible routes to a designated destination entity.
-
Agent->Agent: Route Failed: action: none An upstream peer is indicating that no possible further upstream route is possible for the applicable query.
-
Agent->Agent: Route Succeeded: action: good An upstream peer is indicating that one or more routes are possible to the requested destination. A lading record is returned indicating a good (or best) approximation of what lift capacity will be available if requested.
Property: target: lift
Any lift message may contain the following as applicable:
- tally: The UUID of the tally this lift is being sent over (and where signature validation is expected to be returned).
Lift messages between a local DB and its agents may contain as applicable:
- sequence: The sequence number (0..n) of the lift intended to be affected by this operation.
- outc: CHIP Address of the output (foreign peer) of the local lift segment.
- topc: CHIP Address of the owner of the top tally in the local lift segment.
- topt: The ID of the top (output) tally in the local lift segment.
- inpc: CHIP Address of the input (foreign peer) of the local lift segment.
- botc: CHIP Address of the owner of the bottom tally in the local lift segment.
- bott: The ID of the bottom (input) tally in the local lift segment.
For the special case of the init action (on the lift originating site), the message will contain:
- init:
- find: Destination CHIP Address to be included in the outgoing packet.
- org: Suggested org (origin) property for the outgoing packet object.
- ref: Suggested ref (referee) property for the outgoing packet object.
- circuit: Boolean indicating if this is a circular (as opposed to linear) lift.
The object property includes the following properties:
- lift: A Universally Unique Identifier(UUID) for this particular lift.
- date: Date/time the lift was created
- life: The number of seconds from creation before this lift will be deemed expired.
- units: The number of milli-CHIPs on this lift.
- find: The CHIP Address of the lift destination entity, containing some or all of the following, as applicable:
- cid: The CHIP ID of the destination/recipient entity (required).
- hid: A hashed CHIP ID of the recipient (for greater privacy)
- agent: Encoded agent public key (required).
- org: The CHIP Address of the lift originator, containing some or all of the following, as applicable:
- agent: Encoded agent public key (required)
- host: Hostname or IP number (optional to allow signature queries from relays)
- port: Port to connect on (ditto)
- plan: Encrypted object from the originator for destination's eyes only
- cid: Originator CHIP address to send final promise to (or from whom linar lift is payment).
- agent: Agent address of originator CID (default to org.agent)
- pay: True if this lift is a linear payment (otherwise, it is a circular, clearing function)
- for: Payment reference, invoice number, register number, etc.
- memo: Note about payment
- ref: The CHIP Address of the lift referee (chip/noise only supported at this time)
- agent: Encoded agent public key (assuming chip protocol)
- host: Hostname or IP number (optional)
- port: Port to connect on (optional)
- valid: A boolean value true or false, based on the referee's decision. This property is always null (or omitted) on a lift's promise phase.
- signed: The digital signature by the referee of a hash of the above properties. If the lift is deemed good (valid), the hash is computed with the valid property set to true. If deemed void (expired), the hash is computed with the valid property set to false.
Lift state transition messages are as follows:
-
DB->Agent: Request Agent to Seek Lift: action: promise The DB requests the agent to send the enclosed lift promise over the indicated tally to its partner peer.
-
Agent->Agent: Forward Promise: action: promise A downstream peer agent is asking us to participate in the attached lift. We agree by forwarding it along a route we believe will be successful. We decline by ignoring the packet and letting it die by timing out. Declining will probably result in our route getting marked as faulty so we may not get more lift requests over this channel until that is repaired.
-
Agent->Agent: Request Lift Status: action: check A peer agent is requesting the status of a lift. This might be because we are the lift originator or because we are a referee site for the lift. Originator sites should answer these queries if they published their port information in the lift. Otherwise, they can ignore them. Referees should always answer such a query. Object packets for this query are only required to include the uuid property--particularly if we are querying an originator or peer site. If we are querying a referee, we should want to include the entire, original object so we can obtain a void signature if the lift is deemed expired.
-
Agent->Agent: Reply With Status: action: status A peer agent is giving us the status of a lift. In this case, the object need only contain properties: uuid, valid, and signed. The recipient is expected to already have the rest of the lift object. No need to share or transmit extra data that is not needed.
-
Agent->Agent: Request Commit: action: commit This message should be transmitted only from the originator to the referee. In this case, the signed property should include the signature of the originator (with the valid property set to true). The referee should replace this signature with its own signature (and not transmit it further). However, if the lift is approved, the referee must keep the originator signature in its own database to prove that it acted only upon approval by the originator.
In addition to real-time, dynamic messages, certain objects may be transmitted over other media or in other ways. For example, a site may issue to a user a connection ticket containing a one-time token and other connection information. Users can issue tally invitations to each other. And users may have to export keys or other data and then re-import that data on another device or platform.
In these various instances, data may often be represented as either a JSON record or as a URI. Data may also be represented as text or as a QR code. Ideally a MyCHIPs mobile app should be able to scan a QR code containing any applicable record and know what to do with it.
- Connection Ticket: Issued by Chip Service Provider
URI:'https://HOST:HTTP_PORT/user.html?&port=WS_PORT&token=TOKEN$%user=USER'
JSON:token: TOKEN, host: HOST, port: WS_PORT, user: USER, expires: EXPIRATION_DATE }}``` - Tally Invitation: Issued from one user to another
URI: 'https://mychips.org/tally?token=TOKEN&chad={cid:w,agent:x,host:y,port:z}'
JSON:
{invite: {
token: TOKEN,
chad: {
cid: USER_CHIP_ID,
agent: USERS_AGENT_PUBLIC_KEY,
host: AGENT_HOST_ADDRESS,
port: AGENT_HOST_PORT,
}
}}
- Connection Key(s): For backup/restore and sharing to another device
JSON (unencrypted, not recommended):
{connect: {
kty: "EC", //Example data
crv: "P-256",
x: "MKBCTNIcKUSDii11ySs3526iDZ8AiTo7Tu6KPAqv7D4",
y: "4Etl6SRW2YiLUrN5vfvVHuhp7x8PxltmWWlbbM4IFyM"
}}
JSON (encrypted):
{connect: {
s: SALT,
i: INITIALIZATION_VECTOR,
d: ENCRYPTED_DATA (should decrypt to serialized JWK key as above)
}}
- Signing Key(s): For backup/restore and sharing to another device
JSON (unencrypted, not recommended):
{sign: {
// jwk key data properties as above //
}}
JSON (encrypted):
{sign: {
// Encrypted key properties as above //
}}
- User Record: For backup/restore and sharing to another provider
JSON:
{user: {
// See data example in test/mocha/user.json
}}
- Tally Record: For backup/restore and presentation to another user
JSON:
{tally: {
// See data example in test/mocha/tally.json
}}
The JSON records above all feature an outer wrapper of sorts where in the top level, there is a property naming the type of record to follow.
Where applicable, multiple such records can follow the named property by simply using a JSON array rather than a JSON object. (See example in test/mocha/users.json.)
Also, in contexts where the type of the record is already clearly know, the top-level named property can be omitted and the inner record can stand alone.