Sep 2024; Copyright MyCHIPs.org
A MyCHIPs network potentially consists of many different processes running on many different machines. A single site is modeled as shown here:
The different components of the system will communicate with each other by passing messages encoded as JSON packets. The common cases, covered in previous sections, are:
- User application <-> User service / Database; (more detail)
- Database <-> Agent service
- Peer Agent service <-> Peer Agent service (peer-to-peer)
In general, the user application will interact with the system by doing one of the following:
- Sending an SQL command, encoded in a JSON record, to the database;
- Invoking a control layer action handler, which will likely further interact with the database.
The mobile application will generally not be able to interact with a peer agent module as this may be an entirely separate process from the User service.
Communications related to (or destined for) peer agent services are handled by one of these subsystems:
- tally
- chit
- lift
Such messages contain a property called "target" indicating the applicable subsystem. name of the applicable subsystem.
MyCHIPs is structured on the model-view-controller pattern. The database is the authoritative repository of the system state (the model). Processes in the control layer (user and agent processes) or in the mobile app or other view applications should consider their data transitory and should rely on the model for the truth..
Each object (such as a tally) in each subsystem contains a status field. The object's status (possibly combined with other key fields) will indicate the state of the object. The state is used to make key decisions about how/when to transition to a new state. This allows erronious or late messages to simply be ignored.
A subsystem may receive asynchronous requests from either a local user or a remote peer agent.
A user request is typically generated by writing a non-null value into the object's DB request field. This implies the need for action by the applicable agent process in order to accomplish the desired state change. The value in the request field typically indicates the next value intended to be written into the object's status field. The database will detect the request and generate a PostgreSQL notice to the agent process on a channel as follows:
ma_AGENT_PUBLIC_KEY
where AGENT_KEY_PUBLIC key indicates the agent address for the process responsible for traffic for the applicable user. Usually the agent process will need to communicate something to an external peer process and then issue a message back to the database indicating it is appropriate now to transition to the requested state (the new status value).
Similarly, an agent process may receive a message from a peer agent at any time. These messages also request a change in state of the targeted object. A corresponding state change command can be issued directly to the database but it is contingent upon the existing state. In this way, commands will only be considered valid if the object is still in the intended state (context) at the instant they are interpreted.
Previous to 2022, the states of subsystem objects (tallies, chits, routes, lifts) were arbitrarily named. Moving to protocol >= 1.0 they utilize a state format encoded as follows:
- phase.status.request
where "status" indicates the current status of the object record. The state might also start with an optional "phase" which is simply there to differentiate states that may be distinct, but share a common status. For example a chit might be pending in several different states, but we need to know the difference between a chit from our user and a chit to our user.
The optional "request" part similarly indicates the value of the request field in the object. Having a request component indicates this is a state that needs attention from the agent process.
States requiring action from the user should have a null request in the object, and there is no direct indication from the state code. However, the DB view should provide a boolean "action" field indicating the record is in one of those states.
Messages from one agent to another peer agent are currently sent over noise protocol. In the future, this may be generalized to use libp2p.
Messages are currently encoded as JSON structures. Future implementations might move to a more compressed binary form (or at least a form where properties are much more abbreviated.) One approach would be to begin binary messages with something other than '{';
Messages destined for another user's agent will typically have the following basic properties:
- target: What kind of object (subsystem) the message applies to. For example, tally, chit, lift.
- action: A code representing what step or function is being requested or communicated via the packet. This may well be the name of the state that is being suggested as the next step in a negotiation.
- object: The contents of the object (tally, chit, lift) itself.
- try: An integer starting at 1 indicating how many times the sender has tried to send this message.
- last: A timestamp for when the message was generated.
- to: The CHIP Address of the recipient, containing some or all of the following, as applicable:
- cuid: The CHIP User ID of the sender.
- agent: Encoded agent public key
- host: Hostname or IP number (eventually deprecated)
- port: Port to connect on (eventually deprecated)
- addr: Libp2p multiaddress (future direction)
- from: The CHIP Address of the sender, containing some or all of the following, as applicable:
- cuid: The CHIP User ID of the sender.
- agent: Encoded agent public key
- host: Hostname or IP number (eventually deprecated)
- port: Port to connect on (eventually deprecated)
- addr: Libp2p multiaddress (future direction)
Once an agent receives a message, it may trim the message to only those properties that are required before forwarding the message on to the database. Some fields not ever required by the database include: target, action, from, last, try.
Some messages from the agent to the database are intended to insert or update the content of the object (tally, chit, lift) itself. These messages may include the object property in its full JSON exchange format. In simpler cases, a message may only need to change the status of the object (for example, voiding or closing) so the whole object may not be necessary. However, at a minimum the object will have to contain any identifiers (uuid, for example) needed to uniquely locate the object in the database.
Agent actions sent to the database also include the following recipe object containing:
- context: The object must be in one of the specified states in this array before the contained state transition command will be allowed to be processed.
- update: An object describing DB fields to update. Typically this is to move the object from one state to another by updating the status field but can also include other DB fields.
- upsert: The object in the DB should be updated according to the attached object. If it doesn't exist, it should be created.
- query: Special directive for route/lift subsystems to query for upstream routes/lifts.
The recipe property may also contain such specialized flags as applicable to the specific transaction handler.
Property: target: tally
The object property for the tally is defined as follows:
- version: Version (1) of the tally format.
- revision: Count of how many times the tally was modified during negotiation
- foil: Certificate and terms applicable to the customer/employer/client.
- stock: Certificate and terms applicable to the provider/employee/vendor.
- uuid: A Universally Unique Identifier for this particular tally.
- comment: Any comments to the tally the parties have agreed to include.
- date: Date/time the tally was created (YYYY-MM-DDTHH:MM:SS.XXXZ)
- agree: A reference to a digital contract containing the general terms and conditions for the tally. Can be the document resource ID hash, or an object containing:
- host: Web address of Issuer/author guaranteed to have authoritative copy of the contract
- source: Base 64v hash of the contract
- signed:
- hash: The SHA-256 hash of the tally (absent 'signed') in a standard serialized format and converted to base64v
- foil: The digital signature of the hash by the foil holder (client)
- stock: The digital signature of the hash by the stock holder (vendor)
Tally state transition messages are as follows:
-
DB->Agent, Agent->Agent; Initiate Tally Connection; The process is responding to a tally invitation by requesting that a connection be opened to the site containing the tally and asking that the actual tally draft be sent back.
- action: ticket;
- ticket: a connection ticket (i.e. not a tally yet!) containing:
- token: connection code
- cert: CHIP certificate of the subject peer who will receive the tally
-
DB->Agent: Request Agent to Send a Tally: action: offer
The DB requests the agent to send the contained tally proposal to the requester or intended partner. -
Agent->Agent: Sending a Tally Offer: action: offer
A peer agent is sending the contained tally intending to execute on the proposed terms. -
DB->Agent: Request Agent to Refuse Tally: action: void
The DB requests the agent to tell the prospective partner of the tally "no thanks." -
Agent->Agent: Refusing a Tally Offer: action: void
A peer agent is indicating that the referenced tally has been refused by its user. -
DB->Agent: Request Agent to Revise Tally: action: draft
The DB requests the agent to tell the prospective partner that we are revising the tally. -
Agent->Agent: Revise a Tally Offer: action: draft
A peer agent is indicating that the referenced tally has been reset to draft by its user with the intent to revise it. -
DB->Agent: Request Agent to Accept Tally: action: open
The DB requests the agent to tell the prospective partner of the contained, signed tally "I accept." -
Agent->Agent: Accepting a Draft Tally Offer: action: open
A peer agent is transmitting a tally that has been accepted and signed by its user.
Property: target: chit
The object property for the chit is defined as follows:
- uuid: A Universally Unique Identifier(UUID) for this particular chit.
- tally: The UUID of the tally this chit belongs to.
- issue: The value 'stock' or 'foil' to indicate which holder issued the chit pledge (or setting).
- units: The number of milli-CHIPs on this chit. This should always be a positive number.
- date: Date/time the chit was became/becomes effective (YYYY-MM-DDTHH:MM:SS.XXXZ)
- type: The value 'tran' (transaction), 'set' (setting) or 'lift' (linear or circular)
- ref: A JSON data structure containing invoice, order, or other references material to the transaction.
For setting chits, this contains the values of the settings:
- For trading variables, use the property values explained here (target,bound,reward,clutch).
- To mark a tally for closing, specify the property 'close' as true.
- To specify a tally call date, use the property 'call'.
- To specify a tally call date, use the property 'call'.
- memo: A human-readable description or comment about the transaction.
- hash: The SHA-256 hash of the rest of the chit in a standard serialized format, rendered in base64v
- signed: The digital signature of the hash by the grantor, whether Client or Vendor
- chain: An initial consensus packet, part of the consensus protocol which rides along with the chit message for convenience
- index: The chain index computed or suggested for this chit.
- hash: The new endHash computed as a result of adding this chit.
- conf: The last confirmed chain index.
Chit state transition messages are as follows:
-
DB->Agent: Request Agent to Invoice: action: pend The DB requests the agent to send the payment request to the indicated peer.
-
DB->Agent: Request Agent to Cancel: action: void The DB requests the agent to notify the peer that the referenced payment is refused.
-
DB->Agent: Request Agent: action: good The DB requests the agent to notify the peer that the referenced payment is approved.
-
Agent->Agent: Request Payment: action: pend A peer is requesting payment from our user on the indicated tally.
-
Agent->Agent: Refuse Payment: action: void A peer is refusing payment to our user on the indicated tally.
-
Agent->Agent: Approve Payment: action: good A peer is confirming/approving payment to our user on the indicated tally.
In addition to the four main subsystems mentioned above, there is a sub-protocol for maintaining consensus between the stock and foil about the order and integrity of chits recorded on the tally (the chit chain).
This can be thought of as a set of sub-states a tally can be in while its main state is open (and/or closing). For example, a stock can be said to be consensed or not yet consensed with its foil counterpart. And we will think of the tally record itself as the initial 'chit zero' or starting block of the chain.
We can also think of consensus as an extension of the chit protocol because individual chits can either be linked into the chit chain or not yet linked. This makes it challenging to attribute a single, granular consensus state to tally. A tally can have some chits linked, others in the process of being linked and still others needing to be linked.
For purposes of negotiating consensus, we will define a substate that is applicable to individual chits. A foil will be considered in consensus when every good chit has been linked into its chain. A stock will be considered in consensus when every good chit has been linked into its chain and the resulting end hash has been confirmed with the foil.
Since attaining consensus is primarily the result of hash-chaining chits, we will utilize the chit message handling module (i.e. property target = chit). Consensus messages specify the property action = 'chain' which means "Hand this message to the consensus processing module."
The consensus state machine references the following message sub-commands:
- upd: (virtual) Stock/foil sends valid chit, accompanied by additional chaining record.
- upd: (explicit) Sender includes an index and hash and an optional list of chits. Attaching chits tells the partner to merge them into its chain. The recipient is then expected to acknowledge (ack or nak) the specified hash at the specified index (assumed to be the end of any new chits added).
- req: Sender requests a specified number of chits starting at the specified index.
- ack: Sender acknowledges the specified hash to be accurate for the specified index.
- nak: Sender disagrees with some prior assertion of the partner and instead finds the specified hash to be correct for the specified index.
The first action is directly correlated with the sending of a chit--a message already occuring within the chit subsystem. It is accomplished simply by populating the additional chain property in the applicable chit message. The latter four will encode the desired command into the property sub.
Chain messages then contain an object with some or all of the following properties:
- action: = chain
- sub: The chaining subcommand
- tally: The uuid of the tally the message pertains to
- hash: The hash at the end of the chain in base64v format
- index: The index of the last chit in the chain
- chits: An array of chits from a section of the chain
- conf: The largest index known to be confirmed by the other side
Property: target: lift
Any lift message may contain the following as applicable:
- uuid: A Universally Unique Identifier(UUID) for this particular lift.
- sequence: The sequence number (0..n) of the lift intended to be affected by this operation.
- tran: Transactional record interpreted by ChipNet
For the special case of the init action (on the lift originating site), the message will contain:
- aux:
- pub: The payor's public key
- pay: The json rendering of the payment the payor digitally signed
- auth: The payor Authorization record
- ref: Reference JSON (PO, invoice, etc)
- memo: Human-readable payment reference/note
- sign: Payor's digital signature
For the special case of the seek, exec actions (on the lift originating site), the message will contain:
- aux:
- origin:
- cuid: The CHIP User ID of the payor
- agent: Payor agent
- trans: Record of ChipNet-specific context data
- origin:
The object property includes the following properties as applicable:
- lift: A Universally Unique Identifier(UUID) for this particular lift.
- date: Date/time the lift was created
- life: The number of seconds from creation before this lift will be deemed expired.
- units: The number of milli-CHIPs on this lift.
- auth: The payor Authorization record (only for originator/terminus)
- ref: Reference JSON (PO, invoice, etc)
- memo: Human-readable payment reference/note
- sign: Payor's digital signature
Lift state transition messages are as follows: (Needs to be completed)
-
DB->Agent: ___: action: ___ The DB requests ...
-
Agent->Agent: ___: action: ___ A downstream peer agent is asking us...
In addition to real-time, dynamic messages, certain objects may be transmitted over alternate media or in other ways. For example, a site may issue a user a connection ticket containing a one-time token and other connection information. Users can issue tally invitations to each other. And users may have to export keys or other data and then re-import that data on another device or platform.
In these various instances, data may often be represented as either a JSON record or as a URI. Data may also be represented as text or as a QR code. Ideally a MyCHIPs mobile app should be able to scan a QR code containing any applicable record and know what to do with it.
Furthermore, any scanner should be able to scan a QR code that contains a deep link URI. If the user does not have an applicable app installed, he will be directed to the mychips.org (or applicable) website where further instructions will be available about how to proceed.
Each function will involve a particular keyword as follows:
- ticket: Connection ticket
- invite: Invitation to tally
- pay: Invoice for payment
- conkey: Connection key
- signkey: Signing key
- user: Import/export user record
- tally: Import/export tally record
The associated data formats are as follows:
- User Connection Ticket: Issued by Chip Service Provider
Web URI:'https://HOST:HTTP_PORT/user.html?&port=WS_PORT&token=TOKEN$%user=USER'
LINK: 'https://mychips.org/ticket?host=HOST%port=PORT%user=USER%token=TOKEN'
JSON:
{ticket: {
host: HOST,
port: WS_PORT,
user: USER,
token: TOKEN,
expires: EXPIRATION_DATE
}}
- Tally Invitation: Issued from one user to another
LINK: 'https://mychips.org/invite?token=TOKEN&chad={cuid:w,agent:x,host:y,port:z}'
JSON:
{invite: {
token: TOKEN,
chad: {
cuid: USER_CHIP_ID,
agent: USERS_AGENT_PUBLIC_KEY,
host: AGENT_HOST_ADDRESS,
port: AGENT_HOST_PORT,
}
}}
- Invoice: Payment request or invitation
LINK: 'https://mychips.org/pay?chad={cuid:w,agent:x,host:y,port:z}...'
LINK: 'https://mychips.org/pay?base64EncodedJSON'
JSON:
{pay: {
chad: {
cuid: USER_CHIP_ID,
agent: USERS_AGENT_PUBLIC_KEY
}
units: 123456,
date: <invoice date>,
ref: {arbitrary-payee-specific-json-data},
memo: 'Human-readable comment',
hint: [array of agents who know me]
}}
- Connection Key(s): For backup/restore and sharing to another device
Unencrypted (not recommended):
JSON
{conkey: {
kty: "EC", //Example data
crv: "P-256",
x: "MKBCTNIcKUSDii11ySs3526iDZ8AiTo7Tu6KPAqv7D4",
y: "4Etl6SRW2YiLUrN5vfvVHuhp7x8PxltmWWlbbM4IFyM"
}}
Encrypted (standard):
LINK: 'https://mychips.org/signkey?s=...,i=...,d=...'
LINK: 'https://mychips.org/signkey?base64EncodedJSON'
JSON:
{conkey: {
s: SALT,
i: INITIALIZATION_VECTOR,
d: ENCRYPTED_DATA (should decrypt to serialized JWK key as above)
}}
- Signing Key(s): For backup/restore and sharing to another device
Unencrypted (not recommended):
JSON
{signkey: {
// jwk key data properties same as shown for conkey //
}}
Encrypted (standard):
LINK: 'https://mychips.org/signkey?s=...,i=...,d=...'
LINK: 'https://mychips.org/signkey?base64EncodedJSON'
JSON (encrypted):
{signkey: {
// Encrypted key data properties same as shown for conkey //
}}
- User Record: For backup/restore and sharing to another provider
JSON:
{user: {
// See data example in test/mocha/user.json
}}
- Tally Record: For backup/restore and presentation to another user
JSON:
{tally: {
// See data example in test/mocha/tally.json
}}
A MyCHIPs server may store tallies, chits and other objects in any format that makes the most sense. But when communicating with other compliant servers, it must send the tally in these standard formats.
Any included date property should be formatted according to ISO 8601, according to UTC time zone, and showing exactly 3 digits of fractional seconds as follows:
2023-12-04T16:50:34.707Z
This will assure that digital hashes and signatures can be produced in a predictable way. Null values should also be omitted from the object.
In order to sign a record, the signature property is first removed then, the rest of the record is serialized with its properties in alphabetical order (compatible with this library, and then the resulting string is hashed and signed using the user's signing key.
Examples of this can be found in the test library.
tally: {
version: 1,
uuid: "9e61301c-86aa-5835-9ea1-25adf13eaf3c",
date: <Begin date of contract>,
memo: <Additional Comments>,
stock: {
cert: <CHIP Certificate>,
terms: <Credit Terms>
}
foil: {
cert: <CHIP Certificate>,
terms: <Credit Terms>
}
agree: <Hash of selected agreement>,
sign: {
foil: <Foil Holder Signature>,
stock: <Stock Holder Signature>,
},
}
The tally contract ("agree" property) can also contain a json object as follows (for cases where the host is not mychips.org):
agree: {
host: "example.com",
source: "BqQZqh3xUtye3JnAKhwdMrCMHem3vX67gV3UevGBr4pE"
}
chit: {
by: <issuer>, //stock or foil
date: <creation date/time>,
type: "tran", //or "lift"
uuid: "2d5d4167-dcdf-5743-861c-e6ae1e62bbb8",
tally: "9e61301c-86aa-5835-9ea1-25adf13eaf3c",
units: 432123, //milli-CHIPs
ref: <External invoice number or other reference or comment>,
memo: <Human readable comments>,
sign: <Pledgor's signature>
}
For linear lifts, an initial payment lift record is signed by the paying user as follows:
lift: {
uuid: "9e61301c-86aa-5835-abcd-25adf13eaf3c",
find: {
cuid: <optional CHIP User ID of payment recipient>
agent: <recipient Agent address>,
host: <optional agent host address>,
port: <optional agent host port>
}.
date: <Lift initiation date/time>,
units: <value of lift>,
memo: <Human-readable text message from payor>,
ref: <JSON invoice or transaction reference>,
}
Agents along the lift pathway will also sign a record as follows:
relay: {
uuid: "9e61301c-86aa-5835-abcd-25adf13eaf3c",
date: <Lift initiation date/time>,
life: <How manys seconds before the lift expires>,
play: <Path, routing, margin, payment information, encoded for agents' eyes only>,
}
The JSON records above all feature an outer wrapper of sorts where in the top level, there is a property naming the type of record to follow.
Where applicable, multiple such records can follow the named property by simply using a JSON array rather than a JSON object. (See example in test/mocha/users.json.)
Also, in contexts where the type of the record is already clearly know, the top-level named property can be omitted and the inner record can stand alone.