diff --git a/docs/api/messages.md b/docs/api/messages.md index 0fce04e..72c982a 100644 --- a/docs/api/messages.md +++ b/docs/api/messages.md @@ -14,75 +14,94 @@ Heartbeat messages are posted to the heartbeat channel periodically. | Field | Datatype | Description | | -------------------------- | -------- | ------------------------------------------- | -| blocknum | int | Current block number | -| head_blocknum | int | Head block number | -| last_irreversible_blocknum | int | block number of the last irreversible block | +| blocknum | `int` | Current block number | +| head_blocknum | `int` | Head block number | +| last_irreversible_blocknum | `int` | block number of the last irreversible block | ### Transaction ### ActionTrace -| Field | Datatype | Description | -| -------------- | ----------------- | ----------------------------------------------------------------- | -| tx_id | string | Transaction ID | -| blocknum | int | Block number where this action trace (and transaction) belongs to | -| blocktimestamp | time | Block timestamp | -| receipt | ActionReceipt | Action receipt | -| receiver | string | Receiver account | -| first_receiver | bool | True if receiver is the first account to get notified | -| contract | string | Contract account | -| action | string | What action was executed on the contract | -| data | any | Contract specific data (decoded using the contracts abi) | -| authorization | PermissionLevel[] | Authorization | +| Field | Datatype | Description | +| -------------- | --------------------------------------- | ----------------------------------------------------------------- | +| tx_id | `string` | Transaction ID | +| blocknum | `int` | Block number where this action trace (and transaction) belongs to | +| blocktimestamp | `time` | Block timestamp | +| receipt | [`ActionReceipt`](#actionreceipt) | Action receipt | +| receiver | `string` | Receiver account | +| first_receiver | `bool` | True if receiver is the first account to get notified | +| contract | `string` | Contract account | +| action | `string` | What action was executed on the contract | +| data | `any` | Contract specific data (decoded using the contracts abi) | +| authorization | [`PermissionLevel[]`](#permissionlevel) | Authorization | ### ActionReceipt -| Field | Datatype | Description | -| --------------- | --------------------- | ------------------ | -| receiver | string | Actor account name | -| act_digest | string | Action digest | -| global_sequence | int | Global sequence | -| recv_sequence | int | Receive sequence | -| auth_sequence | AccountAuthSequence[] | Auth sequence | -| code_sequence | int | Code sequence | -| abi_sequence | int | ABI sequence | +| Field | Datatype | Description | +| --------------- | ----------------------------------------------- | ------------------ | +| receiver | `string` | Actor account name | +| act_digest | `string` | Action digest | +| global_sequence | `int` | Global sequence | +| recv_sequence | `int` | Receive sequence | +| auth_sequence | [`AccountAuthSequence[]`](#accountauthsequence) | Auth sequence | +| code_sequence | `int` | Code sequence | +| abi_sequence | `int` | ABI sequence | -### PermisssionLevel +### PermissionLevel | Field | Datatype | Description | | ---------- | -------- | -------------------------------- | -| actor | string | Actor account name | -| permission | string | Permission (for example: active) | +| actor | `string` | Actor account name | +| permission | `string` | Permission (for example: active) | ### AccountAuthSequence | Field | Datatype | Description | | -------- | -------- | ------------ | -| account | string | Account name | -| sequence | int | Sequence | +| account | `string` | Account name | +| sequence | `int` | Sequence | ### RollbackMessage | Field | Datatype | Description | | --------- | -------- | ------------------------------------ | -| new_block | int | The current block number | -| old_block | int | Last block number that was received. | +| new_block | `int` | The current block number | +| old_block | `int` | Last block number that was received. | ### TableDelta -| Field | Datatype | Description | -| -------------- | --------------- | ----------------- | -| blocknum | int | Block number | -| blocktimestamp | time | Block timestamp | -| name | string | Table name | -| rows | TableDeltaRow[] | Rows in the delta | +| Field | Datatype | Description | +| -------------- | ----------------------------------- | ----------------- | +| blocknum | `int` | Block number | +| blocktimestamp | `time` | Block timestamp | +| name | `string` | Table name | +| rows | [`TableDeltaRow[]`](#tabledeltarow) | Rows in the delta | ### TableDeltaRow -| Field | Datatype | Description | -| -------- | ------------- | --------------------------------- | -| present | bool | - | -| data | any | Decoded data | -| raw_data | base64_string | Raw data in base64 encoded string | +| Field | Datatype | Description | +| -------- | ---------------------------------------------------------- | --------------------------------- | +| present | `bool` | - | +| data | [`TableDeltaRowContract`](#tabledeltarowcontract) \| `any` | Decoded data | +| raw_data | `base64_string` | Raw data in base64 encoded string | + +### TableDeltaRowContract + +These messages are only relevant when listening to [Table deltas](redis-channels#table-delta) channel when name is `contract_row`. + + + +| Field | Datatyp | Description | +| ----------- | ------------------------ | ---------------------------------------------------------- | +| table | `string` | table name | +| scope | `string` | table scope | +| primary_key | `string` \| `int` | Primary ID for the row. | +| code | `string` | Account name where that has the contract code | +| payer | `string` | Account name that payed for the ram that this row consumes | +| value | `map` \| `base64_string` | Actual data in the table | + + +Thalos can decode contract row data in `vaule` using the contracts abi. +If Thalos could not decode the data then `value` will contain a `base64_string` of the raw value instead. diff --git a/docs/architecture/benchmark.md b/docs/architecture/benchmark.md index e0a28be..0b93b01 100644 --- a/docs/architecture/benchmark.md +++ b/docs/architecture/benchmark.md @@ -31,7 +31,7 @@ Consequently, it is important to consider that the performance on these hardware ![Live Data msg per sec](./live.png) ::: details -\* Redis discards messages for clients because they cannot be processed in time. resulting in dataloss. +\* Redis discards messages for clients because they cannot be processed in time. Resulting in data loss. ::: @@ -43,7 +43,8 @@ Raw data: These results are quite interesting. High and mid-end hardware are about equal even for 100 clients. And the low-end hardware is not so far of. capable of handling atleast 10 clients. -Note that one block contains alot of transactions and actions resulting in alot more then one redis message per block. therefore even if there is just 2 blocks per second. There is alot more redis messages sent out. +Note that one block contains a lot of transactions and actions resulting in a lot more then one Redis message per block. +Therefore even if there is just 2 blocks per second. There is a lot more Redis messages sent out. ## Replay @@ -66,6 +67,9 @@ it is just as 10xlow, not usable. ## Conclusion By looking at the live data graphs they process roughly the same amount of messages per second. That is because the bottleneck is the blockchain -itself. If we look at the replay data, redis clients can't keep up as number of clients increases. however, it is still pretty fast. consider that all 10x clients performs roughly equal. +itself. If we look at the replay data, redis clients can't keep up as number of clients increases. +However, it is still pretty fast. Consider that all 10x clients performs roughly equal. -Also the fact that the benchmark tests fetchees **all** messages. That is pretty unrealistic as applications should in 99% of the cases only care about actions on a subset of contracts. Only case where you would want all actions is if you are building some sort of blockexplorer. \ No newline at end of file +Also the fact that the benchmark tests fetches **all** messages. That is pretty unrealistic as applications +should in 99% of the cases only care about actions on a subset of contracts. +Only case where you would want all actions is if you are building some sort of blockexplorer. diff --git a/docs/architecture/index.md b/docs/architecture/index.md index ff4b6eb..382afa1 100644 --- a/docs/architecture/index.md +++ b/docs/architecture/index.md @@ -37,7 +37,10 @@ However, these advantages can also be considered drawbacks in certain situations - **Immediate message consumption**: Pub/Sub requires subscribers to consume messages immediately upon publication. If a client is unable to read a message promptly, it is discarded. -In contrast, Streams in Redis provide built-in message persistence. Messages are stored as entries in the stream and can be consumed at any time, even if they were published before the subscriber connected. +In contrast, Streams in Redis provide built-in message persistence. +Messages are stored as entries in the stream and can be consumed at any time, even if they were published before the subscriber connected. This feature addresses the drawbacks of immediate message consumption and message loss associated with Pub/Sub. -Currently, Streams are not implemented in Thalos. However, there are plans to consider implementing Streams in the future, based on the demand for such functionality. It's important to note that Streams introduce additional complexity to the client implementation. \ No newline at end of file +Currently, Streams are not implemented in Thalos. +However, there are plans to consider implementing Streams in the future, based on the demand for such functionality. +It's important to note that Streams introduce additional complexity to the client implementation. diff --git a/docs/configuration.md b/docs/configuration.md index 27bf1d9..2170f14 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -69,6 +69,28 @@ Thalos will treat `ship.blacklist` as a whitelist. `redis.prefix` ([`string`](#string)) - Key prefix, this will be prepended to all channels that thalos is using to avoid name collision +### Cache + +These settings control the cache used by Thalos to cache abi definitions and internal state. + +`cache.storage` ([`string`](#string) default: `redis`) - Cache storage to use. + +Avaible values are: `memory`, `redis` + +`cache.options` (`map`) - Options for the cache storage, see section for each cache storage. + +#### Memory + +No configuration. + +#### Redis + +`cache.options.stats` ([`boolean`](#string) default: `false`) - True if statistics should be collected + +`cache.options.size` ([`number`](#string) default: `1000`) - How many items to store in process local memory for faster lookup of popular items + +`cache.options.ttl` ([`number`](#string) default: `10`) - How long (in minutes) each item should be kept in process local memory before being discared (and has to be fetched from redis again.) + ### Logging This block configures how thalos will log information. diff --git a/docs/installation/index.md b/docs/installation/index.md index 75c5c0c..51fa8ec 100644 --- a/docs/installation/index.md +++ b/docs/installation/index.md @@ -19,7 +19,7 @@ There are several ways to install thalos, via package manager, downloading a pre You can get the latest archive package [here](https://github.com/eosswedenorg/thalos/releases/latest) -Simply download using your webbrowser or via curl: +Simply download using your web browser or via curl: ```shell curl -Ls https://github.com/eosswedenorg/thalos/releases/download//thalos-server--linux-amd64.tar.gz | tar -z --one-top-level=thalos -xvf - @@ -31,7 +31,7 @@ Using curl command above, the files are extracted into the `thalos` subdirectory ### Compiling from source -You will need golang version `1.20` or later to compile the source. +You will need golang version `1.21` or later to compile the source. #### Compile using make @@ -51,4 +51,4 @@ After building the binary you can install it along with basic config file and st ```shell ./install.sh /path/to/your/directory/of/choice -``` \ No newline at end of file +``` diff --git a/docs/redis/security/acl.md b/docs/redis/security/acl.md index 88b44be..a54a6d7 100644 --- a/docs/redis/security/acl.md +++ b/docs/redis/security/acl.md @@ -12,7 +12,8 @@ The special account called `default` serves as the default account for unauthori configured with a password. Connections can authenticate against this account without specifying a username. Thalos utilizes this account as the default user account. -Additionally, it is advisable to restrict the Thalos server account as an added precaution against any unauthorized actions it may inadvertently perform, although such occurrences are highly unlikely. +Additionally, it is advisable to restrict the Thalos server account as an added precaution against any +unauthorized actions it may inadvertently perform, although such occurrences are highly unlikely. The ACL in thalos is simple and uses 2 accounts: @@ -112,4 +113,4 @@ user thalos-client on >client_password resetchannels &ship::* +@connection +subs ## Useful links * [Config File Example](https://redis.io/docs/management/config-file) -* [Official ACL Documentation](https://redis.io/docs/management/security/acl) \ No newline at end of file +* [Official ACL Documentation](https://redis.io/docs/management/security/acl) diff --git a/docs/redis/security/index.md b/docs/redis/security/index.md index e4194fa..b15fd19 100644 --- a/docs/redis/security/index.md +++ b/docs/redis/security/index.md @@ -1,15 +1,15 @@ # Securing redis -This documentation primarily focuses on setups where Redis is exposed to the internet -or an internal network where there is not a complete control over the clients. +This documentation primarily focuses on setups where Redis is exposed to the internet or an internal +network where there is not complete control over the clients. For example, you may want to grant access to your Thalos instance to a friend. While trusting your friend is reasonable, it is essential to consider potential future scenarios where trust may no longer exist or their server could be compromised. -If you intend to run Thalos for internal use only, such as having internal applications -that are relying on a blockchain stream, it is perfectly acceptable to skip these -steps if you have complete control over all involved servers and do not expose the instance over a public IP. +If you intend to run Thalos for internal use only, such as having internal applications that are relying on a blockchain stream, +it is perfectly acceptable to skip these steps if you have complete control over all involved +servers and do not expose the instance over a public IP. ## Isolating redis @@ -42,8 +42,8 @@ bind * -::* # like the default, all available interfaces ## Firewall -Make sure you setup your firewall rules correctly. only allowing the ip's you trust to access the redis port. -This is out of scope of this documentation. consult your operating system or router manuals. +Make sure you setup your firewall rules correctly. Only allowing the IP addresses you trust to access the Redis port. +This is out of scope of this documentation. Consult your operating system or router manuals. ## Useful links