eosswedenorg/thalos-docs
1
0
Fork 0
mirror of https://github.com/eosswedenorg/thalos-docs synced 2026-06-16 04:34:55 +02:00
Code Releases Activity

Merge branch 'next'

Browse source
This commit is contained in:
Henrik Hautakoski 2024-08-29 15:14:36 +02:00
commit 4f73831d3d
7 changed files with 109 additions and 60 deletions
Download patch file Download diff file Expand all files Collapse all files

103
docs/api/messages.md
View file

@ -14,75 +14,94 @@ Heartbeat messages are posted to the heartbeat channel periodically.
| Field | Datatype | Description |
| -------------------------- | -------- | ------------------------------------------- |
| blocknum | int | Current block number |
| head_blocknum | int | Head block number |
| last_irreversible_blocknum | int | block number of the last irreversible block |
| blocknum | `int` | Current block number |
| head_blocknum | `int` | Head block number |
| last_irreversible_blocknum | `int` | block number of the last irreversible block |
### Transaction
### ActionTrace
| Field | Datatype | Description |
| -------------- | ----------------- | ----------------------------------------------------------------- |
| tx_id | string | Transaction ID |
| blocknum | int | Block number where this action trace (and transaction) belongs to |
| blocktimestamp | time | Block timestamp |
| receipt | ActionReceipt | Action receipt |
| receiver | string | Receiver account |
| first_receiver | bool | True if receiver is the first account to get notified |
| contract | string | Contract account |
| action | string | What action was executed on the contract |
| data | any | Contract specific data (decoded using the contracts abi) |
| authorization | PermissionLevel[] | Authorization |
| Field | Datatype | Description |
| -------------- | --------------------------------------- | ----------------------------------------------------------------- |
| tx_id | `string` | Transaction ID |
| blocknum | `int` | Block number where this action trace (and transaction) belongs to |
| blocktimestamp | `time` | Block timestamp |
| receipt | [`ActionReceipt`](#actionreceipt) | Action receipt |
| receiver | `string` | Receiver account |
| first_receiver | `bool` | True if receiver is the first account to get notified |
| contract | `string` | Contract account |
| action | `string` | What action was executed on the contract |
| data | `any` | Contract specific data (decoded using the contracts abi) |
| authorization | [`PermissionLevel[]`](#permissionlevel) | Authorization |
### ActionReceipt
| Field | Datatype | Description |
| --------------- | --------------------- | ------------------ |
| receiver | string | Actor account name |
| act_digest | string | Action digest |
| global_sequence | int | Global sequence |
| recv_sequence | int | Receive sequence |
| auth_sequence | AccountAuthSequence[] | Auth sequence |
| code_sequence | int | Code sequence |
| abi_sequence | int | ABI sequence |
| Field | Datatype | Description |
| --------------- | ----------------------------------------------- | ------------------ |
| receiver | `string` | Actor account name |
| act_digest | `string` | Action digest |
| global_sequence | `int` | Global sequence |
| recv_sequence | `int` | Receive sequence |
| auth_sequence | [`AccountAuthSequence[]`](#accountauthsequence) | Auth sequence |
| code_sequence | `int` | Code sequence |
| abi_sequence | `int` | ABI sequence |
### PermisssionLevel
### PermissionLevel
| Field | Datatype | Description |
| ---------- | -------- | -------------------------------- |
| actor | string | Actor account name |
| permission | string | Permission (for example: active) |
| actor | `string` | Actor account name |
| permission | `string` | Permission (for example: active) |
### AccountAuthSequence
| Field | Datatype | Description |
| -------- | -------- | ------------ |
| account | string | Account name |
| sequence | int | Sequence |
| account | `string` | Account name |
| sequence | `int` | Sequence |
### RollbackMessage
| Field | Datatype | Description |
| --------- | -------- | ------------------------------------ |
| new_block | int | The current block number |
| old_block | int | Last block number that was received. |
| new_block | `int` | The current block number |
| old_block | `int` | Last block number that was received. |
### TableDelta
| Field | Datatype | Description |
| -------------- | --------------- | ----------------- |
| blocknum | int | Block number |
| blocktimestamp | time | Block timestamp |
| name | string | Table name |
| rows | TableDeltaRow[] | Rows in the delta |
| Field | Datatype | Description |
| -------------- | ----------------------------------- | ----------------- |
| blocknum | `int` | Block number |
| blocktimestamp | `time` | Block timestamp |
| name | `string` | Table name |
| rows | [`TableDeltaRow[]`](#tabledeltarow) | Rows in the delta |
### TableDeltaRow
| Field | Datatype | Description |
| -------- | ------------- | --------------------------------- |
| present | bool | - |
| data | any | Decoded data |
| raw_data | base64_string | Raw data in base64 encoded string |
| Field | Datatype | Description |
| -------- | ---------------------------------------------------------- | --------------------------------- |
| present | `bool` | - |
| data | [`TableDeltaRowContract`](#tabledeltarowcontract) \| `any` | Decoded data |
| raw_data | `base64_string` | Raw data in base64 encoded string |
### TableDeltaRowContract
These messages are only relevant when listening to [Table deltas](redis-channels#table-delta) channel when name is `contract_row`.
| Field | Datatyp | Description |
| ----------- | ------------------------ | ---------------------------------------------------------- |
| table | `string` | table name |
| scope | `string` | table scope |
| primary_key | `string` \| `int` | Primary ID for the row. |
| code | `string` | Account name where that has the contract code |
| payer | `string` | Account name that payed for the ram that this row consumes |
| value | `map` \| `base64_string` | Actual data in the table |
Thalos can decode contract row data in `vaule` using the contracts abi.
If Thalos could not decode the data then `value` will contain a `base64_string` of the raw value instead.

12
docs/architecture/benchmark.md
View file

@ -31,7 +31,7 @@ Consequently, it is important to consider that the performance on these hardware
![Live Data msg per sec](./live.png)
::: details
\* Redis discards messages for clients because they cannot be processed in time. resulting in dataloss.
\* Redis discards messages for clients because they cannot be processed in time. Resulting in data loss.
:::
@ -43,7 +43,8 @@ Raw data:
These results are quite interesting. High and mid-end hardware are about equal even for 100 clients. And the low-end hardware is not so far of. capable of handling atleast 10 clients.
Note that one block contains alot of transactions and actions resulting in alot more then one redis message per block. therefore even if there is just 2 blocks per second. There is alot more redis messages sent out.
Note that one block contains a lot of transactions and actions resulting in a lot more then one Redis message per block.
Therefore even if there is just 2 blocks per second. There is a lot more Redis messages sent out.
## Replay
@ -66,6 +67,9 @@ it is just as 10xlow, not usable.
## Conclusion
By looking at the live data graphs they process roughly the same amount of messages per second. That is because the bottleneck is the blockchain
itself. If we look at the replay data, redis clients can't keep up as number of clients increases. however, it is still pretty fast. consider that all 10x clients performs roughly equal.
itself. If we look at the replay data, redis clients can't keep up as number of clients increases.
However, it is still pretty fast. Consider that all 10x clients performs roughly equal.
Also the fact that the benchmark tests fetchees **all** messages. That is pretty unrealistic as applications should in 99% of the cases only care about actions on a subset of contracts. Only case where you would want all actions is if you are building some sort of blockexplorer.
Also the fact that the benchmark tests fetches **all** messages. That is pretty unrealistic as applications
should in 99% of the cases only care about actions on a subset of contracts.
Only case where you would want all actions is if you are building some sort of blockexplorer.

7
docs/architecture/index.md
View file

@ -37,7 +37,10 @@ However, these advantages can also be considered drawbacks in certain situations
- **Immediate message consumption**: Pub/Sub requires subscribers to consume messages immediately upon publication. If a client is unable to read a message promptly, it is discarded.
In contrast, Streams in Redis provide built-in message persistence. Messages are stored as entries in the stream and can be consumed at any time, even if they were published before the subscriber connected.
In contrast, Streams in Redis provide built-in message persistence.
Messages are stored as entries in the stream and can be consumed at any time, even if they were published before the subscriber connected.
This feature addresses the drawbacks of immediate message consumption and message loss associated with Pub/Sub.
Currently, Streams are not implemented in Thalos. However, there are plans to consider implementing Streams in the future, based on the demand for such functionality. It's important to note that Streams introduce additional complexity to the client implementation.
Currently, Streams are not implemented in Thalos.
However, there are plans to consider implementing Streams in the future, based on the demand for such functionality.
It's important to note that Streams introduce additional complexity to the client implementation.

22
docs/configuration.md
View file

@ -69,6 +69,28 @@ Thalos will treat `ship.blacklist` as a whitelist.
`redis.prefix` ([`string`](#string)) - Key prefix, this will be prepended to all channels that thalos is using to avoid name collision
### Cache
These settings control the cache used by Thalos to cache abi definitions and internal state.
`cache.storage` ([`string`](#string) default: `redis`) - Cache storage to use.
Avaible values are: `memory`, `redis`
`cache.options` (`map`) - Options for the cache storage, see section for each cache storage.
#### Memory
No configuration.
#### Redis
`cache.options.stats` ([`boolean`](#string) default: `false`) - True if statistics should be collected
`cache.options.size` ([`number`](#string) default: `1000`) - How many items to store in process local memory for faster lookup of popular items
`cache.options.ttl` ([`number`](#string) default: `10`) - How long (in minutes) each item should be kept in process local memory before being discared (and has to be fetched from redis again.)
### Logging
This block configures how thalos will log information.

6
docs/installation/index.md
View file

@ -19,7 +19,7 @@ There are several ways to install thalos, via package manager, downloading a pre
You can get the latest archive package [here](https://github.com/eosswedenorg/thalos/releases/latest)
Simply download using your webbrowser or via curl:
Simply download using your web browser or via curl:
```shell
curl -Ls https://github.com/eosswedenorg/thalos/releases/download/<version>/thalos-server-<version>-linux-amd64.tar.gz | tar -z --one-top-level=thalos -xvf -
@ -31,7 +31,7 @@ Using curl command above, the files are extracted into the `thalos` subdirectory
### Compiling from source
You will need golang version `1.20` or later to compile the source.
You will need golang version `1.21` or later to compile the source.
#### Compile using make
@ -51,4 +51,4 @@ After building the binary you can install it along with basic config file and st
```shell
./install.sh /path/to/your/directory/of/choice
```
```

5
docs/redis/security/acl.md
View file

@ -12,7 +12,8 @@ The special account called `default` serves as the default account for unauthori
configured with a password. Connections can authenticate against this account without specifying a username.
Thalos utilizes this account as the default user account.
Additionally, it is advisable to restrict the Thalos server account as an added precaution against any unauthorized actions it may inadvertently perform, although such occurrences are highly unlikely.
Additionally, it is advisable to restrict the Thalos server account as an added precaution against any
unauthorized actions it may inadvertently perform, although such occurrences are highly unlikely.
The ACL in thalos is simple and uses 2 accounts:
@ -112,4 +113,4 @@ user thalos-client on >client_password resetchannels &ship::* +@connection +subs
## Useful links
* [Config File Example](https://redis.io/docs/management/config-file)
* [Official ACL Documentation](https://redis.io/docs/management/security/acl)
* [Official ACL Documentation](https://redis.io/docs/management/security/acl)

14
docs/redis/security/index.md
View file

@ -1,15 +1,15 @@
# Securing redis
This documentation primarily focuses on setups where Redis is exposed to the internet
or an internal network where there is not a complete control over the clients.
This documentation primarily focuses on setups where Redis is exposed to the internet or an internal
network where there is not complete control over the clients.
For example, you may want to grant access to your Thalos instance to a friend.
While trusting your friend is reasonable, it is essential to consider potential future scenarios where
trust may no longer exist or their server could be compromised.
If you intend to run Thalos for internal use only, such as having internal applications
that are relying on a blockchain stream, it is perfectly acceptable to skip these
steps if you have complete control over all involved servers and do not expose the instance over a public IP.
If you intend to run Thalos for internal use only, such as having internal applications that are relying on a blockchain stream,
it is perfectly acceptable to skip these steps if you have complete control over all involved
servers and do not expose the instance over a public IP.
## Isolating redis
@ -42,8 +42,8 @@ bind * -::* # like the default, all available interfaces
## Firewall
Make sure you setup your firewall rules correctly. only allowing the ip's you trust to access the redis port.
This is out of scope of this documentation. consult your operating system or router manuals.
Make sure you setup your firewall rules correctly. Only allowing the IP addresses you trust to access the Redis port.
This is out of scope of this documentation. Consult your operating system or router manuals.
## Useful links