v1.1.20

• Optimized start of output stream in developer mode when start block is in reversible segment and output module does not have any stores in its dependencies.
• Fixed bug where the first streamable block of a chain was not processed correctly when the start block was set to the default zero value.

v1.1.19

Changed

• Codegen: Now generates separate substreams.{target}.yaml files for sql, clickhouse and graphql sink targets.

Added

• Codegen: Added support for clickhouse in schema.sql

Fixed

• Fixed metrics for time spent in eth_calls within modules stats (server and GUI)
• Fixed undo json message in 'run' command
• Fixed stream ending immediately in dev mode when start/end blocks are both 0.
• Sink-serve: fix missing output details on docker-compose apply errors
• Codegen: Fixed pluralized entity created for db_out and graph_out

v1.1.18

Fixed

• Fixed a regression where start block was not resolved correctly when it was in the reversible segment of the chain, causing the substreams to reprocess a segment in tier 2 instead of linearly in tier 1.

v1.1.17

Fixed

• Missing decrement on metrics substreams_active_requests

v1.1.16

Added

• substreams_active_requests and substreams_counter metrics to substreams-tier1

Changed

• evt_block_time in ms to timestamp in lib.rs, proto definition and schema.sql

v1.1.15

Highlights

• This release brings the substreams init command out of alpha! You can quickly generate a Substreams from an Ethereum ABI:
  

• New Alpha feature: deploy your Substreams Sink as a deployable unit to a local docker environment!
  

• See those two new features in action in this tutorial

Added

• Sink configs can now use protobuf annotations (aka Field Options) to determine how the field will be interpreted in
  substreams.yaml:

  • load_from_file will put the content of the file directly in the field (string and bytes contents are supported).
  • zip_from_folder will create a zip archive and put its content in the field (field type must be bytes).

  Example protobuf definition:

  import "sf/substreams/v1/options.proto";
  
  message HostedPostgresDatabase {
    bytes schema = 1 [ (sf.substreams.v1.options).load_from_file = true ];
    bytes extra_config_files = 2 [ (sf.substreams.v1.options).zip_from_folder = true ];
  }
  

  Example manifest file:

  [...]
  network: mainnet
  
  sink:
    module: main:db_out
    type: sf.substreams.sink.sql.v1.Service
    config:
      schema: "./schema.sql"
      wire_protocol_access: true
      postgraphile_frontend:
        enabled: true
      pgweb_frontend:
        enabled: true

• substreams info command now properly displays the content of sink configs, optionally writing the fields that were bundled from files to disk with --output-sinkconfig-files-path=</some/path>

Changed

• substreams alpha init renamed to substreams init. It now includes db_out module and schema.sql to support the substreams-sql-sink directly.

• The override feature has been overhauled. Users may now override an existing substreams by pointing to an override file in run or gui command.
  This override manifest will have a deriveFrom field which points to the original substreams which is to be overriden.
  This is useful to port a substreams to one network to another.
  Example of an override manifest:

  deriveFrom: path/to/mainnet-substreams.spkg #this can also be a remote url
  
  package:
    name: "polygon-substreams"
    version: "100.0.0"
  
  network: polygon
  
  initialBlocks:
    module1: 17500000
  params:
    module1: "address=2a75ca72679cf1299936d6104d825c9654489058"
  

• The substreams run and substreams gui commands now determine the endpoint from the 'network' field in the manifest if no value is passed in the --substreams-endpoint flag.

• The endpoint for each network can be set by using an environment variable SUBSTREAMS_ENDPOINTS_CONFIG_<network_name>, ex: SUBSTREAMS_ENDPOINTS_CONFIG_MAINNET=my-endpoint:443

• The substreams alpha init has been moved to substreams init

Fixed

• fixed the substreams gui command to correctly compute the stop-block when given a relative value (ex: '-t +10')

v1.1.14

Bug fixes

• Fixed (bumped) substreams protobuf definitions that get embedded in spkg to match the new progress messages from v1.1.12.
• Regression fix: fixed a bug where negative start blocks would not be resolved correctly when using substreams run
  or substreams gui.
• In the request plan, the process previously panicked when errors related to block number validation occurred. Now the
  error will be returned to the client.

v1.1.13

Bug fixes

• If the initial block or start block is less than the first block in the chain, the substreams will now start from the
  first block in the chain. Previously, setting the initial block to a block before the first block in the chain would
  cause the substreams to hang.
• Fixed a bug where the substreams would fail if the start block was set to a future block. The substreams will now wait
  for the block to be produced before starting.

v1.1.12

Highlights

• Complete redesign of the progress messages:
  • Tier2 internal stats are aggregated on Tier1 and sent out every 500ms (no more bursts)
  • No need to collect events on client: a single message now represents the current state
  • Message now includes list of running jobs and information about execution stages
  • Performance metrics has been added to show which modules are executing slowly and where the time is spent (eth calls, store operations, etc.)

Upgrading client and server

Important

The client and servers will both need to be upgraded at the same time for the new progress messages to be parsed:

• The new Substreams servers will NOT send the old modules field as part of its progress message, only the new running_jobs, modules_stats, stages.
• The new Substreams clients will NOT be able to decode the old progress information when connecting to older servers.

However, the actual data (and cursor) will work correctly between versions. Only incompatible progress information will be ignored.

CLI

Changed

• Bumped substreams and substreams-ethereum to latest in substreams alpha init.

• Improved error message when <module_name> is not received, previously this would lead to weird error message, now, if the input is likely a manifest, the error message will be super clear.

Fixed

• Fixed compilation errors when tracking some contracts when using substreams alpha init.

Added

• substreams info now takes an optional second parameter <output-module> to show how the substreams modules can be divided into stages
• Pack command: added -c flag to allow overriding of certain substreams.yaml values by passing in the path of a yaml file.
  example yaml contents:

  package:
    name: my_custom_package_name 
  
  network: arbitrum-one
  initialBlocks:
    module_name_1: 123123123
  params:
    mod1: "custom_parameter"

Backend

Removed

• Removed Config.RequestStats, stats are now always enabled.

v1.1.11

Fixes

• Added metering of live blocks