Module sources

This module defines four traits Source, SourcePartition, PartitionParser, and Produce to define a source. This module also contains source implementations for various databases.

Modules

• bigquery
  Source implementation for Google BigQuery
• csv
  Source implementation for CSV files.
• dummy
  A dummy source that generates different values based on an internal counter. This source is for test purpose.
• mssql
  Source implementation for SQL Server.
• mysql
  Source implementation for MySQL database.
• oracle
• postgres
  Source implementation for Postgres database, including the TLS support (client only).
• sqlite
  Source implementation for SQLite embedded database.
• trino

Traits

• PartitionParser
• Produce
  A type implemented Produce<T> means that it can produce a value T by consuming part of it’s raw data buffer.
• Source
• SourcePartition
  In general, a DataSource abstracts the data source as a stream, which can produce a sequence of values of variate types by repetitively calling the function produce.