Data Transfer API, gRPC: EndpointService
Статья создана
- Calls EndpointService
- Get
- GetEndpointRequest
- Endpoint
- EndpointSettings
- MysqlSource
- MysqlConnection
- OnPremiseMysql
- TLSMode
- TLSConfig
- Secret
- MysqlObjectTransferSettings
- PostgresSource
- PostgresConnection
- OnPremisePostgres
- PostgresObjectTransferSettings
- MongoSource
- MongoConnection
- MongoConnectionOptions
- OnPremiseMongo
- MongoCollection
- ClickhouseSource
- ClickhouseConnection
- ClickhouseConnectionOptions
- OnPremiseClickhouse
- ClickhouseShard
- MysqlTarget
- PostgresTarget
- ClickhouseTarget
- AltName
- ClickhouseSharding
- ColumnValueHash
- ColumnValueMapping
- ValueToShard
- ColumnValue
- MongoTarget
- List
- ListEndpointsRequest
- ListEndpointsResponse
- Endpoint
- EndpointSettings
- MysqlSource
- MysqlConnection
- OnPremiseMysql
- TLSMode
- TLSConfig
- Secret
- MysqlObjectTransferSettings
- PostgresSource
- PostgresConnection
- OnPremisePostgres
- PostgresObjectTransferSettings
- MongoSource
- MongoConnection
- MongoConnectionOptions
- OnPremiseMongo
- MongoCollection
- ClickhouseSource
- ClickhouseConnection
- ClickhouseConnectionOptions
- OnPremiseClickhouse
- ClickhouseShard
- MysqlTarget
- PostgresTarget
- ClickhouseTarget
- AltName
- ClickhouseSharding
- ColumnValueHash
- ColumnValueMapping
- ValueToShard
- ColumnValue
- MongoTarget
- Create
- CreateEndpointRequest
- EndpointSettings
- MysqlSource
- MysqlConnection
- OnPremiseMysql
- TLSMode
- TLSConfig
- Secret
- MysqlObjectTransferSettings
- PostgresSource
- PostgresConnection
- OnPremisePostgres
- PostgresObjectTransferSettings
- MongoSource
- MongoConnection
- MongoConnectionOptions
- OnPremiseMongo
- MongoCollection
- ClickhouseSource
- ClickhouseConnection
- ClickhouseConnectionOptions
- OnPremiseClickhouse
- ClickhouseShard
- MysqlTarget
- PostgresTarget
- ClickhouseTarget
- AltName
- ClickhouseSharding
- ColumnValueHash
- ColumnValueMapping
- ValueToShard
- ColumnValue
- MongoTarget
- Operation
- Update
- UpdateEndpointRequest
- EndpointSettings
- MysqlSource
- MysqlConnection
- OnPremiseMysql
- TLSMode
- TLSConfig
- Secret
- MysqlObjectTransferSettings
- PostgresSource
- PostgresConnection
- OnPremisePostgres
- PostgresObjectTransferSettings
- MongoSource
- MongoConnection
- MongoConnectionOptions
- OnPremiseMongo
- MongoCollection
- ClickhouseSource
- ClickhouseConnection
- ClickhouseConnectionOptions
- OnPremiseClickhouse
- ClickhouseShard
- MysqlTarget
- PostgresTarget
- ClickhouseTarget
- AltName
- ClickhouseSharding
- ColumnValueHash
- ColumnValueMapping
- ValueToShard
- ColumnValue
- MongoTarget
- Operation
- Delete
Call | Description |
---|---|
Get | |
List | |
Create | |
Update | |
Delete |
Calls EndpointService
Get
rpc Get (GetEndpointRequest) returns (Endpoint)
GetEndpointRequest
Field | Description |
---|---|
endpoint_id | string |
Endpoint
Field | Description |
---|---|
id | string |
folder_id | string |
name | string |
description | string |
labels | map<string,string> |
settings | EndpointSettings |
EndpointSettings
Field | Description |
---|---|
settings | oneof: mysql_source , postgres_source , mongo_source , clickhouse_source , mysql_target , postgres_target , clickhouse_target or mongo_target |
mysql_source | endpoint.MysqlSource |
postgres_source | endpoint.PostgresSource |
mongo_source | endpoint.MongoSource |
clickhouse_source | endpoint.ClickhouseSource |
mysql_target | endpoint.MysqlTarget |
postgres_target | endpoint.PostgresTarget |
clickhouse_target | endpoint.ClickhouseTarget |
mongo_target | endpoint.MongoTarget |
MysqlSource
Field | Description |
---|---|
connection | MysqlConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name You can leave it empty, then it will be possible to transfer tables from several databases at the same time from this source. |
service_database | string Database for service tables Default: data source database. Here created technical tables (__tm_keeper, __tm_gtid_keeper). |
user | string Username User for database access. |
password | Secret Password Password for database access. |
include_tables_regex[] | string |
exclude_tables_regex[] | string |
timezone | string Database timezone Is used for parsing timestamps for saving source timezones. Accepts values from IANA timezone database. Default: local timezone. |
object_transfer_settings | MysqlObjectTransferSettings Schema migration Select database objects to be transferred during activation or deactivation. |
MysqlConnection
Field | Description |
---|---|
connection | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string Managed cluster Managed Service for MySQL cluster ID |
on_premise | OnPremiseMysql On-premise Connection options for on-premise MySQL |
OnPremiseMysql
Field | Description |
---|---|
hosts[] | string |
port | int64 Database port Default: 3306. |
tls_mode | TLSMode TLS mode TLS settings for server connection. Disabled by default. |
subnet_id | string Network interface for endpoint Default: public IPv4. |
TLSMode
Field | Description |
---|---|
tls_mode | oneof: disabled or enabled |
disabled | google.protobuf.Empty |
enabled | TLSConfig |
TLSConfig
Field | Description |
---|---|
ca_certificate | string CA certificate X.509 certificate of the certificate authority which issued the server's certificate, in PEM format. When CA certificate is specified TLS is used to connect to the server. |
Secret
Field | Description |
---|---|
value | oneof: raw |
raw | string Password |
MysqlObjectTransferSettings
Field | Description |
---|---|
view | enum ObjectTransferStage Views CREATE VIEW ... |
routine | enum ObjectTransferStage Routines CREATE PROCEDURE ...; CREATE FUNCTION ...; |
trigger | enum ObjectTransferStage Triggers CREATE TRIGGER ... |
PostgresSource
Field | Description |
---|---|
connection | PostgresConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name |
user | string Username User for database access. |
password | Secret Password Password for database access. |
include_tables[] | string Included tables If none or empty list is presented, all tables are replicated. Full table name with schema. Can contain schema_name.* patterns. |
exclude_tables[] | string Excluded tables If none or empty list is presented, all tables are replicated. Full table name with schema. Can contain schema_name.* patterns. |
slot_byte_lag_limit | int64 Maximum WAL size for the replication slot Maximum WAL size held by the replication slot. Exceeding this limit will result in a replication failure and deletion of the replication slot. Unlimited by default. |
service_schema | string Database schema for service tables Default: public. Here created technical tables (__consumer_keeper, __data_transfer_mole_finder). |
object_transfer_settings | PostgresObjectTransferSettings Schema migration Select database objects to be transferred during activation or deactivation. |
PostgresConnection
Field | Description |
---|---|
connection | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string Managed cluster Managed Service for PostgreSQL cluster ID |
on_premise | OnPremisePostgres On-premise Connection options for on-premise PostgreSQL |
OnPremisePostgres
Field | Description |
---|---|
hosts[] | string |
port | int64 Database port Will be used if the cluster ID is not specified. Default: 6432. |
tls_mode | TLSMode TLS mode TLS settings for server connection. Disabled by default. |
subnet_id | string Network interface for endpoint Default: public IPv4. |
PostgresObjectTransferSettings
Field | Description |
---|---|
sequence | enum ObjectTransferStage Sequences CREATE SEQUENCE ... |
sequence_owned_by | enum ObjectTransferStage Owned sequences CREATE SEQUENCE ... OWNED BY ... |
table | enum ObjectTransferStage Tables CREATE TABLE ... |
primary_key | enum ObjectTransferStage Primary keys ALTER TABLE ... ADD PRIMARY KEY ... |
fk_constraint | enum ObjectTransferStage Foreign keys ALTER TABLE ... ADD FOREIGN KEY ... |
default_values | enum ObjectTransferStage Default values ALTER TABLE ... ALTER COLUMN ... SET DEFAULT ... |
constraint | enum ObjectTransferStage Constraints ALTER TABLE ... ADD CONSTRAINT ... |
index | enum ObjectTransferStage Indexes CREATE INDEX ... |
view | enum ObjectTransferStage Views CREATE VIEW ... |
function | enum ObjectTransferStage Functions CREATE FUNCTION ... |
trigger | enum ObjectTransferStage Triggers CREATE TRIGGER ... |
type | enum ObjectTransferStage Types CREATE TYPE ... |
rule | enum ObjectTransferStage Rules CREATE RULE ... |
collation | enum ObjectTransferStage Collations CREATE COLLATION ... |
policy | enum ObjectTransferStage Policies CREATE POLICY ... |
cast | enum ObjectTransferStage Casts CREATE CAST ... |
materialized_view | enum ObjectTransferStage Materialized views CREATE MATERIALIZED VIEW ... |
MongoSource
Field | Description |
---|---|
connection | MongoConnection |
subnet_id | string |
security_groups[] | string Security groups |
collections[] | MongoCollection |
excluded_collections[] | MongoCollection |
secondary_preferred_mode | bool |
MongoConnection
Field | Description |
---|---|
connection | oneof: connection_options |
connection_options | MongoConnectionOptions |
MongoConnectionOptions
Field | Description |
---|---|
address | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string |
on_premise | OnPremiseMongo |
user | string |
password | Secret |
auth_source | string |
OnPremiseMongo
Field | Description |
---|---|
hosts[] | string |
port | int64 |
tls_mode | TLSMode |
replica_set | string |
MongoCollection
Field | Description |
---|---|
database_name | string |
collection_name | string |
ClickhouseSource
Field | Description |
---|---|
connection | ClickhouseConnection |
subnet_id | string |
security_groups[] | string |
include_tables[] | string |
exclude_tables[] | string |
ClickhouseConnection
Field | Description |
---|---|
connection | oneof: connection_options |
connection_options | ClickhouseConnectionOptions |
ClickhouseConnectionOptions
Field | Description |
---|---|
address | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string |
on_premise | OnPremiseClickhouse |
database | string |
user | string |
password | Secret |
OnPremiseClickhouse
Field | Description |
---|---|
shards[] | ClickhouseShard |
http_port | int64 |
native_port | int64 |
tls_mode | TLSMode |
ClickhouseShard
Field | Description |
---|---|
name | string |
hosts[] | string |
MysqlTarget
Field | Description |
---|---|
connection | MysqlConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name Allowed to leave it empty, then the tables will be created in databases with the same names as on the source. If this field is empty, then you must fill below db schema for service table. |
user | string Username User for database access. |
password | Secret Password Password for database access. |
sql_mode | string sql_mode Default: NO_AUTO_VALUE_ON_ZERO,NO_DIR_IN_CREATE,NO_ENGINE_SUBSTITUTION. |
skip_constraint_checks | bool Disable constraints checks Recommend to disable for increase replication speed, but if schema contain cascading operations we don't recommend to disable. This option set FOREIGN_KEY_CHECKS=0 and UNIQUE_CHECKS=0. |
timezone | string Database timezone Is used for parsing timestamps for saving source timezones. Accepts values from IANA timezone database. Default: local timezone. |
cleanup_policy | enum CleanupPolicy Cleanup policy Cleanup policy for activate, reactivate and reupload processes. Default is DISABLED. |
service_database | string Database schema for service table Default: db name. Here created technical tables (__tm_keeper, __tm_gtid_keeper). |
PostgresTarget
Field | Description |
---|---|
connection | PostgresConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name |
user | string Username User for database access. |
password | Secret Password Password for database access. |
cleanup_policy | enum CleanupPolicy Cleanup policy Cleanup policy for activate, reactivate and reupload processes. Default is DISABLED. |
ClickhouseTarget
Field | Description |
---|---|
connection | ClickhouseConnection |
subnet_id | string |
security_groups[] | string |
clickhouse_cluster_name | string |
alt_names[] | AltName |
sharding | ClickhouseSharding |
cleanup_policy | enum ClickhouseCleanupPolicy |
AltName
Field | Description |
---|---|
from_name | string From table name |
to_name | string To table name |
ClickhouseSharding
Field | Description |
---|---|
sharding | oneof: column_value_hash , custom_mapping or transfer_id |
column_value_hash | ColumnValueHash |
custom_mapping | ColumnValueMapping |
transfer_id | google.protobuf.Empty |
ColumnValueHash
Field | Description |
---|---|
column_name | string |
ColumnValueMapping
Field | Description |
---|---|
column_name | string |
mapping[] | ValueToShard |
ValueToShard
Field | Description |
---|---|
column_value | ColumnValue |
shard_name | string |
ColumnValue
Field | Description |
---|---|
value | oneof: string_value |
string_value | string |
MongoTarget
Field | Description |
---|---|
connection | MongoConnection |
subnet_id | string |
security_groups[] | string Security groups |
database | string |
cleanup_policy | enum CleanupPolicy |
List
rpc List (ListEndpointsRequest) returns (ListEndpointsResponse)
ListEndpointsRequest
Field | Description |
---|---|
folder_id | string Identifier of the folder containing the endpoints to be listed. |
page_size | int64 The maximum number of endpoints to be sent in the response message. If the folder contains more endpoints than page_size, next_page_token will be included in the response message. Include it into the subsequent ListEndpointRequest to fetch the next page. Defaults to 100 if not specified. The maximum allowed value for this field is 500. |
page_token | string Opaque value identifying the endpoints page to be fetched. Should be empty in the first ListEndpointsRequest. Subsequent request should have this field filled with the next_page_token from the previous ListEndpointsResponse. |
ListEndpointsResponse
Field | Description |
---|---|
endpoints[] | Endpoint The list of endpoints. If there are more endpoints in the folder, then next_page_token is a non-empty string to be included into the subsequent ListEndpointsRequest to fetch the next endpoints page. |
next_page_token | string Opaque value identifying the next endpoints page. This field is empty if there are no more endpoints in the folder. Otherwise it is non-empty and should be included in the subsequent ListEndpointsRequest to fetch the next endpoints page. |
Endpoint
Field | Description |
---|---|
id | string |
folder_id | string |
name | string |
description | string |
labels | map<string,string> |
settings | EndpointSettings |
EndpointSettings
Field | Description |
---|---|
settings | oneof: mysql_source , postgres_source , mongo_source , clickhouse_source , mysql_target , postgres_target , clickhouse_target or mongo_target |
mysql_source | endpoint.MysqlSource |
postgres_source | endpoint.PostgresSource |
mongo_source | endpoint.MongoSource |
clickhouse_source | endpoint.ClickhouseSource |
mysql_target | endpoint.MysqlTarget |
postgres_target | endpoint.PostgresTarget |
clickhouse_target | endpoint.ClickhouseTarget |
mongo_target | endpoint.MongoTarget |
MysqlSource
Field | Description |
---|---|
connection | MysqlConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name You can leave it empty, then it will be possible to transfer tables from several databases at the same time from this source. |
service_database | string Database for service tables Default: data source database. Here created technical tables (__tm_keeper, __tm_gtid_keeper). |
user | string Username User for database access. |
password | Secret Password Password for database access. |
include_tables_regex[] | string |
exclude_tables_regex[] | string |
timezone | string Database timezone Is used for parsing timestamps for saving source timezones. Accepts values from IANA timezone database. Default: local timezone. |
object_transfer_settings | MysqlObjectTransferSettings Schema migration Select database objects to be transferred during activation or deactivation. |
MysqlConnection
Field | Description |
---|---|
connection | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string Managed cluster Managed Service for MySQL cluster ID |
on_premise | OnPremiseMysql On-premise Connection options for on-premise MySQL |
OnPremiseMysql
Field | Description |
---|---|
hosts[] | string |
port | int64 Database port Default: 3306. |
tls_mode | TLSMode TLS mode TLS settings for server connection. Disabled by default. |
subnet_id | string Network interface for endpoint Default: public IPv4. |
TLSMode
Field | Description |
---|---|
tls_mode | oneof: disabled or enabled |
disabled | google.protobuf.Empty |
enabled | TLSConfig |
TLSConfig
Field | Description |
---|---|
ca_certificate | string CA certificate X.509 certificate of the certificate authority which issued the server's certificate, in PEM format. When CA certificate is specified TLS is used to connect to the server. |
Secret
Field | Description |
---|---|
value | oneof: raw |
raw | string Password |
MysqlObjectTransferSettings
Field | Description |
---|---|
view | enum ObjectTransferStage Views CREATE VIEW ... |
routine | enum ObjectTransferStage Routines CREATE PROCEDURE ...; CREATE FUNCTION ...; |
trigger | enum ObjectTransferStage Triggers CREATE TRIGGER ... |
PostgresSource
Field | Description |
---|---|
connection | PostgresConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name |
user | string Username User for database access. |
password | Secret Password Password for database access. |
include_tables[] | string Included tables If none or empty list is presented, all tables are replicated. Full table name with schema. Can contain schema_name.* patterns. |
exclude_tables[] | string Excluded tables If none or empty list is presented, all tables are replicated. Full table name with schema. Can contain schema_name.* patterns. |
slot_byte_lag_limit | int64 Maximum WAL size for the replication slot Maximum WAL size held by the replication slot. Exceeding this limit will result in a replication failure and deletion of the replication slot. Unlimited by default. |
service_schema | string Database schema for service tables Default: public. Here created technical tables (__consumer_keeper, __data_transfer_mole_finder). |
object_transfer_settings | PostgresObjectTransferSettings Schema migration Select database objects to be transferred during activation or deactivation. |
PostgresConnection
Field | Description |
---|---|
connection | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string Managed cluster Managed Service for PostgreSQL cluster ID |
on_premise | OnPremisePostgres On-premise Connection options for on-premise PostgreSQL |
OnPremisePostgres
Field | Description |
---|---|
hosts[] | string |
port | int64 Database port Will be used if the cluster ID is not specified. Default: 6432. |
tls_mode | TLSMode TLS mode TLS settings for server connection. Disabled by default. |
subnet_id | string Network interface for endpoint Default: public IPv4. |
PostgresObjectTransferSettings
Field | Description |
---|---|
sequence | enum ObjectTransferStage Sequences CREATE SEQUENCE ... |
sequence_owned_by | enum ObjectTransferStage Owned sequences CREATE SEQUENCE ... OWNED BY ... |
table | enum ObjectTransferStage Tables CREATE TABLE ... |
primary_key | enum ObjectTransferStage Primary keys ALTER TABLE ... ADD PRIMARY KEY ... |
fk_constraint | enum ObjectTransferStage Foreign keys ALTER TABLE ... ADD FOREIGN KEY ... |
default_values | enum ObjectTransferStage Default values ALTER TABLE ... ALTER COLUMN ... SET DEFAULT ... |
constraint | enum ObjectTransferStage Constraints ALTER TABLE ... ADD CONSTRAINT ... |
index | enum ObjectTransferStage Indexes CREATE INDEX ... |
view | enum ObjectTransferStage Views CREATE VIEW ... |
function | enum ObjectTransferStage Functions CREATE FUNCTION ... |
trigger | enum ObjectTransferStage Triggers CREATE TRIGGER ... |
type | enum ObjectTransferStage Types CREATE TYPE ... |
rule | enum ObjectTransferStage Rules CREATE RULE ... |
collation | enum ObjectTransferStage Collations CREATE COLLATION ... |
policy | enum ObjectTransferStage Policies CREATE POLICY ... |
cast | enum ObjectTransferStage Casts CREATE CAST ... |
materialized_view | enum ObjectTransferStage Materialized views CREATE MATERIALIZED VIEW ... |
MongoSource
Field | Description |
---|---|
connection | MongoConnection |
subnet_id | string |
security_groups[] | string Security groups |
collections[] | MongoCollection |
excluded_collections[] | MongoCollection |
secondary_preferred_mode | bool |
MongoConnection
Field | Description |
---|---|
connection | oneof: connection_options |
connection_options | MongoConnectionOptions |
MongoConnectionOptions
Field | Description |
---|---|
address | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string |
on_premise | OnPremiseMongo |
user | string |
password | Secret |
auth_source | string |
OnPremiseMongo
Field | Description |
---|---|
hosts[] | string |
port | int64 |
tls_mode | TLSMode |
replica_set | string |
MongoCollection
Field | Description |
---|---|
database_name | string |
collection_name | string |
ClickhouseSource
Field | Description |
---|---|
connection | ClickhouseConnection |
subnet_id | string |
security_groups[] | string |
include_tables[] | string |
exclude_tables[] | string |
ClickhouseConnection
Field | Description |
---|---|
connection | oneof: connection_options |
connection_options | ClickhouseConnectionOptions |
ClickhouseConnectionOptions
Field | Description |
---|---|
address | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string |
on_premise | OnPremiseClickhouse |
database | string |
user | string |
password | Secret |
OnPremiseClickhouse
Field | Description |
---|---|
shards[] | ClickhouseShard |
http_port | int64 |
native_port | int64 |
tls_mode | TLSMode |
ClickhouseShard
Field | Description |
---|---|
name | string |
hosts[] | string |
MysqlTarget
Field | Description |
---|---|
connection | MysqlConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name Allowed to leave it empty, then the tables will be created in databases with the same names as on the source. If this field is empty, then you must fill below db schema for service table. |
user | string Username User for database access. |
password | Secret Password Password for database access. |
sql_mode | string sql_mode Default: NO_AUTO_VALUE_ON_ZERO,NO_DIR_IN_CREATE,NO_ENGINE_SUBSTITUTION. |
skip_constraint_checks | bool Disable constraints checks Recommend to disable for increase replication speed, but if schema contain cascading operations we don't recommend to disable. This option set FOREIGN_KEY_CHECKS=0 and UNIQUE_CHECKS=0. |
timezone | string Database timezone Is used for parsing timestamps for saving source timezones. Accepts values from IANA timezone database. Default: local timezone. |
cleanup_policy | enum CleanupPolicy Cleanup policy Cleanup policy for activate, reactivate and reupload processes. Default is DISABLED. |
service_database | string Database schema for service table Default: db name. Here created technical tables (__tm_keeper, __tm_gtid_keeper). |
PostgresTarget
Field | Description |
---|---|
connection | PostgresConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name |
user | string Username User for database access. |
password | Secret Password Password for database access. |
cleanup_policy | enum CleanupPolicy Cleanup policy Cleanup policy for activate, reactivate and reupload processes. Default is DISABLED. |
ClickhouseTarget
Field | Description |
---|---|
connection | ClickhouseConnection |
subnet_id | string |
security_groups[] | string |
clickhouse_cluster_name | string |
alt_names[] | AltName |
sharding | ClickhouseSharding |
cleanup_policy | enum ClickhouseCleanupPolicy |
AltName
Field | Description |
---|---|
from_name | string From table name |
to_name | string To table name |
ClickhouseSharding
Field | Description |
---|---|
sharding | oneof: column_value_hash , custom_mapping or transfer_id |
column_value_hash | ColumnValueHash |
custom_mapping | ColumnValueMapping |
transfer_id | google.protobuf.Empty |
ColumnValueHash
Field | Description |
---|---|
column_name | string |
ColumnValueMapping
Field | Description |
---|---|
column_name | string |
mapping[] | ValueToShard |
ValueToShard
Field | Description |
---|---|
column_value | ColumnValue |
shard_name | string |
ColumnValue
Field | Description |
---|---|
value | oneof: string_value |
string_value | string |
MongoTarget
Field | Description |
---|---|
connection | MongoConnection |
subnet_id | string |
security_groups[] | string Security groups |
database | string |
cleanup_policy | enum CleanupPolicy |
Create
rpc Create (CreateEndpointRequest) returns (operation.Operation)
CreateEndpointRequest
Field | Description |
---|---|
folder_id | string |
name | string |
description | string |
labels | map<string,string> |
settings | EndpointSettings |
EndpointSettings
Field | Description |
---|---|
settings | oneof: mysql_source , postgres_source , mongo_source , clickhouse_source , mysql_target , postgres_target , clickhouse_target or mongo_target |
mysql_source | endpoint.MysqlSource |
postgres_source | endpoint.PostgresSource |
mongo_source | endpoint.MongoSource |
clickhouse_source | endpoint.ClickhouseSource |
mysql_target | endpoint.MysqlTarget |
postgres_target | endpoint.PostgresTarget |
clickhouse_target | endpoint.ClickhouseTarget |
mongo_target | endpoint.MongoTarget |
MysqlSource
Field | Description |
---|---|
connection | MysqlConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name You can leave it empty, then it will be possible to transfer tables from several databases at the same time from this source. |
service_database | string Database for service tables Default: data source database. Here created technical tables (__tm_keeper, __tm_gtid_keeper). |
user | string Username User for database access. |
password | Secret Password Password for database access. |
include_tables_regex[] | string |
exclude_tables_regex[] | string |
timezone | string Database timezone Is used for parsing timestamps for saving source timezones. Accepts values from IANA timezone database. Default: local timezone. |
object_transfer_settings | MysqlObjectTransferSettings Schema migration Select database objects to be transferred during activation or deactivation. |
MysqlConnection
Field | Description |
---|---|
connection | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string Managed cluster Managed Service for MySQL cluster ID |
on_premise | OnPremiseMysql On-premise Connection options for on-premise MySQL |
OnPremiseMysql
Field | Description |
---|---|
hosts[] | string |
port | int64 Database port Default: 3306. |
tls_mode | TLSMode TLS mode TLS settings for server connection. Disabled by default. |
subnet_id | string Network interface for endpoint Default: public IPv4. |
TLSMode
Field | Description |
---|---|
tls_mode | oneof: disabled or enabled |
disabled | google.protobuf.Empty |
enabled | TLSConfig |
TLSConfig
Field | Description |
---|---|
ca_certificate | string CA certificate X.509 certificate of the certificate authority which issued the server's certificate, in PEM format. When CA certificate is specified TLS is used to connect to the server. |
Secret
Field | Description |
---|---|
value | oneof: raw |
raw | string Password |
MysqlObjectTransferSettings
Field | Description |
---|---|
view | enum ObjectTransferStage Views CREATE VIEW ... |
routine | enum ObjectTransferStage Routines CREATE PROCEDURE ...; CREATE FUNCTION ...; |
trigger | enum ObjectTransferStage Triggers CREATE TRIGGER ... |
PostgresSource
Field | Description |
---|---|
connection | PostgresConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name |
user | string Username User for database access. |
password | Secret Password Password for database access. |
include_tables[] | string Included tables If none or empty list is presented, all tables are replicated. Full table name with schema. Can contain schema_name.* patterns. |
exclude_tables[] | string Excluded tables If none or empty list is presented, all tables are replicated. Full table name with schema. Can contain schema_name.* patterns. |
slot_byte_lag_limit | int64 Maximum WAL size for the replication slot Maximum WAL size held by the replication slot. Exceeding this limit will result in a replication failure and deletion of the replication slot. Unlimited by default. |
service_schema | string Database schema for service tables Default: public. Here created technical tables (__consumer_keeper, __data_transfer_mole_finder). |
object_transfer_settings | PostgresObjectTransferSettings Schema migration Select database objects to be transferred during activation or deactivation. |
PostgresConnection
Field | Description |
---|---|
connection | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string Managed cluster Managed Service for PostgreSQL cluster ID |
on_premise | OnPremisePostgres On-premise Connection options for on-premise PostgreSQL |
OnPremisePostgres
Field | Description |
---|---|
hosts[] | string |
port | int64 Database port Will be used if the cluster ID is not specified. Default: 6432. |
tls_mode | TLSMode TLS mode TLS settings for server connection. Disabled by default. |
subnet_id | string Network interface for endpoint Default: public IPv4. |
PostgresObjectTransferSettings
Field | Description |
---|---|
sequence | enum ObjectTransferStage Sequences CREATE SEQUENCE ... |
sequence_owned_by | enum ObjectTransferStage Owned sequences CREATE SEQUENCE ... OWNED BY ... |
table | enum ObjectTransferStage Tables CREATE TABLE ... |
primary_key | enum ObjectTransferStage Primary keys ALTER TABLE ... ADD PRIMARY KEY ... |
fk_constraint | enum ObjectTransferStage Foreign keys ALTER TABLE ... ADD FOREIGN KEY ... |
default_values | enum ObjectTransferStage Default values ALTER TABLE ... ALTER COLUMN ... SET DEFAULT ... |
constraint | enum ObjectTransferStage Constraints ALTER TABLE ... ADD CONSTRAINT ... |
index | enum ObjectTransferStage Indexes CREATE INDEX ... |
view | enum ObjectTransferStage Views CREATE VIEW ... |
function | enum ObjectTransferStage Functions CREATE FUNCTION ... |
trigger | enum ObjectTransferStage Triggers CREATE TRIGGER ... |
type | enum ObjectTransferStage Types CREATE TYPE ... |
rule | enum ObjectTransferStage Rules CREATE RULE ... |
collation | enum ObjectTransferStage Collations CREATE COLLATION ... |
policy | enum ObjectTransferStage Policies CREATE POLICY ... |
cast | enum ObjectTransferStage Casts CREATE CAST ... |
materialized_view | enum ObjectTransferStage Materialized views CREATE MATERIALIZED VIEW ... |
MongoSource
Field | Description |
---|---|
connection | MongoConnection |
subnet_id | string |
security_groups[] | string Security groups |
collections[] | MongoCollection |
excluded_collections[] | MongoCollection |
secondary_preferred_mode | bool |
MongoConnection
Field | Description |
---|---|
connection | oneof: connection_options |
connection_options | MongoConnectionOptions |
MongoConnectionOptions
Field | Description |
---|---|
address | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string |
on_premise | OnPremiseMongo |
user | string |
password | Secret |
auth_source | string |
OnPremiseMongo
Field | Description |
---|---|
hosts[] | string |
port | int64 |
tls_mode | TLSMode |
replica_set | string |
MongoCollection
Field | Description |
---|---|
database_name | string |
collection_name | string |
ClickhouseSource
Field | Description |
---|---|
connection | ClickhouseConnection |
subnet_id | string |
security_groups[] | string |
include_tables[] | string |
exclude_tables[] | string |
ClickhouseConnection
Field | Description |
---|---|
connection | oneof: connection_options |
connection_options | ClickhouseConnectionOptions |
ClickhouseConnectionOptions
Field | Description |
---|---|
address | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string |
on_premise | OnPremiseClickhouse |
database | string |
user | string |
password | Secret |
OnPremiseClickhouse
Field | Description |
---|---|
shards[] | ClickhouseShard |
http_port | int64 |
native_port | int64 |
tls_mode | TLSMode |
ClickhouseShard
Field | Description |
---|---|
name | string |
hosts[] | string |
MysqlTarget
Field | Description |
---|---|
connection | MysqlConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name Allowed to leave it empty, then the tables will be created in databases with the same names as on the source. If this field is empty, then you must fill below db schema for service table. |
user | string Username User for database access. |
password | Secret Password Password for database access. |
sql_mode | string sql_mode Default: NO_AUTO_VALUE_ON_ZERO,NO_DIR_IN_CREATE,NO_ENGINE_SUBSTITUTION. |
skip_constraint_checks | bool Disable constraints checks Recommend to disable for increase replication speed, but if schema contain cascading operations we don't recommend to disable. This option set FOREIGN_KEY_CHECKS=0 and UNIQUE_CHECKS=0. |
timezone | string Database timezone Is used for parsing timestamps for saving source timezones. Accepts values from IANA timezone database. Default: local timezone. |
cleanup_policy | enum CleanupPolicy Cleanup policy Cleanup policy for activate, reactivate and reupload processes. Default is DISABLED. |
service_database | string Database schema for service table Default: db name. Here created technical tables (__tm_keeper, __tm_gtid_keeper). |
PostgresTarget
Field | Description |
---|---|
connection | PostgresConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name |
user | string Username User for database access. |
password | Secret Password Password for database access. |
cleanup_policy | enum CleanupPolicy Cleanup policy Cleanup policy for activate, reactivate and reupload processes. Default is DISABLED. |
ClickhouseTarget
Field | Description |
---|---|
connection | ClickhouseConnection |
subnet_id | string |
security_groups[] | string |
clickhouse_cluster_name | string |
alt_names[] | AltName |
sharding | ClickhouseSharding |
cleanup_policy | enum ClickhouseCleanupPolicy |
AltName
Field | Description |
---|---|
from_name | string From table name |
to_name | string To table name |
ClickhouseSharding
Field | Description |
---|---|
sharding | oneof: column_value_hash , custom_mapping or transfer_id |
column_value_hash | ColumnValueHash |
custom_mapping | ColumnValueMapping |
transfer_id | google.protobuf.Empty |
ColumnValueHash
Field | Description |
---|---|
column_name | string |
ColumnValueMapping
Field | Description |
---|---|
column_name | string |
mapping[] | ValueToShard |
ValueToShard
Field | Description |
---|---|
column_value | ColumnValue |
shard_name | string |
ColumnValue
Field | Description |
---|---|
value | oneof: string_value |
string_value | string |
MongoTarget
Field | Description |
---|---|
connection | MongoConnection |
subnet_id | string |
security_groups[] | string Security groups |
database | string |
cleanup_policy | enum CleanupPolicy |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any The normal response of the operation in case of success. If the original method returns no data on success, such as Delete, the response is google.protobuf.Empty. If the original method is the standard Create/Update, the response should be the target resource of the operation. Any method that returns a long-running operation should document the response type, if any. |
Update
rpc Update (UpdateEndpointRequest) returns (operation.Operation)
UpdateEndpointRequest
Field | Description |
---|---|
endpoint_id | string Identifier of the endpoint to be updated. |
name | string The new endpoint name. Must be unique within the folder. |
description | string The new description for the endpoint. |
labels | map<string,string> |
settings | EndpointSettings The new endpoint name. Must be unique within the folder. |
update_mask | google.protobuf.FieldMask Field mask specifying endpoint fields to be updated. Semantics for this field is described here: https://pkg.go.dev/google.golang.org/protobuf/types/known/fieldmaskpb#FieldMask The only exception is that if the repeated field is specified in the mask, then the new value replaces the old one instead of being appended to the old one. |
EndpointSettings
Field | Description |
---|---|
settings | oneof: mysql_source , postgres_source , mongo_source , clickhouse_source , mysql_target , postgres_target , clickhouse_target or mongo_target |
mysql_source | endpoint.MysqlSource |
postgres_source | endpoint.PostgresSource |
mongo_source | endpoint.MongoSource |
clickhouse_source | endpoint.ClickhouseSource |
mysql_target | endpoint.MysqlTarget |
postgres_target | endpoint.PostgresTarget |
clickhouse_target | endpoint.ClickhouseTarget |
mongo_target | endpoint.MongoTarget |
MysqlSource
Field | Description |
---|---|
connection | MysqlConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name You can leave it empty, then it will be possible to transfer tables from several databases at the same time from this source. |
service_database | string Database for service tables Default: data source database. Here created technical tables (__tm_keeper, __tm_gtid_keeper). |
user | string Username User for database access. |
password | Secret Password Password for database access. |
include_tables_regex[] | string |
exclude_tables_regex[] | string |
timezone | string Database timezone Is used for parsing timestamps for saving source timezones. Accepts values from IANA timezone database. Default: local timezone. |
object_transfer_settings | MysqlObjectTransferSettings Schema migration Select database objects to be transferred during activation or deactivation. |
MysqlConnection
Field | Description |
---|---|
connection | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string Managed cluster Managed Service for MySQL cluster ID |
on_premise | OnPremiseMysql On-premise Connection options for on-premise MySQL |
OnPremiseMysql
Field | Description |
---|---|
hosts[] | string |
port | int64 Database port Default: 3306. |
tls_mode | TLSMode TLS mode TLS settings for server connection. Disabled by default. |
subnet_id | string Network interface for endpoint Default: public IPv4. |
TLSMode
Field | Description |
---|---|
tls_mode | oneof: disabled or enabled |
disabled | google.protobuf.Empty |
enabled | TLSConfig |
TLSConfig
Field | Description |
---|---|
ca_certificate | string CA certificate X.509 certificate of the certificate authority which issued the server's certificate, in PEM format. When CA certificate is specified TLS is used to connect to the server. |
Secret
Field | Description |
---|---|
value | oneof: raw |
raw | string Password |
MysqlObjectTransferSettings
Field | Description |
---|---|
view | enum ObjectTransferStage Views CREATE VIEW ... |
routine | enum ObjectTransferStage Routines CREATE PROCEDURE ...; CREATE FUNCTION ...; |
trigger | enum ObjectTransferStage Triggers CREATE TRIGGER ... |
PostgresSource
Field | Description |
---|---|
connection | PostgresConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name |
user | string Username User for database access. |
password | Secret Password Password for database access. |
include_tables[] | string Included tables If none or empty list is presented, all tables are replicated. Full table name with schema. Can contain schema_name.* patterns. |
exclude_tables[] | string Excluded tables If none or empty list is presented, all tables are replicated. Full table name with schema. Can contain schema_name.* patterns. |
slot_byte_lag_limit | int64 Maximum WAL size for the replication slot Maximum WAL size held by the replication slot. Exceeding this limit will result in a replication failure and deletion of the replication slot. Unlimited by default. |
service_schema | string Database schema for service tables Default: public. Here created technical tables (__consumer_keeper, __data_transfer_mole_finder). |
object_transfer_settings | PostgresObjectTransferSettings Schema migration Select database objects to be transferred during activation or deactivation. |
PostgresConnection
Field | Description |
---|---|
connection | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string Managed cluster Managed Service for PostgreSQL cluster ID |
on_premise | OnPremisePostgres On-premise Connection options for on-premise PostgreSQL |
OnPremisePostgres
Field | Description |
---|---|
hosts[] | string |
port | int64 Database port Will be used if the cluster ID is not specified. Default: 6432. |
tls_mode | TLSMode TLS mode TLS settings for server connection. Disabled by default. |
subnet_id | string Network interface for endpoint Default: public IPv4. |
PostgresObjectTransferSettings
Field | Description |
---|---|
sequence | enum ObjectTransferStage Sequences CREATE SEQUENCE ... |
sequence_owned_by | enum ObjectTransferStage Owned sequences CREATE SEQUENCE ... OWNED BY ... |
table | enum ObjectTransferStage Tables CREATE TABLE ... |
primary_key | enum ObjectTransferStage Primary keys ALTER TABLE ... ADD PRIMARY KEY ... |
fk_constraint | enum ObjectTransferStage Foreign keys ALTER TABLE ... ADD FOREIGN KEY ... |
default_values | enum ObjectTransferStage Default values ALTER TABLE ... ALTER COLUMN ... SET DEFAULT ... |
constraint | enum ObjectTransferStage Constraints ALTER TABLE ... ADD CONSTRAINT ... |
index | enum ObjectTransferStage Indexes CREATE INDEX ... |
view | enum ObjectTransferStage Views CREATE VIEW ... |
function | enum ObjectTransferStage Functions CREATE FUNCTION ... |
trigger | enum ObjectTransferStage Triggers CREATE TRIGGER ... |
type | enum ObjectTransferStage Types CREATE TYPE ... |
rule | enum ObjectTransferStage Rules CREATE RULE ... |
collation | enum ObjectTransferStage Collations CREATE COLLATION ... |
policy | enum ObjectTransferStage Policies CREATE POLICY ... |
cast | enum ObjectTransferStage Casts CREATE CAST ... |
materialized_view | enum ObjectTransferStage Materialized views CREATE MATERIALIZED VIEW ... |
MongoSource
Field | Description |
---|---|
connection | MongoConnection |
subnet_id | string |
security_groups[] | string Security groups |
collections[] | MongoCollection |
excluded_collections[] | MongoCollection |
secondary_preferred_mode | bool |
MongoConnection
Field | Description |
---|---|
connection | oneof: connection_options |
connection_options | MongoConnectionOptions |
MongoConnectionOptions
Field | Description |
---|---|
address | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string |
on_premise | OnPremiseMongo |
user | string |
password | Secret |
auth_source | string |
OnPremiseMongo
Field | Description |
---|---|
hosts[] | string |
port | int64 |
tls_mode | TLSMode |
replica_set | string |
MongoCollection
Field | Description |
---|---|
database_name | string |
collection_name | string |
ClickhouseSource
Field | Description |
---|---|
connection | ClickhouseConnection |
subnet_id | string |
security_groups[] | string |
include_tables[] | string |
exclude_tables[] | string |
ClickhouseConnection
Field | Description |
---|---|
connection | oneof: connection_options |
connection_options | ClickhouseConnectionOptions |
ClickhouseConnectionOptions
Field | Description |
---|---|
address | oneof: mdb_cluster_id or on_premise |
mdb_cluster_id | string |
on_premise | OnPremiseClickhouse |
database | string |
user | string |
password | Secret |
OnPremiseClickhouse
Field | Description |
---|---|
shards[] | ClickhouseShard |
http_port | int64 |
native_port | int64 |
tls_mode | TLSMode |
ClickhouseShard
Field | Description |
---|---|
name | string |
hosts[] | string |
MysqlTarget
Field | Description |
---|---|
connection | MysqlConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name Allowed to leave it empty, then the tables will be created in databases with the same names as on the source. If this field is empty, then you must fill below db schema for service table. |
user | string Username User for database access. |
password | Secret Password Password for database access. |
sql_mode | string sql_mode Default: NO_AUTO_VALUE_ON_ZERO,NO_DIR_IN_CREATE,NO_ENGINE_SUBSTITUTION. |
skip_constraint_checks | bool Disable constraints checks Recommend to disable for increase replication speed, but if schema contain cascading operations we don't recommend to disable. This option set FOREIGN_KEY_CHECKS=0 and UNIQUE_CHECKS=0. |
timezone | string Database timezone Is used for parsing timestamps for saving source timezones. Accepts values from IANA timezone database. Default: local timezone. |
cleanup_policy | enum CleanupPolicy Cleanup policy Cleanup policy for activate, reactivate and reupload processes. Default is DISABLED. |
service_database | string Database schema for service table Default: db name. Here created technical tables (__tm_keeper, __tm_gtid_keeper). |
PostgresTarget
Field | Description |
---|---|
connection | PostgresConnection Connection settings Database connection settings |
security_groups[] | string Security groups |
database | string Database name |
user | string Username User for database access. |
password | Secret Password Password for database access. |
cleanup_policy | enum CleanupPolicy Cleanup policy Cleanup policy for activate, reactivate and reupload processes. Default is DISABLED. |
ClickhouseTarget
Field | Description |
---|---|
connection | ClickhouseConnection |
subnet_id | string |
security_groups[] | string |
clickhouse_cluster_name | string |
alt_names[] | AltName |
sharding | ClickhouseSharding |
cleanup_policy | enum ClickhouseCleanupPolicy |
AltName
Field | Description |
---|---|
from_name | string From table name |
to_name | string To table name |
ClickhouseSharding
Field | Description |
---|---|
sharding | oneof: column_value_hash , custom_mapping or transfer_id |
column_value_hash | ColumnValueHash |
custom_mapping | ColumnValueMapping |
transfer_id | google.protobuf.Empty |
ColumnValueHash
Field | Description |
---|---|
column_name | string |
ColumnValueMapping
Field | Description |
---|---|
column_name | string |
mapping[] | ValueToShard |
ValueToShard
Field | Description |
---|---|
column_value | ColumnValue |
shard_name | string |
ColumnValue
Field | Description |
---|---|
value | oneof: string_value |
string_value | string |
MongoTarget
Field | Description |
---|---|
connection | MongoConnection |
subnet_id | string |
security_groups[] | string Security groups |
database | string |
cleanup_policy | enum CleanupPolicy |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any The normal response of the operation in case of success. If the original method returns no data on success, such as Delete, the response is google.protobuf.Empty. If the original method is the standard Create/Update, the response should be the target resource of the operation. Any method that returns a long-running operation should document the response type, if any. |
Delete
rpc Delete (DeleteEndpointRequest) returns (operation.Operation)
DeleteEndpointRequest
Field | Description |
---|---|
endpoint_id | string |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any The normal response of the operation in case of success. If the original method returns no data on success, such as Delete, the response is google.protobuf.Empty. If the original method is the standard Create/Update, the response should be the target resource of the operation. Any method that returns a long-running operation should document the response type, if any. |