PostgreSQL
| Function | Description |
|---|---|
Schema Migration | If the target schema does not exist, BladePipe will automatically generate and execute CREATE statements based on the source metadata and the mapping rule. |
Full Data Migration | Migrate data by sequentially scanning data in tables and writing it in batches to the target database. |
Incremental Data Sync | Sync of common DML like INSERT, UPDATE, DELETE is supported. |
Data Verification and Correction | Verify all existing data. Optionally, you can correct the inconsistent data based on verification results. Scheduled DataTasks are supported. |
Subscription Modification | Add, delete, or modify the subscribed tables with support for historical data migration. For more information, see Modify Subscription. |
Table Name Mapping | Support the mapping rules, namely, keeping the name the same as that in Source, converting the text to lowercase, converting the text to uppercase, truncating the name by "_digit" suffix. |
DDL Sync | PostgreSQL DDL sync is realized by triggers. The user should have the permissions on triggers and tables. For more information, see Permissions Required for PostgreSQL |
Metadata Retrieval | Retrieve the target metadata with filtering conditions from the source table. |
Advanced Functions
| Function | Description |
|---|---|
Handling of Zero Value for Time | Allow setting zero value for time to different data types to prevent errors when writing to the Target. |
Custom Code | For more information, see Custom Code Processing, Debug Custom Code and Logging in Custom Code. |
Data Filtering Conditions | Support data filtering using WHERE conditions, with SQL-92 as the SQL language. For more information, see Data Filtering. |
Setting Target Primary Key | Change the primary key to another field to facilitate data aggregation and other operations. |
Limits
| Limit | Description |
|---|---|
Primary Key Conflict Handling | PostgreSQL <= 9.4 or Greenplum <= 6 does not support conflict skipping or replacement. Performance may be affected when there are a large number of primary key conflicts. |
Prerequisites
| Prerequisite | Description |
|---|---|
Permissions for Account | Required permissions (taking a self-managed database as an example):
|
Incremental Data Sync Preparation | Prepare as follows:
|
Port Preparation | Allow the migration and sync node (Worker) to connect to the PostgreSQL port (e.g., port 5432). |
Parameters
| Parameter | Description |
|---|---|
fullFetchSize | Fetch size for scaning full data. |
eventStoreSize | Cache size for parsed incremental events. |
ignoreGisSRID | Whether to ignore SRID when parsing GIS data types. |
defaultGisSRID | Set the SRID for GIS data types. |
Tips: To modify the general parameters, see General Parameters and Functions.
Prerequisites
| Prerequisite | Description |
|---|---|
Permissions for Account | Required permissions include SELECT, INSERT, DELETE, UPDATE, and common DDL permissions. |
Port Preparation | Allow the migration and sync node (Worker) to connect to the PostgreSQL / Greenplum / Aliyun AnalyticDB for PostgreSQL port (e.g., port 5432). |
Parameters
| Parameter | Description |
|---|---|
keyConflictStrategy | Strategy for handling primary key conflicts during write in Incremental DataTask:
|
dstWholeReplace | Convert INSERT and UPDATE operations into full row replacement in the Target. |
enableTimeZoneProcess | Enable time zone conversion to datetime fields. |
timezone | Target time zone, e.g., +08:00, Asia/Shanghai, America/New_York. |
defaultZeroDate | Default value for replacing '0000-00-00 00:00:00' / '0000-00-00' values. Optional values include:
|
caseSensitive | Case sensitivity strategy for SQL statements, including:
|
writeStrategy | Strategy of writing data to the Target, including:
|
defaultGisSRID | Set the SRID for GIS data types. |
Tips: To modify the general parameters, see General Parameters and Functions.