Created
December 9, 2025 09:01
-
-
Save cabecada/efe116c63ebab2b57fe362ae1941e5ad to your computer and use it in GitHub Desktop.
postgresql data recovery utility
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| testing the utility on https://github.com/wublabdubdub/PDU-PostgresqlDataUnloader | |
| --setup archiving on disk for now | |
| postgres@ubuntu:/tmp$ grep '^archive_command' db1/postgresql.conf | |
| archive_command = 'test ! -f /tmp/archive/%f && cp %p /tmp/archive/%f' # command to use to archive a WAL file | |
| postgres@ubuntu:/tmp$ pg_ctl -D db1 -l logfile start | |
| waiting for server to start.... done | |
| server started | |
| --create a simple table with 100000 records | |
| --do updates and recover pre update data | |
| --do deleted and recover pre delete data | |
| postgres@ubuntu:/tmp$ psql | |
| psql (18.1 (Ubuntu 18.1-1.pgdg24.04+2)) | |
| Type "help" for help. | |
| postgres=# create table t(col1 bigint, col2 timestamptz, col3 date); | |
| CREATE TABLE | |
| postgres=# insert into t select x, x * '1 min'::interval + now(), current_date + x * '1 day'::interval from generate_series(1, 100000) x; | |
| INSERT 0 100000 | |
| postgres=# select pg_current_xact_id(); | |
| pg_current_xact_id | |
| -------------------- | |
| 754 | |
| (1 row) | |
| postgres=# checkpoint; | |
| CHECKPOINT | |
| postgres=# update t set col3 = current_date where col1 < 100; | |
| UPDATE 99 | |
| postgres=# checkpoint; | |
| CHECKPOINT | |
| postgres=# select pg_current_xact_id(); | |
| pg_current_xact_id | |
| -------------------- | |
| 756 | |
| (1 row) | |
| postgres=# delete from t where col1 % 100 = 0 and col1 < 501; | |
| DELETE 5 | |
| postgres=# checkpoint; | |
| CHECKPOINT | |
| postgres=# select pg_current_xact_id(); | |
| pg_current_xact_id | |
| -------------------- | |
| 759 | |
| (1 row) | |
| postgres=# select * from t where col1 % 100 = 0 and col1 < 501; | |
| col1 | col2 | col3 | |
| ------+------+------ | |
| (0 rows) | |
| postgres=# | |
| \q | |
| ---------------------------------------------------- | |
| use the pdu tool; | |
| ----------------------------------------------------- | |
| --list config | |
| cat pdu.ini | |
| #Postgresql Data Directory | |
| PGDATA=/tmp/db1 | |
| #Postgresql Archive Directory | |
| ARCHIVE_DEST=/tmp/archive | |
| #the Disk that dropScan need to scan | |
| DISK_PATH= | |
| #The number of data blocks skipped during dropScan. The smaller the value, the more comprehensive the disk coverage, and the slower the speed. | |
| BLOCK_INTERVAL=1 | |
| #PGDATA to be excluded during dropscan. | |
| PGDATA_EXCLUDE= | |
| ---------------------------------------------------- | |
| run the util | |
| -------------------------------------------- | |
| postgres@ubuntu:~/PDU-PostgresqlDataUnloader/PDU-PostgresqlDataUnloader-PDU_binary_packages$ ./pdu18 | |
| ╔══════════════════════════════════════════════════════╗ | |
| ║ Copyright 2024-2025 ZhangChen. All rights reserved ║ | |
| ║ PDU: PostgreSQL Data Unloader ║ | |
| ║ Version 3.0.25.12 (2025-12-08) ║ | |
| ║ Valid Until 2124-12-08 10:00:00 ║ | |
| ╚══════════════════════════════════════════════════════╝ | |
| Current DB Supported Version: | |
| ────────────────────────── | |
| • PostgreSQL 18 | |
| ╔═══════════════════════════════════════════╗ | |
| ║ COMMUNITY VERSION ║ | |
| ╠═══════════════════════════════════════════╣ | |
| ║ • Max 100000 Records Per Table (unload) ║ | |
| ║ • Max 100000 Records Per Table (restore) ║ | |
| ║ • Max 1 GB Per Table ║ | |
| ║ • Max 50 Columns Per Table ║ | |
| ║ • Max 500 Object Per Schema ║ | |
| ╚═══════════════════════════════════════════╝ | |
| Contact Me: | |
| ─────────────────── | |
| • WeChat: x1987LJ2020929 | |
| • Email: 1109315180@qq.com | |
| • Tel: 15251853831 | |
| PDU.public=# b; | |
| Initializing... | |
| -pg_database:</tmp/db1/global/1262> | |
| Database:postgres | |
| -pg_schema:</tmp/db1/base/5/2615> | |
| -pg_class:</tmp/db1/base/5/1259> 84 Records | |
| -pg_attribute:</tmp/db1/base/5/1249> 3170 Records | |
| Schema: | |
| ▌ public 1 tables | |
| PDU.public=# use postgres; | |
| ┌────────────────────────────────────────┐ | |
| │ Schema │ Tab Num │ | |
| ├────────────────────────────────────────┤ | |
| │ public │ 1 │ | |
| └────────────────────────────────────────┘ | |
| postgres.public=# \? | |
| postgres.public=# ; | |
| PDU Data Rescue Tool | Command Reference | |
| ┌──────────────────────────────────────────────────────────────────────────────────────────────────┐ | |
| **Basic Operations** | |
| b; │ Initialize database metadata | |
| <exit>;<\q>; │ Exit the tool | |
| ----------------------------------------.-------------------------------- | |
| **Database Context** | |
| use <db>; │ Set current database (e.g. use logs;) | |
| set <schema>; │ Set current schema (e.g. set recovery;) | |
| ----------------------------------------.-------------------------------- | |
| **Metadata Display** | |
| \l; │ List all databases | |
| \dn; │ List all schema in current database | |
| \dt; │ List tables in current schema | |
| \d+ <table>; │ View table structure details (e.g. \d+ users;) | |
| \d <table>; │ View column types (e.g. \d users;) | |
| ----------------------------------------.-------------------------------- | |
| **Data Export** | |
| u|unload tab <table>; │ Export table to CSV (e.g. unload tab orders;) | |
| u|unload sch <schema>; │ Export entire schema (e.g. unload sch public;) | |
| u|unload ddl; │ Generate DDL statements of current schema | |
| u|unload copy; │ Generate COPY statements for CSVs | |
| ----------------------------------------.-------------------------------- | |
| **Accidental Operation Data Recovery** | |
| scan [t1|manual]; │ Scan deleted/update records of tables/Init metadata from manual | |
| restore del/upd [<TxID>|all]; │ Restore data by transaction ID/time range | |
| add <filenode> <table> <columns>; │ Manually add table info (e.g. add 12345 t1 varchar,...) [!] Datafile should be put into path 'restore/datafile' | |
| restore db <db> <path>; │ Init customized database directory (e.g. restore db xmandb /home/...) | |
| ----------------------------------------.-------------------------------- | |
| **Drop Table Recovery** | |
| dropscan/ds; │ Perform fragment scan recovery for tables configured in restore/tab.config | |
| dropscan/ds repair; │ Recover previously failed TOAST table scans | |
| dropscan/ds clean; │ Delete all directories under restore/dropscan | |
| dropscan/ds copy; │ Generate COPY commands for all table files under restore/dropscan | |
| ----------------------------------------.-------------------------------- | |
| **Parameters** | |
| p|param startwal/endwal <WAL>; │ Set WAL scan range (default archive boundaries) | |
| p|param starttime/endtime <TIME>; │ Set time scan range (e.g. 2025-01-01 00:00:00) | |
| p|param resmode tx|time; │ Set recovery mode (Transaction/Time) | |
| p|param restype delete|update; │ Set recovery type (Delete/Update) | |
| p|param exmode csv|sql; │ Set export format (default CSV) | |
| p|param encoding utf8|gbk; │ Set character encoding (default utf8) | |
| reset <parameter>|all; │ Reset specified parameter|all parameter | |
| show; │ Display all parameters | |
| t; │ Display all supported datatypes | |
| └──────────────────────────────────────────────────────────────────────────────────────────────────┘ | |
| Syntax Rules | |
| ◈ All commands must end with `;` | |
| ---------------------------------------------------- | |
| set resmode param to update, this will help recovery pre update data | |
| ---------------------------------------------------- | |
| postgres.public=# p resmode tx; | |
| ┌─────────────────────────────────────────────────────────────────┐ | |
| │ parameter │ value │ | |
| ├─────────────────────────────────────────────────────────────────┤ | |
| │ startwal │ │ | |
| │ endwal │ │ | |
| │ startlsnt │ │ | |
| │ endlsnt │ │ | |
| │ starttime │ │ | |
| │ endtime │ │ | |
| │ resmode(Data Restore Mode) │ TX │ | |
| │ exmode(Data Export Mode) │ CSV │ | |
| │ encoding │ UTF8 │ | |
| │ restype(Data Restore Type) │ DELETE │ | |
| │ isomode │ off │ | |
| └─────────────────────────────────────────────────────────────────┘ | |
| postgres.public=# p restype update; | |
| ┌─────────────────────────────────────────────────────────────────┐ | |
| │ parameter │ value │ | |
| ├─────────────────────────────────────────────────────────────────┤ | |
| │ startwal │ │ | |
| │ endwal │ │ | |
| │ startlsnt │ │ | |
| │ endlsnt │ │ | |
| │ starttime │ │ | |
| │ endtime │ │ | |
| │ resmode(Data Restore Mode) │ TX │ | |
| │ exmode(Data Export Mode) │ CSV │ | |
| │ encoding │ UTF8 │ | |
| │ restype(Data Restore Type) │ UPDATE │ | |
| │ isomode │ off │ | |
| └─────────────────────────────────────────────────────────────────┘ | |
| --recover updates | |
| postgres.public=# scan t; | |
| Scanning updated Records for table<t>... | |
| ▌ Scanning Archived Wal Directory | |
| StartWal: 000000010000000000000001 | |
| EndWal: 000000010000000000000003 | |
| ▌ Tx Restore Mode [Displayed by Tx groups] | |
| ──────────────────────────────────────── | |
| ▌ End of Scanning, current time range: | |
| Start: 2025-12-09 07:25:01.797475 UTC | |
| End: 2025-12-09 07:26:40.676049 UTC | |
| ▌ Tx Details | |
| ┌─────────────────────────────────────────────────────────┐ | |
| Timestamp: 2025-12-09 07:25:49.755581 UTC | |
| LSN: 0/01F1ECA8 - 0/024258E8 | |
| Recommanded startwal: 000000010000000000000001 | |
| Recommanded endwal: 000000010000000000000002 | |
| -------------------.-------------------- | |
| ● Tx Number: 756 ● Records updated by the TX: 99 | |
| ● Datafiel OID: 16384 ● Toastfile OID: 0 | |
| └─────────────────────────────────────────────────────────┘ | |
| [!] Note: The 'Recommended startwal' indicates the suggested value to set for startwal during transaction recovery | |
| The 'Recommended endwal' mandates that startwal must be set to this value, otherwise recovery may fail | |
| Execution Time 0.12 seconds | |
| ------------- | |
| postgres.public=# scan t; | |
| Scanning updated Records for table<t>... | |
| ▌ Scanning Archived Wal Directory | |
| StartWal: 000000010000000000000001 | |
| EndWal: 000000010000000000000003 | |
| ▌ Tx Restore Mode [Displayed by Tx groups] | |
| ──────────────────────────────────────── | |
| ▌ End of Scanning, current time range: | |
| Start: 2025-12-09 07:25:01.797475 UTC | |
| End: 2025-12-09 07:26:40.676049 UTC | |
| ▌ Tx Details | |
| ┌─────────────────────────────────────────────────────────┐ | |
| Timestamp: 2025-12-09 07:25:49.755581 UTC | |
| LSN: 0/01F1ECA8 - 0/024258E8 | |
| Recommanded startwal: 000000010000000000000001 | |
| Recommanded endwal: 000000010000000000000002 | |
| -------------------.-------------------- | |
| ● Tx Number: 756 ● Records updated by the TX: 99 | |
| ● Datafiel OID: 16384 ● Toastfile OID: 0 | |
| └─────────────────────────────────────────────────────────┘ | |
| [!] Note: The 'Recommended startwal' indicates the suggested value to set for startwal during transaction recovery | |
| The 'Recommended endwal' mandates that startwal must be set to this value, otherwise recovery may fail | |
| Execution Time 0.12 seconds | |
| postgres.public=# restore upd 756; | |
| ▌ Scanning Archived Wal Directory | |
| StartWal: 000000010000000000000001 | |
| EndWal: 000000010000000000000003 | |
| ▌ Tx Restore Mode [Displayed by Tx groups] | |
| ──────────────────────────────────────── | |
| |-Records Parsed: 10invalid max offset number | |
| |-Records Parsed: 11invalid max offset number | |
| |-Records Parsed: 12invalid max offset number | |
| |-Records Parsed: 13invalid max offset number | |
| |-Records Parsed: 14invalid max offset number | |
| |-Records Parsed: 15invalid max offset number | |
| |-Records Parsed: 16invalid max offset number | |
| |-Records Parsed: 17invalid max offset number | |
| |-Records Parsed: 18invalid max offset number | |
| |-Records Parsed: 19invalid max offset number | |
| |-Records Parsed: 20invalid max offset number | |
| |-Records Parsed: 21invalid max offset number | |
| |-Records Parsed: 22invalid max offset number | |
| |-Records Parsed: 23invalid max offset number | |
| |-Records Parsed: 24invalid max offset number | |
| |-Records Parsed: 25invalid max offset number | |
| |-Records Parsed: 26invalid max offset number | |
| |-Records Parsed: 27invalid max offset number | |
| |-Records Parsed: 28invalid max offset number | |
| |-Records Parsed: 29invalid max offset number | |
| |-Records Parsed: 30invalid max offset number | |
| |-Records Parsed: 31invalid max offset number | |
| |-Records Parsed: 32invalid max offset number | |
| |-Records Parsed: 33invalid max offset number | |
| |-Records Parsed: 34invalid max offset number | |
| |-Records Parsed: 35invalid max offset number | |
| |-Records Parsed: 36invalid max offset number | |
| |-Records Parsed: 37invalid max offset number | |
| |-Records Parsed: 38invalid max offset number | |
| |-Records Parsed: 39invalid max offset number | |
| |-Records Parsed: 40invalid max offset number | |
| |-Records Parsed: 41invalid max offset number | |
| |-Records Parsed: 42invalid max offset number | |
| |-Records Parsed: 43invalid max offset number | |
| |-Records Parsed: 44invalid max offset number | |
| |-Records Parsed: 45invalid max offset number | |
| |-Records Parsed: 46invalid max offset number | |
| |-Records Parsed: 47invalid max offset number | |
| |-Records Parsed: 48invalid max offset number | |
| |-Records Parsed: 49invalid max offset number | |
| |-Records Parsed: 50invalid max offset number | |
| |-Records Parsed: 51invalid max offset number | |
| |-Records Parsed: 52invalid max offset number | |
| |-Records Parsed: 53invalid max offset number | |
| |-Records Parsed: 54invalid max offset number | |
| |-Records Parsed: 55invalid max offset number | |
| |-Records Parsed: 56invalid max offset number | |
| |-Records Parsed: 57invalid max offset number | |
| |-Records Parsed: 58invalid max offset number | |
| |-Records Parsed: 59invalid max offset number | |
| |-Records Parsed: 60invalid max offset number | |
| |-Records Parsed: 61invalid max offset number | |
| |-Records Parsed: 62invalid max offset number | |
| |-Records Parsed: 63invalid max offset number | |
| |-Records Parsed: 64invalid max offset number | |
| |-Records Parsed: 65invalid max offset number | |
| |-Records Parsed: 66invalid max offset number | |
| |-Records Parsed: 67invalid max offset number | |
| |-Records Parsed: 68invalid max offset number | |
| |-Records Parsed: 69invalid max offset number | |
| |-Records Parsed: 70invalid max offset number | |
| |-Records Parsed: 71invalid max offset number | |
| |-Records Parsed: 72invalid max offset number | |
| |-Records Parsed: 73invalid max offset number | |
| |-Records Parsed: 74invalid max offset number | |
| |-Records Parsed: 75invalid max offset number | |
| |-Records Parsed: 76invalid max offset number | |
| |-Records Parsed: 77invalid max offset number | |
| |-Records Parsed: 78invalid max offset number | |
| |-Records Parsed: 79invalid max offset number | |
| |-Records Parsed: 80invalid max offset number | |
| |-Records Parsed: 81invalid max offset number | |
| |-Records Parsed: 82invalid max offset number | |
| |-Records Parsed: 83invalid max offset number | |
| |-Records Parsed: 84invalid max offset number | |
| |-Records Parsed: 85invalid max offset number | |
| |-Records Parsed: 86invalid max offset number | |
| |-Records Parsed: 87invalid max offset number | |
| |-Records Parsed: 88invalid max offset number | |
| |-Records Parsed: 89invalid max offset number | |
| |-Records Parsed: 90invalid max offset number | |
| |-Records Parsed: 91invalid max offset number | |
| |-Records Parsed: 92invalid max offset number | |
| |-Records Parsed: 93invalid max offset number | |
| |-Records Parsed: 94invalid max offset number | |
| |-Records Parsed: 95invalid max offset number | |
| |-Records Parsed: 96invalid max offset number | |
| |-Records Parsed: 97invalid max offset number | |
| |-Records Parsed: 98invalid max offset number | |
| |-Records Parsed: 99 | |
| ▌ Restore Complete | |
| ┌───────────────────────────────────────────────────────────────┐ | |
| Table <t> | |
| ● Records Restored: 99 | |
| ● Success: 99 ● Same Values for Updates: 0 ● Failure: 0 | |
| ● File Path: restore/public/t_756.sql | |
| └───────────────────────────────────────────────────────────────┘ | |
| Execution Time 0.88 seconds | |
| ---------------------------------------------------- | |
| data in the csv has the relevant pre update data, but idk the max offset number error ... | |
| ---------------------------------------------------- | |
| --confirm from file on disk , it has that data | |
| postgres@ubuntu:~/PDU-PostgresqlDataUnloader/PDU-PostgresqlDataUnloader-PDU_binary_packages$ head restore/public/t_756.sql | |
| UPDATE t SET AND col3='2025-12-10' | |
| WHERE col1=1 AND col2='2025-12-09 07:26:34.837528+00' AND col3='2025-12-09'; | |
| UPDATE t SET AND col3='2025-12-11' | |
| WHERE col1=2 AND col2='2025-12-09 07:27:34.837528+00' AND col3='2025-12-09'; | |
| UPDATE t SET AND col3='2025-12-12' | |
| WHERE col1=3 AND col2='2025-12-09 07:28:34.837528+00' AND col3='2025-12-09'; | |
| UPDATE t SET AND col3='2025-12-13' | |
| WHERE col1=4 AND col2='2025-12-09 07:29:34.837528+00' AND col3='2025-12-09'; | |
| UPDATE t SET AND col3='2025-12-14' | |
| WHERE col1=5 AND col2='2025-12-09 07:30:34.837528+00' AND col3='2025-12-09'; | |
| ... | |
| ---------------------------------------------------- | |
| now to restore deletes | |
| ---------------------------------------------------- | |
| postgres.public=# p restype delete; | |
| ┌─────────────────────────────────────────────────────────────────┐ | |
| │ parameter │ value │ | |
| ├─────────────────────────────────────────────────────────────────┤ | |
| │ startwal │ │ | |
| │ endwal │ │ | |
| │ startlsnt │ 0/01F1ECA8 │ | |
| │ endlsnt │ 0/024258E8 │ | |
| │ starttime │ │ | |
| │ endtime │ │ | |
| │ resmode(Data Restore Mode) │ TX │ | |
| │ exmode(Data Export Mode) │ CSV │ | |
| │ encoding │ UTF8 │ | |
| │ restype(Data Restore Type) │ DELETE │ | |
| │ isomode │ off │ | |
| └─────────────────────────────────────────────────────────────────┘ | |
| postgres.public=# scan t; | |
| Scanning deleted Records for table<t>... | |
| ▌ Scanning Archived Wal Directory | |
| StartWal: 000000010000000000000001 | |
| EndWal: 000000010000000000000003 | |
| ▌ Tx Restore Mode [Displayed by Tx groups] | |
| ──────────────────────────────────────── | |
| ▌ End of Scanning, current time range: | |
| Start: 2025-12-09 07:25:01.797475 UTC | |
| End: 2025-12-09 07:26:40.676049 UTC | |
| ▌ Tx Details | |
| ┌─────────────────────────────────────────────────────────┐ | |
| Timestamp: 2025-12-09 07:26:04.470004 UTC | |
| LSN: 0/02427B60 - 0/02430FB8 | |
| Recommanded startwal: 000000010000000000000002 | |
| Recommanded endwal: 000000010000000000000002 | |
| -------------------.-------------------- | |
| ● Tx Number: 757 ● Records deleted by the TX: 5 | |
| ● Datafiel OID: 16384 ● Toastfile OID: 0 | |
| └─────────────────────────────────────────────────────────┘ | |
| [!] Note: The 'Recommended startwal' indicates the suggested value to set for startwal during transaction recovery | |
| The 'Recommended endwal' mandates that startwal must be set to this value, otherwise recovery may fail | |
| Execution Time 0.12 seconds | |
| ----------------------------------------- | |
| postgres.public=# restore del 757; | |
| ----------------------------------------- | |
| ▌ Scanning Archived Wal Directory | |
| StartWal: 000000010000000000000001 | |
| EndWal: 000000010000000000000003 | |
| ▌ Tx Restore Mode [Displayed by Tx groups] | |
| ──────────────────────────────────────── | |
| |-Records Parsed: 5 | |
| ▌ Restore Complete | |
| ┌───────────────────────────────────────────────────────────────┐ | |
| Table <t> | |
| ● Records Restored: 5 | |
| ● Success: 5 ● Failure: 0 | |
| ● File Path: restore/public/t_757.csv | |
| └───────────────────────────────────────────────────────────────┘ | |
| Execution Time 0.88 seconds | |
| ------------------- confirm from datafile -- pre delete data recovered from wals | |
| postgres@ubuntu:~/PDU-PostgresqlDataUnloader/PDU-PostgresqlDataUnloader-PDU_binary_packages$ cat restore/public/t_757.csv | |
| 100 2025-12-09 09:05:34.837528+00 2026-03-19 | |
| 200 2025-12-09 10:45:34.837528+00 2026-06-27 | |
| 300 2025-12-09 12:25:34.837528+00 2026-10-05 | |
| 400 2025-12-09 14:05:34.837528+00 2027-01-13 | |
| 500 2025-12-09 15:45:34.837528+00 2027-04-23 | |
| ---------------------------------------------------------------------- | |
| now lets test corruption | |
| ---------------------------------------------------------------------- | |
| postgres@ubuntu:/tmp$ psql | |
| psql (18.1 (Ubuntu 18.1-1.pgdg24.04+2)) | |
| Type "help" for help. | |
| postgres=# create table t(col1 bigint, col2 timestamptz, col3 date); | |
| CREATE TABLE | |
| postgres=# insert into t select x, x * '1 min'::interval + now(), current_date + x * '1 day'::interval from generate_series(1, 1000) x; | |
| INSERT 0 1000 | |
| postgres=# select count(1) from t; | |
| count | |
| ------- | |
| 1000 | |
| (1 row) | |
| postgres=# create extension pageinspect; | |
| CREATE EXTENSION | |
| postgres=# SELECT * FROM page_header(get_raw_page('t', 1)); lsn | checksum | flags | lower | upper | special | pagesize | version | prune_xid | |
| -----------+----------+-------+-------+-------+---------+----------+---------+----------- | |
| 0/177BEF8 | 0 | 0 | 652 | 656 | 8192 | 8192 | 4 | 0 | |
| (1 row) | |
| postgres=# SELECT * FROM page_header(get_raw_page('t', 2)); | |
| lsn | checksum | flags | lower | upper | special | pagesize | version | prune_xid | |
| -----------+----------+-------+-------+-------+---------+----------+---------+----------- | |
| 0/177F038 | 0 | 0 | 652 | 656 | 8192 | 8192 | 4 | 0 | |
| (1 row) | |
| postgres=# SELECT * FROM page_header(get_raw_page('t', 3)); | |
| lsn | checksum | flags | lower | upper | special | pagesize | version | prune_xid | |
| -----------+----------+-------+-------+-------+---------+----------+---------+----------- | |
| 0/1782178 | 0 | 0 | 652 | 656 | 8192 | 8192 | 4 | 0 | |
| (1 row) | |
| postgres=# select pg_relation_filepath('t'); | |
| pg_relation_filepath | |
| ---------------------- | |
| base/5/16384 | |
| (1 row) | |
| postgres=# | |
| \q | |
| postgres@ubuntu:/tmp$ pg_ctl -D db1 -l logfile stop | |
| waiting for server to shut down.... done | |
| server stopped | |
| ## we intentionally corrupt page 2 on disk for table t | |
| postgres@ubuntu:/tmp$ dd if=/dev/urandom seek=2 bs=8k of=db1/base/5/16384 conv=notrunc,noerror count=1 | |
| 1+0 records in | |
| 1+0 records out | |
| 8192 bytes (8.2 kB, 8.0 KiB) copied, 9.7007e-05 s, 84.4 MB/s | |
| ----- lets recover remaining data from table t to file on disk as csv | |
| ----- page 2 corrupted, we need to recover whatever data exists and export to file on disk as csv | |
| postgres@ubuntu:~/PDU-PostgresqlDataUnloader/PDU-PostgresqlDataUnloader-PDU_binary_packages$ ./pdu18 | |
| ╔══════════════════════════════════════════════════════╗ | |
| ║ Copyright 2024-2025 ZhangChen. All rights reserved ║ | |
| ║ PDU: PostgreSQL Data Unloader ║ | |
| ║ Version 3.0.25.12 (2025-12-08) ║ | |
| ║ Valid Until 2124-12-08 10:00:00 ║ | |
| ╚══════════════════════════════════════════════════════╝ | |
| Current DB Supported Version: | |
| ────────────────────────── | |
| • PostgreSQL 18 | |
| ╔═══════════════════════════════════════════╗ | |
| ║ COMMUNITY VERSION ║ | |
| ╠═══════════════════════════════════════════╣ | |
| ║ • Max 100000 Records Per Table (unload) ║ | |
| ║ • Max 100000 Records Per Table (restore) ║ | |
| ║ • Max 1 GB Per Table ║ | |
| ║ • Max 50 Columns Per Table ║ | |
| ║ • Max 500 Object Per Schema ║ | |
| ╚═══════════════════════════════════════════╝ | |
| Contact Me: | |
| ─────────────────── | |
| • WeChat: x1987LJ2020929 | |
| • Email: 1109315180@qq.com | |
| • Tel: 15251853831 | |
| PDU.public=# b; | |
| Initializing... | |
| -pg_database:</tmp/db1/global/1262> | |
| Database:postgres | |
| -pg_schema:</tmp/db1/base/5/2615> | |
| -pg_class:</tmp/db1/base/5/1259> 84 Records | |
| -pg_attribute:</tmp/db1/base/5/1249> 3170 Records | |
| Schema: | |
| ▌ public 1 tables | |
| PDU.public=# use postgres; | |
| ┌────────────────────────────────────────┐ | |
| │ Schema │ Tab Num │ | |
| ├────────────────────────────────────────┤ | |
| │ public │ 1 │ | |
| └────────────────────────────────────────┘ | |
| postgres.public=# unload tab t; | |
| Table <t>. Pages Parsed: 7, Records Parsed: 843 | |
| ▌ Parse Complete | |
| ┌─────────────────────────────────────────────────────────┐ | |
| Table t(/tmp/db1/base/5/16384) | |
| ● Pages: 7 ● 1192 Records in total | |
| ● Success: 843 ● Faliure: 349 | |
| ● File Path: postgres/public/t.csv | |
| └─────────────────────────────────────────────────────────┘ | |
| toast file 0 does not exist, all related records will not be parsed | |
| FAIL PARSING TABLE<t>,please check log log/postgres_public_unload_t_err.txt | |
| Execution Time 0.10 seconds | |
| postgres.public=# \q; | |
| -------------- | |
| we get 843 records from the table t, whatever was readable except page 2 | |
| to verify we zero page 2 on disk for table t and query the table again to confirm 843 records | |
| -------------- | |
| postgres@ubuntu:/tmp$ dd if=/dev/zero seek=2 bs=8k of=db1/base/5/16384 conv=notrunc,noerror count=1 | |
| 1+0 records in | |
| 1+0 records out | |
| 8192 bytes (8.2 kB, 8.0 KiB) copied, 5.5003e-05 s, 149 MB/s | |
| postgres@ubuntu:/tmp$ pg_ctl -D db1 -l logfile start | |
| waiting for server to start.... done | |
| server started | |
| postgres@ubuntu:/tmp$ psql | |
| psql (18.1 (Ubuntu 18.1-1.pgdg24.04+2)) | |
| Type "help" for help. | |
| postgres=# select * from t; | |
| postgres=# select count(1) from t; | |
| count | |
| ------- | |
| 843 | |
| (1 row) | |
| postgres=# | |
| \q | |
| postgres@ubuntu:/tmp$ wc -l /var/lib/postgresql/PDU-PostgresqlDataUnloader/PDU-PostgresqlDataUnloader-PDU_binary_packages/postgres/public/t.csv | |
| 843 /var/lib/postgresql/PDU-PostgresqlDataUnloader/PDU-PostgresqlDataUnloader-PDU_binary_packages/postgres/public/t.csv | |
| postgres@ubuntu:/tmp$ pg_ctl -D db1 -l logfile stop | |
| waiting for server to shut down.... done | |
| server stopped | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment