Skip to content

Commit f7bd300

Browse files
pgr0sspgr0ss
authored
Add scripts to show working examples of using pgledger (#57)
These scripts are fully working examples of how to use pgledger. They show how to create accounts, how to create transfers between those accounts, and how to query the resulting data. They show both simple and complicated workflows, such as multi-currency transfers. These scripts serve as documentation and also help guide pgledger design. The output from running these scripts are also stored in the repository so it's easy to read through the examples and see both the SQL commands and what they produce.
1 parent 5d35ff9 commit f7bd300

8 files changed

Lines changed: 440 additions & 6 deletions

File tree

‎.sqlfluff‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,9 @@
44
dialect = postgres
55
max_line_length = 120
66

7+
# Allow "SELECT *"
8+
exclude_rules = ambiguous.column_count
9+
710
# CPU processes to use while linting.
811
# The default is "single threaded" to allow easy debugging, but this
912
# is often undesirable at scale.
@@ -19,6 +22,10 @@ processes = -1
1922
# https://docs.sqlfluff.com/en/stable/configuration/layout.html#implicit-indents
2023
allow_implicit_indents = True
2124

25+
# This says to do "FROM table alias" instead of "FROM table AS alias"
26+
[sqlfluff:rules:aliasing.table]
27+
aliasing = implicit
28+
2229
# Capitalize keywords, types, etc but not our identifiers
2330
[sqlfluff:rules:capitalisation.keywords]
2431
capitalisation_policy = upper
@@ -30,3 +37,8 @@ extended_capitalisation_policy = lower
3037
capitalisation_policy = upper
3138
[sqlfluff:rules:capitalisation.types]
3239
extended_capitalisation_policy = upper
40+
41+
# This fixes the \gset syntax like :'user1_id'
42+
[sqlfluff:layout:type:colon]
43+
spacing_before = touch
44+
spacing_after = touch

‎README.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,10 @@ The implementation is currently in a single file: [pgledger.sql](/pgledger.sql).
1212

1313
## Usage
1414

15+
`pgledger` is primarily a set of functions and views. The ledger is appended via functions (such as creating accounts and transfers) and is queried via views (which present the underlying tables in friendly ways).
16+
17+
For more detailed usage guides, check out the [examples](examples) directory. Each sql file is executable, and its output is stored in the corresponding `.sql.out` file. This file allows you to read the comments, the sql commands, and the output of those commands in one place.
18+
1519
Set up your accounts:
1620

1721
```sql
@@ -179,7 +183,8 @@ Bytes/transfer: 743
179183

180184
## TODO
181185

182-
- Make the transfer and account views include the account names as well as the ids
186+
- Make the transfer view include the currency and account names as well as the ids
187+
- Update the multi-currency example to show the transfers for a conversion
183188
- Make create_transfers function return account balances as well
184189
- Add metadata to accounts and transfers - json column?
185190
- Once we have transfer metadata, update receivables example to show how we can group by payment_id to see when incoming and outgoing don't match

‎examples/basic-example.sql‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
-- This is a fully working example script that shows how to use pgledger
2+
--
3+
-- Note that it uses `\gset` to store sql responses as variables. For example,
4+
-- `\gset foo_` creates variables for each column in the response like
5+
-- `foo_col1`, `foo_col2`, etc. These variables can then be used like
6+
-- `:'foo1_col`.
7+
8+
-- The entire script can be passed to psql. If you are running postgres via the
9+
-- pgledger docker compose, you can run this script with:
10+
--
11+
-- cat basic-example.sql | \
12+
-- docker compose exec --no-TTY postgres psql -U pgledger --echo-queries --no-psqlrc
13+
--
14+
15+
-- We're going to simulate a simple payment flow. First, we create our accounts:
16+
SELECT id FROM pgledger_create_account('user1.external', 'USD') \gset user1_external_
17+
SELECT id FROM pgledger_create_account('user1.receivables', 'USD') \gset user1_receivables_
18+
SELECT id FROM pgledger_create_account('user1.available', 'USD') \gset user1_available_
19+
SELECT id FROM pgledger_create_account('user1.pending_outbound', 'USD') \gset user1_pending_outbound_
20+
21+
-- We can query an account to see what it looks like at the beginning.
22+
SELECT * FROM pgledger_accounts_view
23+
WHERE id =:'user1_external_id';
24+
25+
-- The first step in the flow is a $50 payment is created and we are waiting for funds to arrive:
26+
SELECT * FROM pgledger_create_transfer(:'user1_external_id',:'user1_receivables_id', 50.00);
27+
28+
-- Next, the funds arrive in our account, so we remove them from receivables and make them available:
29+
SELECT * FROM pgledger_create_transfer(:'user1_receivables_id',:'user1_available_id', 50.00);
30+
31+
-- Now, we can query the accounts and see the balances. We aren't waiting on
32+
-- any more funds, so the receivables balance is 0:
33+
SELECT balance FROM pgledger_accounts_view
34+
WHERE id =:'user1_receivables_id';
35+
36+
-- And we can see the entries for the receivables account:
37+
SELECT * FROM pgledger_entries_view
38+
WHERE account_id =:'user1_receivables_id'
39+
ORDER BY account_version;
40+
41+
-- Continuing the example, let's issue a partial refund of the payment. When we
42+
-- issue the refund, we move the money into the pending_outbound account to
43+
-- hold it until we get confirmation that it was sent
44+
SELECT * FROM pgledger_create_transfer(:'user1_available_id',:'user1_pending_outbound_id', 20.00);
45+
46+
-- Once we get confirmation that that refund was sent, We can move the money
47+
-- back to the user's external account (e.g. their credit/debit card). Often,
48+
-- this confirmation will come as a webhook or bank file or similar, so we can
49+
-- record the event time in the confirmation separately from the time we record
50+
-- the ledger transfer (event_at vs created_at):
51+
SELECT *
52+
FROM
53+
pgledger_create_transfer(
54+
:'user1_pending_outbound_id',
55+
:'user1_external_id',
56+
20.00,
57+
event_at => '2025-07-21T12:45:54.123Z'
58+
);
59+
60+
-- Now, we can query the current state. The external account has -$30 ($50
61+
-- payment minus $20 refund) and our account for the user has $30. Nothing is
62+
-- in flight, so the receivables and pending accounts are 0.
63+
SELECT
64+
name,
65+
balance
66+
FROM pgledger_accounts_view
67+
WHERE id IN (:'user1_external_id',:'user1_receivables_id',:'user1_available_id',:'user1_pending_outbound_id');
68+
69+
-- Next, we can simulate an unexpected case. Let's say we initiate a payment
70+
-- for $10 but we only receive $8 (e.g. due to unexpected fees):
71+
SELECT * FROM pgledger_create_transfer(:'user1_external_id',:'user1_receivables_id', 10.00);
72+
SELECT * FROM pgledger_create_transfer(:'user1_receivables_id',:'user1_available_id', 8.00);
73+
74+
-- Now, we can see that our receivables balance is not $0 like we expect:
75+
SELECT balance FROM pgledger_accounts_view
76+
WHERE id =:'user1_receivables_id';
77+
78+
-- And we can look at the entries to figure out what happened:
79+
SELECT * FROM pgledger_entries_view
80+
WHERE account_id =:'user1_receivables_id'
81+
ORDER BY account_version;

‎examples/basic-example.sql.out‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
-- This file contains the sql queries plus their output, but we set the filetype to sql for better syntax highlighting
2+
-- vim: set filetype=sql:
3+
4+
-- This is a fully working example script that shows how to use pgledger
5+
--
6+
-- Note that it uses `\gset` to store sql responses as variables. For example,
7+
-- `\gset foo_` creates variables for each column in the response like
8+
-- `foo_col1`, `foo_col2`, etc. These variables can then be used like
9+
-- `:'foo1_col`.
10+
-- The entire script can be passed to psql. If you are running postgres via the
11+
-- pgledger docker compose, you can run this script with:
12+
--
13+
-- cat basic-example.sql | \
14+
-- docker compose exec --no-TTY postgres psql -U pgledger --echo-queries --no-psqlrc
15+
--
16+
-- We're going to simulate a simple payment flow. First, we create our accounts:
17+
SELECT id FROM pgledger_create_account('user1.external', 'USD') \gset user1_external_
18+
SELECT id FROM pgledger_create_account('user1.receivables', 'USD') \gset user1_receivables_
19+
SELECT id FROM pgledger_create_account('user1.available', 'USD') \gset user1_available_
20+
SELECT id FROM pgledger_create_account('user1.pending_outbound', 'USD') \gset user1_pending_outbound_
21+
-- We can query an account to see what it looks like at the beginning.
22+
SELECT * FROM pgledger_accounts_view
23+
WHERE id =:'user1_external_id';
24+
id | name | currency | balance | version | allow_negative_balance | allow_positive_balance | created_at | updated_at
25+
---------------------------------+----------------+----------+---------+---------+------------------------+------------------------+-------------------------------+-------------------------------
26+
pgla_01K398HBM8EAPBGD23PGR118JA | user1.external | USD | 0 | 0 | t | t | 2025-08-22 16:07:09.702979+00 | 2025-08-22 16:07:09.702979+00
27+
(1 row)
28+
29+
-- The first step in the flow is a $50 payment is created and we are waiting for funds to arrive:
30+
SELECT * FROM pgledger_create_transfer(:'user1_external_id',:'user1_receivables_id', 50.00);
31+
id | from_account_id | to_account_id | amount | created_at | event_at
32+
---------------------------------+---------------------------------+---------------------------------+--------+-------------------------------+-------------------------------
33+
pglt_01K398HBMCECNRPT2S9M0MCE42 | pgla_01K398HBM8EAPBGD23PGR118JA | pgla_01K398HBM9EYB8WNG5B2N031VS | 50.00 | 2025-08-22 16:07:09.706692+00 | 2025-08-22 16:07:09.706692+00
34+
(1 row)
35+
36+
-- Next, the funds arrive in our account, so we remove them from receivables and make them available:
37+
SELECT * FROM pgledger_create_transfer(:'user1_receivables_id',:'user1_available_id', 50.00);
38+
id | from_account_id | to_account_id | amount | created_at | event_at
39+
---------------------------------+---------------------------------+---------------------------------+--------+-------------------------------+-------------------------------
40+
pglt_01K398HBMDFGTRV884F9P83FF4 | pgla_01K398HBM9EYB8WNG5B2N031VS | pgla_01K398HBM9FRTVBHCAXZ37RB85 | 50.00 | 2025-08-22 16:07:09.709565+00 | 2025-08-22 16:07:09.709565+00
41+
(1 row)
42+
43+
-- Now, we can query the accounts and see the balances. We aren't waiting on
44+
-- any more funds, so the receivables balance is 0:
45+
SELECT balance FROM pgledger_accounts_view
46+
WHERE id =:'user1_receivables_id';
47+
balance
48+
---------
49+
0.00
50+
(1 row)
51+
52+
-- And we can see the entries for the receivables account:
53+
SELECT * FROM pgledger_entries_view
54+
WHERE account_id =:'user1_receivables_id'
55+
ORDER BY account_version;
56+
id | account_id | transfer_id | amount | account_previous_balance | account_current_balance | account_version | created_at | event_at
57+
---------------------------------+---------------------------------+---------------------------------+--------+--------------------------+-------------------------+-----------------+-------------------------------+-------------------------------
58+
pgle_01K398HBMDEFFVVH0981XGBTDA | pgla_01K398HBM9EYB8WNG5B2N031VS | pglt_01K398HBMCECNRPT2S9M0MCE42 | 50.00 | 0.00 | 50.00 | 1 | 2025-08-22 16:07:09.706692+00 | 2025-08-22 16:07:09.706692+00
59+
pgle_01K398HBMDFSDBJ43MKBX8FADZ | pgla_01K398HBM9EYB8WNG5B2N031VS | pglt_01K398HBMDFGTRV884F9P83FF4 | -50.00 | 50.00 | 0.00 | 2 | 2025-08-22 16:07:09.709565+00 | 2025-08-22 16:07:09.709565+00
60+
(2 rows)
61+
62+
-- Continuing the example, let's issue a partial refund of the payment. When we
63+
-- issue the refund, we move the money into the pending_outbound account to
64+
-- hold it until we get confirmation that it was sent
65+
SELECT * FROM pgledger_create_transfer(:'user1_available_id',:'user1_pending_outbound_id', 20.00);
66+
id | from_account_id | to_account_id | amount | created_at | event_at
67+
---------------------------------+---------------------------------+---------------------------------+--------+-------------------------------+-------------------------------
68+
pglt_01K398HBMEFHQRQ2CKPRET4M9V | pgla_01K398HBM9FRTVBHCAXZ37RB85 | pgla_01K398HBMAED49QW4QJ6HM4M0B | 20.00 | 2025-08-22 16:07:09.710622+00 | 2025-08-22 16:07:09.710622+00
69+
(1 row)
70+
71+
-- Once we get confirmation that that refund was sent, We can move the money
72+
-- back to the user's external account (e.g. their credit/debit card). Often,
73+
-- this confirmation will come as a webhook or bank file or similar, so we can
74+
-- record the event time in the confirmation separately from the time we record
75+
-- the ledger transfer (event_at vs created_at):
76+
SELECT *
77+
FROM
78+
pgledger_create_transfer(
79+
:'user1_pending_outbound_id',
80+
:'user1_external_id',
81+
20.00,
82+
event_at => '2025-07-21T12:45:54.123Z'
83+
);
84+
id | from_account_id | to_account_id | amount | created_at | event_at
85+
---------------------------------+---------------------------------+---------------------------------+--------+-------------------------------+----------------------------
86+
pglt_01K398HBMFETR9XDE61HQ2NH8Q | pgla_01K398HBMAED49QW4QJ6HM4M0B | pgla_01K398HBM8EAPBGD23PGR118JA | 20.00 | 2025-08-22 16:07:09.711208+00 | 2025-07-21 12:45:54.123+00
87+
(1 row)
88+
89+
-- Now, we can query the current state. The external account has -$30 ($50
90+
-- payment minus $20 refund) and our account for the user has $30. Nothing is
91+
-- in flight, so the receivables and pending accounts are 0.
92+
SELECT
93+
name,
94+
balance
95+
FROM pgledger_accounts_view
96+
WHERE id IN (:'user1_external_id',:'user1_receivables_id',:'user1_available_id',:'user1_pending_outbound_id');
97+
name | balance
98+
------------------------+---------
99+
user1.external | -30.00
100+
user1.receivables | 0.00
101+
user1.available | 30.00
102+
user1.pending_outbound | 0.00
103+
(4 rows)
104+
105+
-- Next, we can simulate an unexpected case. Let's say we initiate a payment
106+
-- for $10 but we only receive $8 (e.g. due to unexpected fees):
107+
SELECT * FROM pgledger_create_transfer(:'user1_external_id',:'user1_receivables_id', 10.00);
108+
id | from_account_id | to_account_id | amount | created_at | event_at
109+
---------------------------------+---------------------------------+---------------------------------+--------+-------------------------------+-------------------------------
110+
pglt_01K398HBMGE7FSH6000VVTHK80 | pgla_01K398HBM8EAPBGD23PGR118JA | pgla_01K398HBM9EYB8WNG5B2N031VS | 10.00 | 2025-08-22 16:07:09.711938+00 | 2025-08-22 16:07:09.711938+00
111+
(1 row)
112+
113+
SELECT * FROM pgledger_create_transfer(:'user1_receivables_id',:'user1_available_id', 8.00);
114+
id | from_account_id | to_account_id | amount | created_at | event_at
115+
---------------------------------+---------------------------------+---------------------------------+--------+-------------------------------+-------------------------------
116+
pglt_01K398HBMGFNWTB8RPDCXDZQAW | pgla_01K398HBM9EYB8WNG5B2N031VS | pgla_01K398HBM9FRTVBHCAXZ37RB85 | 8.00 | 2025-08-22 16:07:09.712689+00 | 2025-08-22 16:07:09.712689+00
117+
(1 row)
118+
119+
-- Now, we can see that our receivables balance is not $0 like we expect:
120+
SELECT balance FROM pgledger_accounts_view
121+
WHERE id =:'user1_receivables_id';
122+
balance
123+
---------
124+
2.00
125+
(1 row)
126+
127+
-- And we can look at the entries to figure out what happened:
128+
SELECT * FROM pgledger_entries_view
129+
WHERE account_id =:'user1_receivables_id'
130+
ORDER BY account_version;
131+
id | account_id | transfer_id | amount | account_previous_balance | account_current_balance | account_version | created_at | event_at
132+
---------------------------------+---------------------------------+---------------------------------+--------+--------------------------+-------------------------+-----------------+-------------------------------+-------------------------------
133+
pgle_01K398HBMDEFFVVH0981XGBTDA | pgla_01K398HBM9EYB8WNG5B2N031VS | pglt_01K398HBMCECNRPT2S9M0MCE42 | 50.00 | 0.00 | 50.00 | 1 | 2025-08-22 16:07:09.706692+00 | 2025-08-22 16:07:09.706692+00
134+
pgle_01K398HBMDFSDBJ43MKBX8FADZ | pgla_01K398HBM9EYB8WNG5B2N031VS | pglt_01K398HBMDFGTRV884F9P83FF4 | -50.00 | 50.00 | 0.00 | 2 | 2025-08-22 16:07:09.709565+00 | 2025-08-22 16:07:09.709565+00
135+
pgle_01K398HBMGEKERD38DNF6C7RKA | pgla_01K398HBM9EYB8WNG5B2N031VS | pglt_01K398HBMGE7FSH6000VVTHK80 | 10.00 | 0.00 | 10.00 | 3 | 2025-08-22 16:07:09.711938+00 | 2025-08-22 16:07:09.711938+00
136+
pgle_01K398HBMGFYVBCTM6YBM8BXN1 | pgla_01K398HBM9EYB8WNG5B2N031VS | pglt_01K398HBMGFNWTB8RPDCXDZQAW | -8.00 | 10.00 | 2.00 | 4 | 2025-08-22 16:07:09.712689+00 | 2025-08-22 16:07:09.712689+00
137+
(4 rows)
138+

‎examples/multi-currency.sql‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
-- This is a fully working example script which demonstrates how to separate
2+
-- accounts by currency and transfer between them.
3+
--
4+
-- Note that it uses `\gset` to store sql responses as variables. For example,
5+
-- `\gset foo_` creates variables for each column in the response like
6+
-- `foo_col1`, `foo_col2`, etc. These variables can then be used like
7+
-- `:'foo1_col`.
8+
9+
-- The entire script can be passed to psql. If you are running postgres via the
10+
-- pgledger docker compose, you can run this script with:
11+
--
12+
-- cat multi-currency.sql | \
13+
-- docker compose exec --no-TTY postgres psql -U pgledger --echo-queries --no-psqlrc
14+
--
15+
16+
-- We're going to simulate a user holding balances in multiple currencies.
17+
-- First, we create an account per currency for the user, since each account is
18+
-- tied to a single currency. One strategy is to use hierarchical account
19+
-- naming to make it clear these accounts are related:
20+
SELECT id FROM pgledger_create_account('user2.usd', 'USD') \gset user2_usd_
21+
SELECT id FROM pgledger_create_account('user2.eur', 'EUR') \gset user2_eur_
22+
23+
-- This style of naming makes it easy to see related accounts:
24+
SELECT * FROM pgledger_accounts_view
25+
WHERE name LIKE 'user2.%';
26+
27+
-- And you can even use PostgreSQL's ltree functionality for querying
28+
-- https://www.postgresql.org/docs/current/ltree.html
29+
CREATE EXTENSION ltree;
30+
31+
SELECT * FROM pgledger_accounts_view
32+
WHERE name::LTREE <@ 'user2';
33+
34+
-- Now, we can see that pgledger prevents transfers between accounts of different currencies:
35+
SELECT * FROM pgledger_create_transfer(:'user2_usd_id',:'user2_eur_id', 10.00);
36+
37+
-- Instead, we need to create liquidity accounts per currency and use those for the transfers:
38+
SELECT id FROM pgledger_create_account('liquidity.usd', 'USD') \gset liquidity_usd_
39+
SELECT id FROM pgledger_create_account('liquidity.eur', 'EUR') \gset liquidity_eur_
40+
41+
-- Now, a currency conversion consist of 2 transfers using the 4 accounts. The
42+
-- difference between these two different amounts (10.00 vs 9.26) is the
43+
-- exchange rate.
44+
SELECT * FROM pgledger_create_transfers(
45+
(:'user2_usd_id',:'liquidity_usd_id', '10.00'),
46+
(:'liquidity_eur_id',:'user2_eur_id', '9.26')
47+
);
48+
49+
-- Note that this used the plural `pgledger_create_transfers` instead of the
50+
-- singular `pgledger_create_transfer` function. It is also possible to call
51+
-- `pgledger_create_transfer` twice in a database transaction, but that is more
52+
-- likely to result in deadlocks since bidrectional transfers will lock the
53+
-- same accounts in reverse order.
54+
55+
-- It is also possible to specify the event_at with `pgledger_create_transfers` using named arguments:
56+
SELECT * FROM pgledger_create_transfers( -- noqa
57+
event_at => '2025-07-21T12:45:54.123Z',
58+
VARIADIC transfer_requests => ARRAY[
59+
(:'user2_usd_id',:'liquidity_usd_id', '10.00'),
60+
(:'liquidity_eur_id',:'user2_eur_id', '9.26')
61+
]::TRANSFER_REQUEST []
62+
);
63+
64+
-- Here is what the transfers look like holistically:
65+
SELECT
66+
t.id,
67+
t.created_at,
68+
t.event_at,
69+
acc_from.name AS acc_from,
70+
acc_to.name AS acc_to,
71+
t.amount
72+
FROM pgledger_transfers_view t
73+
LEFT JOIN pgledger_accounts_view acc_from ON t.from_account_id = acc_from.id
74+
LEFT JOIN pgledger_accounts_view acc_to ON t.to_account_id = acc_to.id
75+
WHERE acc_from.name LIKE 'user2.%' OR acc_from.name LIKE 'liquidity.%';

0 commit comments

Comments
 (0)