feat: remove public ip for app and db servers #4402

phot0n · 2026-01-06T21:44:14Z

addresses: #4386

need to implement the nat failover (maybe automatic?) - will do it in a separate pr - have the base for it by using a secondary private ip in this pr itself

codecov · 2026-01-06T21:51:53Z

Codecov Report

❌ Patch coverage is 33.33333% with 108 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.74%. Comparing base (f734da3) to head (d9fce0c).
⚠️ Report is 30 commits behind head on develop.

Files with missing lines	Patch %	Lines
...s/press/doctype/virtual_machine/virtual_machine.py	34.48%	38 Missing ⚠️
press/press/doctype/nat_server/nat_server.py	0.00%	26 Missing ⚠️
press/press/doctype/server/server.py	40.00%	15 Missing ⚠️
press/api/monitoring.py	0.00%	14 Missing ⚠️
...ess/press/doctype/proxy_failover/proxy_failover.py	11.11%	8 Missing ⚠️
press/agent.py	70.00%	6 Missing ⚠️
press/press/doctype/site/site.py	0.00%	1 Missing ⚠️

❌ Your patch check has failed because the patch coverage (33.33%) is below the target coverage (75.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #4402      +/-   ##
===========================================
- Coverage    50.81%   50.74%   -0.07%     
===========================================
  Files          846      848       +2     
  Lines        67670    67842     +172     
  Branches       284      284              
===========================================
+ Hits         34387    34428      +41     
- Misses       33255    33386     +131     
  Partials        28       28

Flag	Coverage Δ
dashboard	`72.89% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

phot0n · 2026-01-07T22:04:44Z

bruh - prolly in tests we dont set public ips? will have to have a proper look

phot0n · 2026-01-22T22:21:56Z

wth did i break lol

* also consider the end server's private ip

… no public ip

…le in get_jobs_id

* also enable the field if no other proxy in the cluster has it enabled

…trollers * also add method for provisioning secondary private ip (for nat server)

* also cleanup the bastion logic by using any proxy server in the cluster

* also consider app and db servers with only private ip for monitoring using proxy server

…ervers

…blic ip or not

…sent)

…ting servers

* chore(agent): adding check for in_test in _get_request_url

…ned public ip

* use proxy server which is active as bastion/jump for ansible

phot0n · 2026-01-07T21:54:22Z

press/agent.py

+		ids = ",".join(agent_job_ids) if isinstance(agent_job_ids, (list, tuple)) else agent_job_ids
+		return self.get(f"agent-jobs/{ids}")


dunno why this started failing just now

phot0n · 2026-01-19T12:03:11Z

press/press/doctype/server/server.py

 								"Type": "A",
 								"TTL": 3600 if self.doctype == "Proxy Server" else 300,
-								"ResourceRecords": [{"Value": self.ip}],
+								"ResourceRecords": [{"Value": self.ip or self.private_ip}],


could be problematic?

phot0n · 2026-01-19T12:05:11Z

press/agent.py

+					"Proxy Server",
+					{"status": "Active", "cluster": cluster, "use_as_proxy_for_agent_and_metrics": 1},
+				)
+


should we error out here if we dont get a proxy?

…lic ip

…ve halt agent jobs checked

phot0n · 2026-01-24T12:08:45Z

press/agent.py

+			server_ip, server_private_ip = frappe.db.get_value(
+				self.server_type, self.server, ("ip", "private_ip")
+			)
+			if not server_ip and server_private_ip and not frappe.flags.in_test:


i dont like that i have to resort to in_test flag for the tests to pass

but in our tests i think we by default dont set any public ip

should i fix the tests? or should i rely on this flag?

maybe i should have another flag?

…ary if set on primary * also fix security group port opening logic only if the provider is AWS

phot0n · 2026-01-26T11:15:56Z

press/playbooks/roles/nat/tasks/main.yml

+- name: Create Netplan configuration for secondary IP
+  copy:
+    dest: /etc/netplan/60-secondary-ip.yaml
+    mode: '0644'
+    content: |
+      network:
+        dhcp4: true
+        ethernets:
+          {{ network_interface.stdout }}:
+            dhcp4: true
+            addresses:
+              - {{ primary_ip }}/16
+              - {{ secondary_ip }}/16
+  when: secondary_ip is defined and secondary_ip
+
+- name: Apply Netplan
+  command: netplan apply
+  when: secondary_ip is defined and secondary_ip


https://easyscale.net/aws-ec2-ubuntu-18-04-secondary-ip-with-netplan/

we can also do sudo ip addr add {{secondary_ip}}/16 dev eth0

mergify bot added the backport-master label Jan 6, 2026

phot0n force-pushed the remove-pub-ip branch from f3743c5 to 30582be Compare January 7, 2026 21:19

phot0n force-pushed the remove-pub-ip branch 2 times, most recently from f157290 to 17100b6 Compare January 22, 2026 18:45

phot0n changed the title ~~feat: remove public ip for app and db servers by default~~ feat: remove public ip for app and db servers Jan 22, 2026

phot0n force-pushed the remove-pub-ip branch 4 times, most recently from e5e848a to ab948ca Compare January 22, 2026 22:02

phot0n added 18 commits January 23, 2026 13:09

feat: remove provisioning public ip for f and m series

a7bc8b4

feat: consider proxy to be jump server for ansible

6c65e2a

* also consider the end server's private ip

feat: barebone support for proxying agent requests for server(s) with…

430eaf9

… no public ip

fix: make job ids comma separated string than passing direct list/tup…

fbf5b6c

…le in get_jobs_id

fix: use proxy_server name as ip in bastion_host

7951c1c

refactor(minor): find proxy using cluster in _get_request_url

27f8d6c

feat: add nat server doctype and setup

f17bf34

feat: add use_as_proxy_for_agent_and_metrics field to proxy server

ba64caf

* also enable the field if no other proxy in the cluster has it enabled

feat(virtual-machine): add nat server bindings in virtual machine con…

13c2e0f

…trollers * also add method for provisioning secondary private ip (for nat server)

fix(server): create dns record for app and db servers with private ip

3a14a40

* also cleanup the bastion logic by using any proxy server in the cluster

fix(monitoring): consider nat server to be monitored as well

eb12a4e

* also consider app and db servers with only private ip for monitoring using proxy server

fix(virtual-machine): get secondary private ip logic

b04af98

chore(nat-server): make secondary private ip field read only

8d765c0

feat(base-server): add nat_gateway_ip property

541bfd1

feat: add nat_iptables ansible role in setup of app, db and unified s…

773fee1

…ervers

fix(ansible): set host port properly

59bfc87

chore: fix linter

82287e3

fix(virtual-machine): reserve 2 private ips for nat series

e2c5ae3

phot0n added 7 commits January 23, 2026 13:09

chore(virtual-machine): add back u series

d7f788e

feat(virtual-machine): add checkbox to when provisioning to assign pu…

84f8edf

…blic ip or not

feat(nat-server): add netplan config to register secondary ip (if pre…

331c892

…sent)

chore: install ip tables persistet regardless

bfe9a21

chore: dont create dns record in test

5eb5388

fix(monitoring): get ip of proxy servers

08d4366

feat(virtual-machine): disassociate auto assigned public ip

df997f3

phot0n force-pushed the remove-pub-ip branch from 4f0a8fa to df997f3 Compare January 23, 2026 07:40

phot0n added 5 commits January 23, 2026 13:13

fix(virtual-machine): set secondary private ip if available when upda…

2bbff1d

…ting servers

feat: action to install nat iptables on app & db server

a64e56f

fix: show the install nat ip tables actions iff public ip is not present

4eb5771

* chore(agent): adding check for in_test in _get_request_url

fix(virtual-machine): force update dns when disassociating auto assig…

cf7450f

…ned public ip

fix: remove steps from nat_iptables ansible role

c99c1a2

* use proxy server which is active as bastion/jump for ansible

phot0n marked this pull request as ready for review January 23, 2026 09:52

phot0n requested review from adityahase and balamurali27 as code owners January 23, 2026 09:52

phot0n commented Jan 23, 2026

View reviewed changes

phot0n added 2 commits January 23, 2026 16:28

fix(site): set nat server ip as outbound ip if server doesn't has pub…

e652b4c

…lic ip

fix(agent): dont consider proxy server for agent proxy calls which ha…

37065fd

…ve halt agent jobs checked

phot0n commented Jan 24, 2026

View reviewed changes

phot0n added 3 commits January 25, 2026 19:22

chore: clean get_private_ip a bit

d8abbaf

fix(proxy-failover): set use_as_proxy_for_agent_and_metrics on second…

7b79f94

…ary if set on primary * also fix security group port opening logic only if the provider is AWS

fix(virtual-machine): assign secondary private ip only to nat instances

d9fce0c

phot0n commented Jan 26, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: remove public ip for app and db servers #4402

feat: remove public ip for app and db servers #4402

phot0n commented Jan 6, 2026 •

edited

Loading

Uh oh!

codecov bot commented Jan 6, 2026 •

edited

Loading

Uh oh!

phot0n commented Jan 7, 2026

Uh oh!

phot0n commented Jan 22, 2026

Uh oh!

phot0n Jan 7, 2026

Uh oh!

phot0n Jan 19, 2026

Uh oh!

phot0n Jan 19, 2026 •

edited

Loading

Uh oh!

phot0n Jan 24, 2026

Uh oh!

phot0n Jan 24, 2026

Uh oh!

phot0n Jan 25, 2026

Uh oh!

phot0n Jan 26, 2026

Uh oh!

phot0n Jan 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		ids = ",".join(agent_job_ids) if isinstance(agent_job_ids, (list, tuple)) else agent_job_ids
		return self.get(f"agent-jobs/{ids}")

feat: remove public ip for app and db servers #4402

Are you sure you want to change the base?

feat: remove public ip for app and db servers #4402

Conversation

phot0n commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

phot0n commented Jan 7, 2026

Uh oh!

phot0n commented Jan 22, 2026

Uh oh!

phot0n Jan 7, 2026

Choose a reason for hiding this comment

Uh oh!

phot0n Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

phot0n Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

phot0n Jan 24, 2026

Choose a reason for hiding this comment

Uh oh!

phot0n Jan 24, 2026

Choose a reason for hiding this comment

Uh oh!

phot0n Jan 25, 2026

Choose a reason for hiding this comment

Uh oh!

phot0n Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

phot0n Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

phot0n commented Jan 6, 2026 •

edited

Loading

codecov bot commented Jan 6, 2026 •

edited

Loading

phot0n Jan 19, 2026 •

edited

Loading