Skip to content

Makes port optional in the URL for serverless compute#1

Closed
jacobphillips99 wants to merge 23 commits intozhouzypaul:mainfrom
jacobphillips99:main
Closed

Makes port optional in the URL for serverless compute#1
jacobphillips99 wants to merge 23 commits intozhouzypaul:mainfrom
jacobphillips99:main

Conversation

@jacobphillips99
Copy link
Copy Markdown

@jacobphillips99 jacobphillips99 commented Apr 16, 2025

Currently the implementation for hosting server-side models relies on binding to a "port" number such as 8000. However, if we want to use cloud-based compute orchestration platforms like Modal, there's no persistent port as the provider handles all the networking. We want to make the port number optional for the URL.

In auto_eval Job SQLModel, policy_server_port is cast as an integer type. In order to prevent database migration, we use -1 as a sentinel value to avoid adding the port to the URL.

See similar PR in auto_eval repo: zhouzypaul/auto_eval#2

@zhouzypaul
Copy link
Copy Markdown
Owner

Thanks for the PR! Merged in zhouzypaul/auto_eval#2

Closing this for now as it seems to have more updates than the intended PR. Feel free to open another with the relevant commits!

@zhouzypaul zhouzypaul closed this May 21, 2025
@jacobphillips99
Copy link
Copy Markdown
Author

apologies! forgot to close after switching base. Thanks for closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants