Skip to content

Unreal Horde Source Build Documentation with ARM64 Support #813

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
cwwalb wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Unreal Horde Source Build Documentation with ARM64 Support #813

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 141 additions & 1 deletion modules/unreal/horde/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,143 @@ For example configurations, please see the [examples](https://github.com/aws-gam
<!-- TODO -->
<!-- ## Deployment Instructions -->

# Horde Server Source Build Deployment Guide

This guide documents how to build Horde Server from source and use the build as part of your Horde deployment with the Cloud Game Development Toolkit.

## Prerequisites

- **Docker**: Docker Desktop installed and running
- **Git**: For cloning repositories
- **.NET SDK**: .NET 8.0 or later

## Fork and clone the Unreal Engine Github repository

You can follow the steps documented in the [Unreal Engine Source Code](https://github.com/EpicGames/UnrealEngine/tree/release). Be sure to run both Setup and GenerateProjectFiles scripts before proceeding.

## Authenticate to your ECR registry
In a terminal window, run the following command to authenticate to your ECR registry:
```bash
aws ecr get-login-password --region <region> | docker login --username AWS --password-stdin <account-id>.dkr.ecr.<region.>.amazonaws.com
```

## Create a repsoitory in Amazon ECR for storing your Horde Server container image:

In a terminal window, run the following command:
```bash
aws ecr create-repository --repository-name horde-server --image-scanning-configuration scanOnPush=true
```

## Build Horde Server container image from source and publish to Amazon ECR

### For devices using ARM64 architecture

1. In an IDE, open the Horde Server Dockerfile located at `/Engine/Source/Programs/Horde/HordeServer/Dockerfile`.
2. Replace the code between `COPY --from=redis /usr/local/bin/redis-server /usr/local/bin/redis-server` and `COPY Source/Programs/Shared/EpicGames.Core/*.csproj ./Source/Programs/Shared/EpicGames.Core/` with the following:

```dockerfile
# Since the .deb does not install in this image, just download it and extract the static binary
RUN ARCH=$(dpkg --print-architecture) && \
if [ "$ARCH" = "arm64" ]; then \
wget https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/7.0/multiverse/binary-arm64/mongodb-org-server_7.0.4_arm64.deb && \
dpkg -x mongodb-org-server_7.0.4_arm64.deb /tmp/mongodb; \
elif [ "$ARCH" = "i386" ]; then \
wget https://repo.mongodb.org/apt/debian/dists/bookworm/mongodb-org/7.0/main/binary-i386/mongodb-org-server_7.0.4_i386.deb && \
dpkg -x mongodb-org-server_7.0.4_i386.deb /tmp/mongodb; \
else \
wget https://repo.mongodb.org/apt/debian/dists/bookworm/mongodb-org/7.0/main/binary-amd64/mongodb-org-server_7.0.4_amd64.deb && \
dpkg -x mongodb-org-server_7.0.4_amd64.deb /tmp/mongodb; \
fi && \
cp /tmp/mongodb/usr/bin/mongod /usr/local/bin/mongod
```
3. Locate the code block with the comment `# Remove native libs not used on Linux x86_64`. We need to modify this so that the build has the libraries associated with the operating system and CPU architecture we are using. To do this, replace that block with the following:
```dockerfile
# Remove native libs not used for current architecture
RUN ARCH=$(dpkg --print-architecture) && \
if [ "$ARCH" = "amd64" ]; then \
rm -rf /app/out/runtimes/osx* && \
rm -rf /app/out/runtimes/win-x86 && \
rm -rf /app/out/runtimes/win-arm* && \
rm -rf /app/out/runtimes/linux-arm* && \
rm -rf /app/out/runtimes/linux-x86 && \
rm -rf /app/out/runtimes/linux/native/libgrpc_csharp_ext.x86.so; \
elif [ "$ARCH" = "arm64" ]; then \
rm -rf /app/out/runtimes/osx-x64 && \
rm -rf /app/out/runtimes/win-x64 && \
rm -rf /app/out/runtimes/win-x86 && \
rm -rf /app/out/runtimes/linux-x64* && \
rm -rf /app/out/runtimes/linux-x86 && \
rm -rf /app/out/runtimes/linux/native/libgrpc_csharp_ext.x64.so && \
rm -rf /app/out/runtimes/linux/native/libgrpc_csharp_ext.x86.so; \
elif [ "$ARCH" = "i386" ]; then \
rm -rf /app/out/runtimes/osx* && \
rm -rf /app/out/runtimes/win-x64 && \
rm -rf /app/out/runtimes/win-arm* && \
rm -rf /app/out/runtimes/linux-x64* && \
rm -rf /app/out/runtimes/linux-arm* && \
rm -rf /app/out/runtimes/linux/native/libgrpc_csharp_ext.x64.so; \
fi
```
## Update server.json (Optional)


## Update the Horde BuildGraph

1. In an IDE, open the file `Engine/Source/Programs/Horde/BuildHorde.xml `.
2. Create a new build target to build the Hord Server with the Dashboard and push the image to your Amazon ECR repository. Be sure to replace the values for `<account-id>` and `<region>` with your own values.
```xml
<Node Name="Build and Publish HordeServer to ECR" Requires="Build HordeServer;Build HordeDashboard">
<!-- Create a new image by combining server and dashboard image into one -->
<Docker-Build BaseDir="Engine/Source/Programs/Horde/HordeServer" Files="Dockerfile*" UseBuildKit="true" Tag="horde-server" DockerFile="Engine/Source/Programs/Horde/HordeServer/Dockerfile.dashboard" />

<!-- Publish the docker image to Amazon ECR -->
<Docker-Push Repository="<account-id>.dkr.ecr.<region>.amazonaws.com" Image="horde-server" TargetImage="horde/horde-server:$(Version)" AwsEcr="True" />
</Node>
```

4. In your terminal window, and navigate to `Engine/Build/BatchFiles`.
```bash
cd Engine/Build/BatchFiles
```

6. Run the following command to build the Horde Server and Dashboard, then publish your docker image to Amazon ECR.
### Windows
```powershell
RunUAT.bat BuildGraph -Script=Engine/Source/Programs/Horde/BuildHorde.xml -Target="Build and Publish HordeServer to ECR"
```

### Linux and Mac
```bash
./RunUAT.sh BuildGraph -Script=Engine/Source/Programs/Horde/BuildHorde.xml -Target="Build and Publish HordeServer to ECR"
```

7. Note the URI of the image that was published to Amazon ECR. It should be in the format of `<account-id>.dkr.ecr.<region>.amazonaws.com/horde/horde-server:<version>`

8. You can also locate the URI of the image in the AWS Console under Amazon ECR, or using the AWS CLI command:
```bash
aws ecr describe-images --repository-name horde-server
```

## Update your Cloud Game Development Toolkit configuration

In order to use your newly created container image as your Horde Server image, you will need to specify the URI of the image as the value for the `image` variable. For this example we will be using the example deployment located at `modules/unreal/horde/examples/complete`
1. In your IDE, open the main.tf located at `modules/unreal/horde/examples/complete/main.tf`.
2. Under the module declaration `"unreal_engine_horde"` add the following line with the value of the image URI you noted earlier. Be sure to also set the value of `is_source_build` to `true` and set the value of `horde_server_architecture` to `ARM64` or `X_86` by passing the value with the `var.horde_server_architecture` variable as shown below:
```python
module "unreal_engine_horde" {
...
image = "<image-uri>"
is_source_build = true
horde_server_architecture = "<horde-server-architecture>"
...
}
```
3. In a new terminal window, navigate to the directory `modules/unreal/horde/examples/complete` and run the following commands:
```bash
terraform init
terraform apply
```

<!-- BEGIN_TF_DOCS -->
## Requirements

Expand Down Expand Up @@ -189,18 +326,21 @@ No modules.
| <a name="input_environment"></a> [environment](#input\_environment) | The current environment (e.g. Development, Staging, Production, etc.). This will tag ressources and set ASPNETCORE\_ENVIRONMENT variable. | `string` | `"Development"` | no |
| <a name="input_existing_security_groups"></a> [existing\_security\_groups](#input\_existing\_security\_groups) | A list of existing security group IDs to attach to the Unreal Horde load balancer. | `list(string)` | `[]` | no |
| <a name="input_github_credentials_secret_arn"></a> [github\_credentials\_secret\_arn](#input\_github\_credentials\_secret\_arn) | A secret containing the Github username and password with permissions to the EpicGames organization. | `string` | `null` | no |
| <a name="input_horde_server_architecture"></a> [horde\_server\_architecture](#input\_horde\_server\_architecture) | The CPU architecture for Horde server container. Valid values: x86 or arm64 | `string` | `"X86_64"` | no |
| <a name="input_image"></a> [image](#input\_image) | The Horde Server image to use in the ECS service. | `string` | `"ghcr.io/epicgames/horde-server:latest-bundled"` | no |
| <a name="input_is_source_build"></a> [is\_source\_build](#input\_is\_source\_build) | Set this flag to true if you are using a custom built Horde Server image from source. | `bool` | `false` | no |
| <a name="input_name"></a> [name](#input\_name) | The name attached to Unreal Engine Horde module resources. | `string` | `"unreal-horde"` | no |
| <a name="input_oidc_audience"></a> [oidc\_audience](#input\_oidc\_audience) | The audience used for validating externally issued tokens. | `string` | `null` | no |
| <a name="input_oidc_authority"></a> [oidc\_authority](#input\_oidc\_authority) | The authority for the OIDC authentication provider used. | `string` | `null` | no |
| <a name="input_oidc_client_id"></a> [oidc\_client\_id](#input\_oidc\_client\_id) | The client ID used for authenticating with the OIDC provider. | `string` | `null` | no |
| <a name="input_oidc_client_secret"></a> [oidc\_client\_secret](#input\_oidc\_client\_secret) | The client secret used for authenticating with the OIDC provider. | `string` | `null` | no |
| <a name="input_oidc_signin_redirect"></a> [oidc\_signin\_redirect](#input\_oidc\_signin\_redirect) | The sign-in redirect URL for the OIDC provider. | `string` | `null` | no |
| <a name="input_operating_system"></a> [operating\_system](#input\_operating\_system) | The operating system for the Horde server container. Valid values: linux or windows | `string` | `"WINDOWS_SERVER_2019_CORE"` | no |
| <a name="input_p4_port"></a> [p4\_port](#input\_p4\_port) | The Perforce server to connect to. | `string` | `null` | no |
| <a name="input_p4_super_user_password_secret_arn"></a> [p4\_super\_user\_password\_secret\_arn](#input\_p4\_super\_user\_password\_secret\_arn) | Optionally provide the ARN of an AWS Secret for the p4d super user password. | `string` | `null` | no |
| <a name="input_p4_super_user_username_secret_arn"></a> [p4\_super\_user\_username\_secret\_arn](#input\_p4\_super\_user\_username\_secret\_arn) | Optionally provide the ARN of an AWS Secret for the p4d super user username. | `string` | `null` | no |
| <a name="input_project_prefix"></a> [project\_prefix](#input\_project\_prefix) | The project prefix for this workload. This is appeneded to the beginning of most resource names. | `string` | `"cgd"` | no |
| <a name="input_tags"></a> [tags](#input\_tags) | Tags to apply to resources. | `map(any)` | <pre>{<br> "iac-management": "CGD-Toolkit",<br> "iac-module": "unreal-horde",<br> "iac-provider": "Terraform"<br>}</pre> | no |
| <a name="input_tags"></a> [tags](#input\_tags) | Tags to apply to resources. | `map(any)` | <pre>{<br/> "iac-management": "CGD-Toolkit",<br/> "iac-module": "unreal-horde",<br/> "iac-provider": "Terraform"<br/>}</pre> | no |
| <a name="input_unreal_horde_alb_access_logs_bucket"></a> [unreal\_horde\_alb\_access\_logs\_bucket](#input\_unreal\_horde\_alb\_access\_logs\_bucket) | ID of the S3 bucket for Unreal Horde ALB access log storage. If access logging is enabled and this is null the module creates a bucket. | `string` | `null` | no |
| <a name="input_unreal_horde_alb_access_logs_prefix"></a> [unreal\_horde\_alb\_access\_logs\_prefix](#input\_unreal\_horde\_alb\_access\_logs\_prefix) | Log prefix for Unreal Horde ALB access logs. If null the project prefix and module name are used. | `string` | `null` | no |
| <a name="input_unreal_horde_cloudwatch_log_retention_in_days"></a> [unreal\_horde\_cloudwatch\_log\_retention\_in\_days](#input\_unreal\_horde\_cloudwatch\_log\_retention\_in\_days) | The log retention in days of the cloudwatch log group for Unreal Horde. | `string` | `365` | no |
Expand Down
25 changes: 16 additions & 9 deletions modules/unreal/horde/ecs.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,20 +31,27 @@ resource "aws_ecs_task_definition" "unreal_horde_task_definition" {
cpu = var.container_cpu
memory = var.container_memory

runtime_platform {
operating_system_family = var.operating_system
cpu_architecture = var.horde_server_architecture
}

volume {
name = "unreal-horde-config"
}

container_definitions = jsonencode(concat([
{
name = var.container_name
image = var.image
repositoryCredentials = var.github_credentials_secret_arn != null ? {
"credentialsParameter" : var.github_credentials_secret_arn
} : null
merge({
name = var.container_name
image = var.image
cpu = var.container_cpu
memory = var.container_memory
essential = true
}, var.github_credentials_secret_arn != null && !var.is_source_build ? {
repositoryCredentials = {
"credentialsParameter" : var.github_credentials_secret_arn
}
} : {}, {
Comment on lines +50 to +54
Copy link
Contributor

@hwkiem hwkiem Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could move some of this logic to a local

portMappings = [
{
containerPort = var.container_api_port
Expand Down Expand Up @@ -111,10 +118,10 @@ resource "aws_ecs_task_definition" "unreal_horde_task_definition" {
condition = "SUCCESS"
}] : []
)
},
}),
{
name = "unreal-horde-docdb-cert",
image = "public.ecr.aws/docker/library/bash:5.3",
image = "public.ecr.aws/docker/library/bash:latest",
Copy link
Contributor

@hwkiem hwkiem Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is probably safer to pin to a specific version of bash. :latest opens us up to upstream issues

Copy link
Contributor

@hwkiem hwkiem Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know we do this elsewhere... probably should open an issue and fix those too.

essential = false
command = ["wget", "https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem", "-P", "/app/config/"]
readonly_root_filesystem = false
Expand All @@ -135,7 +142,7 @@ resource "aws_ecs_task_definition" "unreal_horde_task_definition" {
}],
local.need_p4_trust ? [{
name = "unreal-horde-p4-trust",
image = "ubuntu:noble"
image = "public.ecr.aws/ubuntu/ubuntu:noble"
Copy link
Contributor

@hwkiem hwkiem Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice fix. I wonder if we should also pin this release to a particular version for the same reason as above.

essential = false
command = ["bash", "-exc", <<-EOF
apt-get update
Expand Down
26 changes: 26 additions & 0 deletions modules/unreal/horde/variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,32 @@ variable "image" {
default = "ghcr.io/epicgames/horde-server:latest-bundled"
}

variable "is_source_build" {
type = bool
description = "Set this flag to true if you are using a custom built Horde Server image from source."
default = false
}
Comment on lines +63 to +67
Copy link
Contributor

@hwkiem hwkiem Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to know if its a source build?

Copy link
Contributor Author

@cwwalb cwwalb Dec 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added that because in ecs.tf we have the current steps to use github credentials as part of the container definition to pull the public horde docker image. I didn't want to break the functionality so created the variable so the user can still use the pre-built docker image if they choose or go for the source build

Copy link
Contributor

@hwkiem hwkiem Dec 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if they provide an image as a variable that accomplishes the same thing right?

  1. Customer specifies "image" and we use it. Credentials are not required.
  2. Customer does not specify image or arch. We use the latest version of horde published and require GHCR credentials?


variable "horde_server_architecture" {
type = string
description = "The CPU architecture for Horde server container. Valid values: x86 or arm64"
default = "X86_64"
validation {
condition = contains(["X86_64", "ARM64"], var.horde_server_architecture)
error_message = "horde_server_architecture must be either 'X_86' or 'ARM64'"
}
}
Comment on lines +69 to +77
Copy link
Contributor

@hwkiem hwkiem Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we determine architecture via data block and a lookup against the image?


variable "operating_system" {
type = string
description = "The operating system for the Horde server container. Valid values: linux or windows"
default = "WINDOWS_SERVER_2019_CORE"
validation {
condition = contains(["LINUX", "WINDOWS_SERVER_2025_FULL", "WINDOWS_SERVER_2025_CORE", "WINDOWS_SERVER_2022_FULL", "WINDOWS_SERVER_2022_CORE", "WINDOWS_SERVER_2019_FULL", "WINDOWS_SERVER_2019_CORE"], var.operating_system)
error_message = "Operating_system must be either LINUX, WINDOWS_SERVER_2025_FULL, WINDOWS_SERVER_2025_CORE, WINDOWS_SERVER_2022_FULL, WINDOWS_SERVER_2022_CORE, WINDOWS_SERVER_2019_FULL, and WINDOWS_SERVER_2019_CORE."
}
}
Comment on lines +79 to +87
Copy link
Contributor

@hwkiem hwkiem Dec 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also this. I am not sure why we want to support a windows OS for Horde unless a customer explicitly asks for it. Open to your thoughts on this!

Copy link
Contributor Author

@cwwalb cwwalb Dec 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was something I wanted to ask. What are the most commonly used OS for Horde?

Copy link
Contributor

@hwkiem hwkiem Dec 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should really only support Linux distributions. I don't think this is required for containerized deployments...


variable "cluster_name" {
type = string
description = "The name of the cluster to deploy the Unreal Horde into. Defaults to null and a cluster will be created."
Expand Down