Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .circleci/.gitattributes

This file was deleted.

42 changes: 35 additions & 7 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# This file was generated by the excavator check 'excavator/manage-circleci' as specified in .circleci/template.sh.
# To request a modification to the general template, file an issue on Excavator.
# To manually manage the CircleCI configuration for this project, remove the .circleci/template.sh file.

version: 2.1

orbs:
win: circleci/windows@5.1.0

jobs:
build:
build-linux:
docker: [{ image: cimg/rust:1.88.0 }]
environment:
CARGO_BUILD_RUSTFLAGS: -Dwarnings
Expand Down Expand Up @@ -50,6 +50,31 @@ jobs:
--no-verify
done \<<< "$packages"

build-windows:
executor: win/default
steps:
- checkout
- run: choco uninstall rust
- run: choco install -y rustup.install
- run: rustup default 1.88.0
- restore_cache:
key: win-registry
- run: cargo generate-lockfile
- save_cache:
key: win-registry-{{ .BuildNum }}
paths:
- C:\Users\circleci\.cargo\registry\index
- run: rustc --version > ..\rust-version
- restore_cache:
key: win-deps-{{ checksum "~/rust-version" }}-{{ checksum "Cargo.lock" }}
- run: cargo test --all
- save_cache:
key: win-deps-{{ checksum "~/rust-version" }}-{{ checksum "Cargo.lock" }}
paths:
- target
- C:\Users\circleci\.cargo\registry\cache
- C:\Users\circleci\.cargo\registry\src

circle-all:
docker: [{ image: 'busybox:1.36.1@sha256:6d9ac9237a84afe1516540f40a0fafdc86859b2141954b4d643af7066d598b74' }]
resource_class: small
Expand All @@ -60,10 +85,13 @@ workflows:
version: 2
build:
jobs:
- build:
- build-linux:
filters: { tags: { only: /.*/ } }

- build-windows:
filters: { tags: { only: /.*/ } }

# Catch-all for all required checks
- circle-all:
requires: [ build ]
requires: [ build-linux, build-windows ]
filters: { tags: { only: /.*/ } }
3 changes: 0 additions & 3 deletions .circleci/template.sh

This file was deleted.

5 changes: 5 additions & 0 deletions changelog/@unreleased/pr-258.v2.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
type: improvement
improvement:
description: Fix build on Windows
links:
- https://github.com/palantir/witchcraft-rust-server/pull/258
7 changes: 5 additions & 2 deletions witchcraft-server-ete/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ http-body-util = "0.1"
hyper-util = { version = "0.1", features = ["tokio"] }
hyper = "1"
libc = "0.2"
openssl = "0.10"
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Easier to just switch the ETE tests over to using rustls for their client rather than getting openssl built/linked on windows.

rustls-pemfile = "2"
tempfile = "3.20.0"
tokio-openssl = "0.6"
tokio-rustls = { version = "0.26.2", default-features = false, features = ["ring", "tls12"] }

[target."cfg(windows)".dev-dependencies]
windows-sys = { version = "0.60.0", features = ["Win32_System_Console"] }
35 changes: 16 additions & 19 deletions witchcraft-server-ete/tests/ete/server/cert.cer
Original file line number Diff line number Diff line change
@@ -1,21 +1,18 @@
-----BEGIN CERTIFICATE-----
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need a new test cert (copied from conjure-rust-runtime) since rustls doesn't allow CAs to be used as leafs. This is a self-signed non-CA cert.

MIIDhTCCAm2gAwIBAgIJALClJS+cq+ykMA0GCSqGSIb3DQEBCwUAMFkxCzAJBgNV
BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX
aWRnaXRzIFB0eSBMdGQxEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xNjExMjgwNTU5
MjNaFw0yNjExMjYwNTU5MjNaMFkxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21l
LVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQxEjAQBgNV
BAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL4k
A0mghV17MPi033tKh1IK4pM6zpzHFrgi2smU97O/kxvSbNnDoZHEdqq3AtRcjELg
HeKTd02jyAHxGoRTAVORIp0p4LCCvlR7EnHH78e3vIq3lkLe5uqyujnx2NJJIrIX
r5Y+Z3QuMUSAix6GreAb19KcTZG82igvh4dOQP/pmlqQsyrPpioLy50O2NuBqU5Q
xyevFRHsWfe3M7ayzJBVwMpDJxg3saOETXgzMzfKtrj2Pw0mfcHQMtsPv7z85ug0
yyd9iXwwLYx2RqZ6epChsWuY2zj7Zfcis3DzbsrW8/J758KNkjZVWS9aJmDGsT3R
xRlVDnIeow/SWi5qtqECAwEAAaNQME4wHQYDVR0OBBYEFNU1F6I+C06y6rN1yjn0
i/ARufw1MB8GA1UdIwQYMBaAFNU1F6I+C06y6rN1yjn0i/ARufw1MAwGA1UdEwQF
MAMBAf8wDQYJKoZIhvcNAQELBQADggEBACvFmTY+QSrc9EIAtuGk20L4OHrkOoRv
veMIu3PAGbrzjE0rRC1qeLqkqudlWCk+xE6nNe90tB0qyY8AOgj68K2OplrJIhqt
rxJ/Ohtbepwi53Q5npRoib6f9aL+FuT0hnVtVon2ngWRizSdH/CY7vCWuJjTtlon
3J8TGPA1cnj8FtEEfF3ISd0/XCE2oar875FOscf7S0eLnORbuunCVU/RaNn25h/r
9EhvoaPZ6cSZpt7UliMkSt6b07/A2SwU5C19BS1XoqGH02P9OV0pmuJn7N/fOGer
aVbDiPpb+UAUHFUSyu32iK6T2/6OuJS7MQ1cI2biB2SWgWNBTmhRF1s=
MIIC9DCCAdygAwIBAgIBATANBgkqhkiG9w0BAQsFADAvMS0wKwYDVQQDEyRiOTA0
YjdkZi03YzhmLTQzNTYtYWE4MC0zYjM1YjljY2UwZjIwHhcNMTYwMzI5MDM0NjIx
WhcNNDMwODE1MDM0NjIxWjAvMS0wKwYDVQQDEyRiOTA0YjdkZi03YzhmLTQzNTYt
YWE4MC0zYjM1YjljY2UwZjIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB
AQCou8IayAObDgxNvwTJ3tKDJ3GIvWXmG7h9D6JlUDJ4pzHCcX1IH2BLfXMR1UeO
4V9sT8ijmYUviBabd141VS+tOzjdKYrP2QPj/97Mj0CkzAgwIOz2RTyHYJzZLatv
ZlujYVuZvvsb88mzabKhG2+R+vjDf6OV0SBYwAAtLvuZvvlAozRE4tkFgyy7aYmT
76bzdwgZsky6fJKld4DSIaK7/GuKs3KhhZqsp+py063iNHYUBK+2sFb60BEshn7r
EPDitOs8GYf7V4rs0C5FhyFXULEcC6lfX0lvVjonHroDBNSvtpLlz19ok49tGp6X
IdlmqPFL9CJhBWdRKmkkuZeHAgMBAAGjGzAZMBcGA1UdEQEB/wQNMAuCCWxvY2Fs
aG9zdDANBgkqhkiG9w0BAQsFAAOCAQEAOjmWlKY90HebW4+5dOw6RgdLEzc7VY3k
qINGOn0RRrgy98MFGtq2xdjCbQuNcn5IfqmtDR1g6O5xDCdhjJJsZ/mQqFpQC2Tm
cut+A6K4fTyrnVF7Y3XJHzrsfOS0xbeuU1Od2qewGHkKmMk8OB9P2kdI8W1SC3Y/
JU0Pq3RmooZdYCkO3TgpBRT/PSLVV3XMYtEgRBbNrrJfced2fYGHSr7gcW3qhKWt
uCvNDBXpkSqXjNfVPnIbfxX7V4JymQ1V7nGDDiPDH1GI4HkrxWNJzP/zNqCejLk5
6HUYnr1a9WBZe74K3cxfBpz5n+Q6+e2jprJ23CVATuzUdaOyTxkLhw==
-----END CERTIFICATE-----
52 changes: 26 additions & 26 deletions witchcraft-server-ete/tests/ete/server/key.pem
Original file line number Diff line number Diff line change
@@ -1,28 +1,28 @@
-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQC+JANJoIVdezD4
tN97SodSCuKTOs6cxxa4ItrJlPezv5Mb0mzZw6GRxHaqtwLUXIxC4B3ik3dNo8gB
8RqEUwFTkSKdKeCwgr5UexJxx+/Ht7yKt5ZC3ubqsro58djSSSKyF6+WPmd0LjFE
gIsehq3gG9fSnE2RvNooL4eHTkD/6ZpakLMqz6YqC8udDtjbgalOUMcnrxUR7Fn3
tzO2ssyQVcDKQycYN7GjhE14MzM3yra49j8NJn3B0DLbD7+8/OboNMsnfYl8MC2M
dkamenqQobFrmNs4+2X3IrNw827K1vPye+fCjZI2VVkvWiZgxrE90cUZVQ5yHqMP
0louarahAgMBAAECggEAd08brQCHjs/1O6orLS7n2IgyAhZ9fQzD6ckdJi5Oe8Cz
K1sPqFlEMbZoi9iIcv6bmH8O4ZSM4O/rWaSTcgKvq2M/qASWE8wGZ/ZN7Y16nQRi
z1xBcjZyCUUa668g0VrI5Z1NNWZ0/gbaLVTHduEli6GM/H/NgKxS67JfRXzJ9onl
d6vrK+xmeHyA7QSOieEDettaNCvm+HjU8mmOb4F1pCNZktDrch5rI8EzQlmFQuq4
y50YLRZGSlK1QLjzMnT//oaP7mHjN/inzZTHBvTzhU2OjcjzEW7l4ry224Sdu/eH
lhEnNk2eq+mH/yESkn3sJcmH4uYIXh8Dyvcy/uVkPQKBgQDzSC89qxT4sls9n0sL
0DfVhq1D7kEXggD/4wNA714N24N/NWi5BYUDZVh9Kxqy9SuWlFYg1L1ZNZHB02aV
GJdEiFMFgRea2E5NHnhWop+qYPq5N9jD72MHmz/6swX9VGi1p5DqjzK2hWMgoih9
4ky1zxMw+P+aDaQ6xwZF1nr+mwKBgQDIFKTvaJYjqQ/lzRMIPLA3sg6RQ+Mqwt/C
BZ9Oc3DGtuglV8F73i7ML2Ptg0GtVZo3NJgGzMerpNvEoc1pDCuZkzSYitcYysQQ
wsailMQFCv9jJ9g28lSGKlEPYhcLejH8ZRi8jH0fObHIvgr7komNvvPIDFnw/uR8
WsgrloD1cwKBgAdlAkqVkKWehjdxSA6r3YaX+Vw/OatFQFKGy+qFXA5/xZdwQCaf
jFN2GSJ01PLrkM+a4qNM1BSKFEwX6N5PSQnEOwHH0rfaK0cczfuUJdY/7F8E24nZ
FOF+TouINX5lumkLFtSKVbhGhaTQSPrKjhpYmPS8HMjJ8Vv4ALDOvB5RAoGBAJAS
RX3bCpmdCESKOdUplh5UyaaSgsZs0qCsWb0s5R1B4cHaAgnGwF3pFgSWCjndNRHh
fkMPPAv9xv49IGMvD0ojtLDO8Pn6L9p91niFtOyIscNdkpRmRLTjTcFM+ZkbIVlE
Ft7WLtbIPZt2NQRXzVLTGEmJk040zKQ63n58flm/AoGBAKt97WLeHB9S/q0dpEGX
Qk+1BXRAH0/4wK9lNrSeaw+npFr8rNN9K3sIBC/XnOwhT+wbKBpOoBT3PNHbNxVr
EPPQ/pPmZ1TcHc7bszJnZon2S2PFJRDN4601X1/eFoTvakBnLlt1096paaolSmCG
nYED9qXuh2VzUU1GgcqPXgf/
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCou8IayAObDgxN
vwTJ3tKDJ3GIvWXmG7h9D6JlUDJ4pzHCcX1IH2BLfXMR1UeO4V9sT8ijmYUviBab
d141VS+tOzjdKYrP2QPj/97Mj0CkzAgwIOz2RTyHYJzZLatvZlujYVuZvvsb88mz
abKhG2+R+vjDf6OV0SBYwAAtLvuZvvlAozRE4tkFgyy7aYmT76bzdwgZsky6fJKl
d4DSIaK7/GuKs3KhhZqsp+py063iNHYUBK+2sFb60BEshn7rEPDitOs8GYf7V4rs
0C5FhyFXULEcC6lfX0lvVjonHroDBNSvtpLlz19ok49tGp6XIdlmqPFL9CJhBWdR
KmkkuZeHAgMBAAECggEAN75Hwvc/EbOTZv/rQhHF9CRMwsLnbhzMVyyaFuRmuCts
KfO6UuBivK3GHScby90y3CKDg2DNekV4SoYl7nKlHJLLF22I+V+VhHZckbHlxkvN
jLDj9/wpogPN7YXD9DqZnaElVOaeKL7ibM5HgCgDNPP3yQZa8byz9f4AUO0Lm9/M
pkivPbGfefIsbFWGzqAwVMkUnsBUzYqxfXonm+t/Vk/JmBWYf1tm4/k8F07AaCmr
RNoJLzXdiK7lPLAEckx5qLPxr54ku9OXLVbotVzbI6Z6kVGLHzV9kvGwVr+a9FWG
aR+PoYbptnFFRd7X4myq2RbUUwR2KYtTKm8oxiVakQKBgQDkKUoMRomnBeujHXoI
I5wFmiYHWo4JDp8xQqdJ6Qp1KYmFOzFPSNpcYpdwnTiXadVJEFtCjEXMu5NF2yTJ
jiklO2p2Z0wzm/pLAREXrWqppCtdRy9vzdgW31B5APBQZYsY6qLJZ+2cFnBuMkFH
UB952cALSnuEWXy4Yr4dbauFLwKBgQC9Ujc3H6t85XQPt86TBOgCY2Ky1DNeaCGj
vWma+b/LzV2zUMJq/PazY51JUHtkJIP6iCIyFyR/gSCPQdqVO8mcCVDiTWuHmBp9
4Q58Q9XZXw76R4cHOZ8qzQruFjUz+H2RRFawlugfFQsMPaBRJkAdVtrDk4Xeq7ol
cgsQqagtKQKBgDcUVKDkAmUEw97LxeLjHc4oaVfLH5DGK94LdjEchf7EuUMliADY
2IYG8wInojrDPM73mOBW1p7XBXWOsrHJlNs9HbVZuIaY9/JVXpf1sFMlo6ZQAf6e
lbTn0hh0Bo+SA+NO+V9frHfK3vdrQGED0aVr0LWpFGpd5ALrK3QVPpvXAoGAHBfg
I6jqbJ0r7M0stTpMJ1aSJdFBUigHneGUky+uW1K9Lce1IeUpwpLsmx17Qe/9OacL
0/0FKHqD115BTlXR4/qp6vrsEvHqNHW6DYyPZjMp6M/GZLcTPGmyzoOzH2QnVfAb
IRczmLEps4ZNjN0tNG1YBx+UBOfctoPGypEyaIkCgYEAhRysmbkQYyXIS0+ppx60
VV2dAR7nGeZJzS+QKHHg9RafRzc8VSKuTvc46d6bzUmfRpnCN5Som4gXBrmlfS+n
cOxyf+Ob+W6wdoGboqSSLQ+PCbejvSk0EBNskcxiTEaEamTGJFBf4uUKqx7Ablye
qZUs42j7GO3+4bnnyJ/KDt4=
-----END PRIVATE KEY-----
64 changes: 41 additions & 23 deletions witchcraft-server-ete/tests/ete/server/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,22 +19,21 @@ use hyper::body::{Body, Incoming};
use hyper::client::conn::{http1, http2};
use hyper::Request;
use hyper_util::rt::{TokioExecutor, TokioIo};
use openssl::ssl::{SslConnector, SslMethod};
use std::error::{self, Error};
use std::fs::File;
use std::future::Future;
use std::io::{Read, Write};
use std::io::Read;
use std::net::TcpListener;
use std::path::{Path, PathBuf};
use std::pin::Pin;
use std::process::{Child, Command, Stdio};
use std::sync::Arc;
use std::time::Duration;
use std::{env, fs, thread};
use tempfile::TempDir;
use tokio::net::TcpStream;
use tokio::sync::oneshot;
use tokio::{task, time};
use tokio_openssl::SslStream;
use tokio_rustls::rustls::{ClientConfig, KeyLogFile, RootCertStore};
use tokio_rustls::TlsConnector;
use witchcraft_server::logging::api::objects::{AuditLogV3, LogLevel, RequestLogV2, ServiceLogV1};

// this is a bit racy, but should work in practice
Expand Down Expand Up @@ -76,7 +75,7 @@ pub struct Server {
dir: PathBuf,
child: Child,
stdout_rx: Option<oneshot::Receiver<String>>,
ctx: SslConnector,
connector: TlsConnector,
port: u16,
management_port: Option<u16>,
shutdown: bool,
Expand Down Expand Up @@ -124,13 +123,21 @@ impl Server {
.parent()
.unwrap()
.join("../witchcraft-server-ete");
let mut child = Command::new(binary)
let mut command = Command::new(binary);
command
.current_dir(dir.path())
.env("HANDLER_TYPE", handler_type)
.stdout(Stdio::piped())
.stderr(Stdio::inherit())
.spawn()
.unwrap();
.stderr(Stdio::inherit());
#[cfg(windows)]
{
use std::os::windows::process::CommandExt;
use windows_sys::Win32::System::Threading::CREATE_NEW_PROCESS_GROUP;

command.creation_flags(CREATE_NEW_PROCESS_GROUP);
}

let mut child = command.spawn().unwrap();

let mut stdout = child.stdout.take().unwrap();
let (tx, stdout_rx) = oneshot::channel();
Expand All @@ -140,24 +147,28 @@ impl Server {
let _ = tx.send(buf);
});

let mut ctx = SslConnector::builder(SslMethod::tls()).unwrap();
ctx.set_ca_file(dir.path().join("var/security/cert.cer"))
let mut roots = RootCertStore::empty();
let certs = fs::read(dir.path().join("var/security/cert.cer")).unwrap();
let certs = rustls_pemfile::certs(&mut &*certs)
.collect::<Result<Vec<_>, _>>()
.unwrap();
roots.add_parsable_certificates(certs);

let mut client_config = ClientConfig::builder()
.with_root_certificates(roots)
.with_no_client_auth();

if builder.http2 {
ctx.set_alpn_protos(b"\x02h2").unwrap();
client_config.alpn_protocols = vec![b"h2".to_vec()];
}
let file = File::create(dir.path().join("keylog")).unwrap();
ctx.set_keylog_callback(move |_, b| {
let _ = (&file).write_all(b.as_bytes());
let _ = (&file).write_all(b"\n");
});
let ctx = ctx.build();

client_config.key_log = Arc::new(KeyLogFile::new());

let server = Server {
dir: dir.keep(),
child,
stdout_rx: Some(stdout_rx),
ctx,
connector: TlsConnector::from(Arc::new(client_config)),
port,
management_port: builder.management_port,
shutdown: false,
Expand Down Expand Up @@ -226,9 +237,10 @@ impl Server {
B::Error: Into<Box<dyn error::Error + Sync + Send>>,
{
let stream = TcpStream::connect(("127.0.0.1", port)).await?;
let ssl = self.ctx.configure()?.into_ssl("localhost")?;
let mut stream = SslStream::new(ssl, stream)?;
Pin::new(&mut stream).connect().await?;
let stream = self
.connector
.connect("localhost".try_into().unwrap(), stream)
.await?;

if self.http2 {
let (client, connection) = http2::Builder::new(TokioExecutor::new())
Expand Down Expand Up @@ -260,7 +272,13 @@ impl Server {

pub fn start_shutdown(&mut self) {
unsafe {
#[cfg(unix)]
libc::kill(self.child.id() as libc::pid_t, libc::SIGINT);
#[cfg(windows)]
windows_sys::Win32::System::Console::GenerateConsoleCtrlEvent(
windows_sys::Win32::System::Console::CTRL_BREAK_EVENT,
self.child.id() as u32,
);
}
}

Expand Down
20 changes: 11 additions & 9 deletions witchcraft-server/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -84,15 +84,6 @@ staged-builder = "0.2.0"
subtle = "2.5"
sync_wrapper = "1.0"
tempfile = "3.10.1"
tikv-jemalloc-ctl = { version = "0.6", features = [
"stats",
"use_std",
], optional = true }
tikv-jemallocator = { version = "0.6", features = [
"unprefixed_malloc_on_supported_platforms",
"background_threads",
"profiling",
], optional = true }
tokio-rustls = { version = "0.26", default-features = false, features = [
"ring",
"logging",
Expand All @@ -115,6 +106,17 @@ witchcraft-server-config = { version = "6.0.0", path = "../witchcraft-server-con
witchcraft-server-macros = { version = "6.0.0", path = "../witchcraft-server-macros" }
zipkin = "1.0"

[target.'cfg(not(windows))'.dependencies]
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tikv-jemallocator doesn't build on windows. The upstream jemalloc project is also archived, so it might be time to think about switching to mimalloc...

tikv-jemalloc-ctl = { version = "0.6", features = [
"stats",
"use_std",
], optional = true }
tikv-jemallocator = { version = "0.6", features = [
"unprefixed_malloc_on_supported_platforms",
"background_threads",
"profiling",
], optional = true }

[dev-dependencies]
tempfile = "3"
tokio = { version = "1", features = ["test-util"] }
4 changes: 2 additions & 2 deletions witchcraft-server/src/debug/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ use regex::Regex;

pub(crate) mod diagnostic_types;
pub(crate) mod endpoint;
#[cfg(feature = "jemalloc")]
#[cfg(all(feature = "jemalloc", not(windows)))]
pub(crate) mod heap_profile;
#[cfg(feature = "jemalloc")]
#[cfg(all(feature = "jemalloc", not(windows)))]
pub(crate) mod heap_stats;
pub(crate) mod metric_names;
#[cfg(target_os = "linux")]
Expand Down
Loading