Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
20 changes: 11 additions & 9 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
## [project-title] Changelog
# Observability Accelerators Changelog

<a name="x.y.z"></a>
# x.y.z (yyyy-mm-dd)
## 1.0 (2023-05-19)

*Features*
* ...
### Features

*Bug Fixes*
* ...
- Initial public release

*Breaking Changes*
* ...
### Bug Fixes

- Not applicable

### Breaking Changes

- Not applicable
56 changes: 31 additions & 25 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Contributing to [project-title]
# Contributing to Observability Accelerators

This project welcomes contributions and suggestions. Most contributions require you to agree to a
This project welcomes contributions and suggestions. Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

Expand All @@ -12,61 +12,67 @@ This project has adopted the [Microsoft Open Source Code of Conduct](https://ope
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.

- [Code of Conduct](#coc)
- [Issues and Bugs](#issue)
- [Feature Requests](#feature)
- [Submission Guidelines](#submit)
- [Code of Conduct](#coc)
- [Issues and Bugs](#issue)
- [Feature Requests](#feature)
- [Submission Guidelines](#submit)

## <a name="coc"></a> Code of Conduct

Help us keep this project open and inclusive. Please read and follow our [Code of Conduct](https://opensource.microsoft.com/codeofconduct/).

## <a name="issue"></a> Found an Issue?

If you find a bug in the source code or a mistake in the documentation, you can help us by
[submitting an issue](#submit-issue) to the GitHub Repository. Even better, you can
[submit a Pull Request](#submit-pr) with a fix.

## <a name="feature"></a> Want a Feature?
You can *request* a new feature by [submitting an issue](#submit-issue) to the GitHub
Repository. If you would like to *implement* a new feature, please submit an issue with

You can _request_ a new feature by [submitting an issue](#submit-issue) to the GitHub
Repository. If you would like to _implement_ a new feature, please submit an issue with
a proposal for your work first, to be sure that we can use it.

* **Small Features** can be crafted and directly [submitted as a Pull Request](#submit-pr).
- **Small Features** can be crafted and directly [submitted as a Pull Request](#submit-pr).

## <a name="submit"></a> Submission Guidelines

### <a name="submit-issue"></a> Submitting an Issue

Before you submit an issue, search the archive, maybe your question was already answered.

If your issue appears to be a bug, and hasn't been reported, open a new issue.
Help us to maximize the effort we can spend fixing issues and adding new
features, by not reporting duplicate issues. Providing the following information will increase the
features, by not reporting duplicate issues. Providing the following information will increase the
chances of your issue being dealt with quickly:

* **Overview of the Issue** - if an error is being thrown a non-minified stack trace helps
* **Version** - what version is affected (e.g. 0.1.2)
* **Motivation for or Use Case** - explain what are you trying to do and why the current behavior is a bug for you
* **Browsers and Operating System** - is this a problem with all browsers?
* **Reproduce the Error** - provide a live example or a unambiguous set of steps
* **Related Issues** - has a similar issue been reported before?
* **Suggest a Fix** - if you can't fix the bug yourself, perhaps you can point to what might be
- **Overview of the Issue** - if an error is being thrown a non-minified stack trace helps
- **Version** - what version is affected (e.g. 0.1.2)
- **Motivation for or Use Case** - explain what are you trying to do and why the current behavior is a bug for you
- **Browsers and Operating System** - is this a problem with all browsers?
- **Reproduce the Error** - provide a live example or a unambiguous set of steps
- **Related Issues** - has a similar issue been reported before?
- **Suggest a Fix** - if you can't fix the bug yourself, perhaps you can point to what might be
causing the problem (line of code or commit)

You can file new issues by providing the above information at the corresponding repository's issues link: https://github.com/[organization-name]/[repository-name]/issues/new].

### <a name="submit-pr"></a> Submitting a Pull Request (PR)

Before you submit your Pull Request (PR) consider the following guidelines:

* Search the repository (https://github.com/[organization-name]/[repository-name]/pulls) for an open or closed PR
- Search the repository (https://github.com/[organization-name]/[repository-name]/pulls) for an open or closed PR
that relates to your submission. You don't want to duplicate effort.

* Make your changes in a new git fork:
- Make your changes in a new git fork:

- Commit your changes using a descriptive commit message
- Push your fork to GitHub:
- In GitHub, create a pull request
- If we suggest changes then:

* Commit your changes using a descriptive commit message
* Push your fork to GitHub:
* In GitHub, create a pull request
* If we suggest changes then:
* Make the required updates.
* Rebase your fork and force push to your GitHub repository (this will update your Pull Request):
- Make the required updates.
- Rebase your fork and force push to your GitHub repository (this will update your Pull Request):

```shell
git rebase master -i
Expand Down
60 changes: 10 additions & 50 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,57 +1,17 @@
# Project Name
# Observability Accelerators

(short, 1-3 sentenced, description of the project)
This repository contains multiple samples that are meant to accelerate development in the Observability and Monitoring space on Azure.

## Features
Each accelerator focuses on a different application architecture. They contain all source code and infrastructure as code necessary to deploy the application, as well as in-depth documentation that details important O&M concepts.

This project framework provides the following features:
Navigate to one of the accelerators in the list below. The README will include instructions on how to get started with that application.

* Feature 1
* Feature 2
* ...
## Accelerator Index

## Getting Started
| Accelerator |
| -------------------------------------------------------------------------------------------------------------------------- |
| [Azure Monitor in a Message-Based Distributed Application on AKS](./accelerators/aks-sb-azmonitor-microservices/README.md) |

### Prerequisites
## Trademarks

(ideally very short, if any)

- OS
- Library version
- ...

### Installation

(ideally very short)

- npm install [package name]
- mvn install
- ...

### Quickstart
(Add steps to get up and running quickly)

1. git clone [repository clone url]
2. cd [repository name]
3. ...


## Demo

A demo app is included to show how to use the project.

To run the demo, follow these steps:

(Add steps to start up the demo)

1.
2.
3.

## Resources

(Any additional resources or related projects)

- Link to supporting information
- Link to similar sample
- ...
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft’s Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party’s policies.
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# See here for image contents: https://github.com/microsoft/vscode-dev-containers/tree/v0.245.2/containers/ubuntu/.devcontainer/base.Dockerfile

# [Choice] Ubuntu version (use ubuntu-22.04 or ubuntu-18.04 on local arm64/Apple Silicon): ubuntu-22.04, ubuntu-20.04, ubuntu-18.04
ARG VARIANT="jammy"
FROM mcr.microsoft.com/vscode/devcontainers/base:0-${VARIANT}

# [Optional] Uncomment this section to install additional OS packages.
RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install --no-install-recommends figlet


ARG USERNAME=vscode
USER $USERNAME

COPY kubelogin.sh /tmp/kubelogin.sh
RUN mkdir -p "/home/$USERNAME/.local/bin" && \
/tmp/kubelogin.sh
ENV PATH="/home/vscode/.local/bin:${PATH}"
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.245.2/containers/ubuntu
{
"name": "aks-sb-azmonitor-microservices",
"build": {
"dockerfile": "Dockerfile",
// Update 'VARIANT' to pick an Ubuntu version: jammy / ubuntu-22.04, focal / ubuntu-20.04, bionic /ubuntu-18.04
// Use ubuntu-22.04 or ubuntu-18.04 on local arm64/Apple Silicon.
"args": {
"VARIANT": "ubuntu-22.04"
}
},
// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],
// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": "uname -a",
// Comment out to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
"remoteUser": "vscode",
"features": {
"ghcr.io/devcontainers/features/terraform:1": {
"version": "1.3"
},
"ghcr.io/devcontainers/features/azure-cli:1": {},
"ghcr.io/stuartleeks/dev-container-features/azure-cli-persistence:0": {},
"ghcr.io/stuartleeks/dev-container-features/shell-history:0": {},
"ghcr.io/devcontainers/features/docker-from-docker:1": {},
"ghcr.io/devcontainers/features/kubectl-helm-minikube:1": {
"helm": "3.10.1"
}
},
"runArgs": [
// Attach dev container to host network so allow accessing services on the host
// when running via docker-compose
"--network", "host"
],
"customizations": {
"vscode": {
"extensions": [
"timonwong.shellcheck",
"hashicorp.terraform",
"ms-azuretools.vscode-bicep",
"humao.rest-client"
]
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/bash
set -e

wget -O /tmp/kubelogin-linux-amd64.zip \
https://github.com/Azure/kubelogin/releases/download/v0.0.24/kubelogin-linux-amd64.zip

unzip /tmp/kubelogin-linux-amd64.zip -d /tmp/kubelogin

cp /tmp/kubelogin/bin/linux_amd64/kubelogin "/home/$USERNAME/.local/bin/kubelogin"
8 changes: 8 additions & 0 deletions accelerators/aks-sb-azmonitor-microservices/.env.sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Unique name to assign in all deployed services, your high school hotmail alias is a great idea!
USERNAME=

# Email address for alert notifications
EMAIL_ADDRESS=

# Uncomment the following line to change the deployment location
# LOCATION=westeurope
6 changes: 6 additions & 0 deletions accelerators/aks-sb-azmonitor-microservices/.gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Ensure that all shell scripts are checked out with LF line endings
# on Windows. This is necessary because Git for Windows defaults to
# CRLF line endings, which breaks the shell scripts.
# NOTE: for best results on Windows, clone the code in a in a file system
# under Windows Subsystem for Linux (WSL) - see https://www.docker.com/blog/docker-desktop-wsl-2-best-practices/
*.sh text eol=lf
7 changes: 7 additions & 0 deletions accelerators/aks-sb-azmonitor-microservices/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
plan.out
terraform.tfvars
azuredeploy.parameters.json
.env

output.json
env.yaml
34 changes: 34 additions & 0 deletions accelerators/aks-sb-azmonitor-microservices/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Azure Monitor in a Message Based Distributed Application

Using Azure Monitor to observe a distributed application comes with unique challenges and considerations. The ability to generate and view traces, ensure service availability, use custom telemetry to track business critical indicators, etc. are all more complex in a distributed environment. The sample is designed to demonstrate how to automatically and manually instrument data in variety of languages within a distributed application, as well as provide similar examples for visualization and alerts based on this incoming data.

The sample contains a conceptual cargo processing application to demonstrate these points. The microservice-based solution is deployed to Azure Kubernetes Service and employs multiple communication protocols, including HTTP and message-based interactions, to enable seamless communication between its services. The services cover a wide variety of programming languages and instrumentation libraries - the Java services utilize OpenTelemetry exporters, while the Node, .NET, and Python services use the Application Insights SDKs for instrumentation purposes.

The sample contains all code and documentation necessary to deploy and monitor the application. Source code for the microservices can be found in the [/src](./src/) folder, while Bicep and Terraform versions (identical output) of the supporting infrastructure can be found in the [/infrastructure/bicep](./infrastructure/bicep/) and [/infrastructure/terraform](./infrastructure/bicep/) folders, respectively.

## Use Case

![Architecture Diagram](./assets/sb-microservice-accelerator-arch-diagram.drawio.png)

A `cargo-processing-api` service (Java) receives a PUT request with an object in the request body containing ports, products, and other cargo related information. The api validates the request schema and places a message containing the cargo object on an Azure Service Bus queue. A `cargo-processing-validator` service (Typescript) validates the internal cargo properties to ensure it can be successfully shipped before placing the cargo object with boolean validation result on a Service Bus topic. Finally, two services (.NET and Python) with subscriptions to the topic receive the final message, filtering for `valid = True` or `valid = False` flags, respectively, before storing the message in a dedicated Cosmos DB container for further processing.

A fifth, `operations-api` service (Java) implements the [async request-reply](https://learn.microsoft.com/azure/architecture/patterns/async-request-reply) pattern, adding a level of resiliency to the long running operation.

Each microservice sends telemetry data to Application Insights, while AKS, Key Vault, Cosmos DB, and Service Bus are each configured to export telemetry data directly to the Log Analytics Workspace associated with the Application Insights resource.

## Docs

Getting started instructions and documentation on observability and monitoring topics within the application can be found in the following pages:

| Topic | Content |
| --------------------------------------- | ----------------------------------------------------------------------------------------------- |
| Getting Started | [getting-started.md](./docs/getting-started.md) |
| Auto vs Manually Instrumented Telemetry | [auto-vs-manually-instrumented-telemetry.md](./docs/auto-vs-manually-instrumented-telemetry.md) |
| Distributed Tracing | [distributed-tracing.md](./docs/distributed-tracing.md) |
| Health Checks | [health-checks.md](./docs/health-checks.md) |
| Custom Dimensions | [custom-dimensions.md](./docs/custom-dimensions.md) |
| Custom Metrics | [custom-metrics.md](./docs/custom-metrics.md) |
| Workbooks | [workbooks.md](./docs/workbooks.md) |
| Alerts | [alerts.md](./docs/alerts.md) |
| Introducing Chaos | [introducing-chaos.md](./docs/introducing-chaos.md) |
| Reducing Telemetry Volume | [reducing-telemetry-volume.md](./docs/reducing-telemetry-volume.md) |
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# [Choice] Node.js version (use -bullseye variants on local arm64/Apple Silicon): 18, 16, 14, 18-bullseye, 16-bullseye, 14-bullseye, 18-buster, 16-buster, 14-buster
ARG VARIANT=16-bullseye
FROM mcr.microsoft.com/vscode/devcontainers/typescript-node:0-${VARIANT}

# [Optional] Uncomment this section to install additional OS packages.
# RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
# && apt-get -y install --no-install-recommends <your-package-list-here>

# [Optional] Uncomment if you want to install an additional version of node using nvm
# ARG EXTRA_NODE_VERSION=10
# RUN su node -c "source /usr/local/share/nvm/nvm.sh && nvm install ${EXTRA_NODE_VERSION}"

RUN su node -c "npm install -g @cadl-lang/compiler"
RUN su node -c "npm install -g cadl-vscode"
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# [Choice] Node.js version (use -bullseye variants on local arm64/Apple Silicon): 18, 16, 14, 18-bullseye, 16-bullseye, 14-bullseye, 18-buster, 16-buster, 14-buster
ARG VARIANT=16-bullseye
FROM mcr.microsoft.com/vscode/devcontainers/javascript-node:0-${VARIANT}

# Install tslint, typescript. eslint is installed by javascript image
ARG NODE_MODULES="tslint-to-eslint-config typescript"
COPY library-scripts/meta.env /usr/local/etc/vscode-dev-containers
RUN su node -c "umask 0002 && npm install -g ${NODE_MODULES}" \
&& npm cache clean --force > /dev/null 2>&1

# [Optional] Uncomment this section to install additional OS packages.
# RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
# && apt-get -y install --no-install-recommends <your-package-list-here>

# [Optional] Uncomment if you want to install an additional version of node using nvm
# ARG EXTRA_NODE_VERSION=10
# RUN su node -c "source /usr/local/share/nvm/nvm.sh && nvm install ${EXTRA_NODE_VERSION}"
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.238.0/containers/typescript-node
{
"name": "Node.js, TypeScript & CADL",
"build": {
"dockerfile": "Dockerfile",
// Update 'VARIANT' to pick a Node version: 18, 16, 14.
// Append -bullseye or -buster to pin to an OS version.
// Use -bullseye variants on local on arm64/Apple Silicon.
"args": {
"VARIANT": "16-bullseye"
}
},

// Configure tool-specific properties.
"customizations": {
// Configure properties specific to VS Code.
"vscode": {
// Add the IDs of extensions you want installed when the container is created.
"extensions": [
"dbaeumer.vscode-eslint",
"/usr/local/share/npm-global/lib/node_modules/cadl-vscode/cadl-vscode-0.16.0.vsix"
]
}
},

// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],

// Comment out to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
"remoteUser": "node"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
node_modules
cadl-output
Loading