Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions content/event/260211.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
title: "From Empirical Study to Runtime Mitigation: Addressing Integration Failures in LLM-Enabled Software"

event: Weekly Talk
event_url:

location: MR24@COM3-02-64
address:
street:
city:
region:
postcode:
country: Singapore

summary:
abstract: "Large language models (LLMs) and Retrieval-Augmented Generation (RAG) are increasingly integrated into software systems to realize intelligent features. However, this integration poses significant challenges due to undefined interface specifications, diverse software context requirements, and complex system management.In this talk, we first present a comprehensive empirical study on the correctness of LLM integration. By analyzing 100 open-source LLM-enabled applications, we identified 18 distinct defect patterns located across the LLM agent, vector database, software components, and system management. Our study reveals that integration defects are widespread, with 77% of these applications containing more than three types of defects that degrade functionality, efficiency, and security. To facilitate future research, we constructed Hydrangea, a defect library containing 546 identified defects.Guided by the findings from our empirical study, we then introduce Comfrey, a runtime framework designed to prevent integration failures in LLM-enabled software. Serving as a middle layer between AI and software components, Comfrey automatically detects and resolves potential integration failures through a three-stage workflow targeting format, syntax, and repetition errors. Our evaluation demonstrates that Comfrey effectively detects 75.1% and prevents 63.3% of potential integration failures with only 8.4% overhead, significantly outperforming existing baselines."

# Talk start and end times.
# End time can optionally be hidden by prefixing the line with `#`.
date: "2026-02-11T14:00:00Z"
date_end: "2026-02-11T15:00:00Z"
all_day: false

# Schedule page publish date (NOT talk date).
publishDate: "2026-02-02T00:00:00Z"

authors: [Yuchen Shao]
tags: [Weekly Talk]

# Is this a featured talk? (true/false)
featured: false

image:
caption: 'Image credit: [**Unsplash**](https://unsplash.com/photos/bzdhc5b3Bxs)'
focal_point: Right

url_code: ""
url_pdf: ""
url_slides: ""
url_video: ""

# Markdown Slides (optional).
# Associate this talk with Markdown slides.
# Simply enter your slide deck's filename without extension.
# E.g. `slides = "example-slides"` references `content/slides/example-slides.md`.
# Otherwise, set `slides = ""`.
slides:

# Projects (optional).
# Associate this post with one or more of your projects.
# Simply enter your project's folder or file name without extension.
# E.g. `projects = ["internal-project"]` references `content/project/deep-learning/index.md`.
# Otherwise, set `projects = []`.
projects:

# Slides can be added in a few ways:
#
# - **Create** slides using Wowchemy's [*Slides*](https://wowchemy.com/docs/managing-content/#create-slides) feature and link using `slides` parameter in the front matter of the talk file
# - **Upload** an existing slide deck to `static/` and link using `url_slides` parameter in the front matter of the talk file
# - **Embed** your slides (e.g. Google Slides) or presentation video on this page using [shortcodes](https://wowchemy.com/docs/writing-markdown-latex/).
#
# Further event details, including page elements such as image galleries, can be added to the body of this page.

---
Speaker Info:

[Yuchen Shao](https://ycshao12.github.io/) is a third-year Ph.D. student at the Software Engineering Institute, East China Normal University (ECNU) and the Shanghai Innovation Institute, co-advised by Prof. Chengcheng Wan and Prof. Ting Su. Her research interests lie in SE/Sys for AI and software testing. Her recent work centers on the correctness and reliability of Large Language Model (LLM) integration in software systems, including analyzing integration patterns and mitigating runtime failures in LLM-enabled software.
67 changes: 67 additions & 0 deletions content/event/260304.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
title: "Software-Style Hardware Testing: Information-Flow Tracking, Fuzzing, and What ML Might Add."

event: Weekly Talk
event_url:

location: MR24@COM3-02-64
address:
street:
city:
region:
postcode:
country: Singapore

summary:
abstract: "Modern computer hardware keeps paying a performance tax for vulnerabilities discovered too late. Each generation brings dozens to hundreds of post-silicon security issues, and fixes often arrive as heavyweight patches such as microcode updates, extra barriers, or tighter speculation controls. Over time, these mitigations permanently shift the performance and security balance. This talk presents pre-silicon hardware security validation techniques inspired by software testing and verification. The focus is on information-flow tracking and how it can be combined with formal analysis and fuzzing to surface security-relevant behavior, including secret leakage, cross-privilege flows, and architectural bugs such as crashes. The goal is to find these issues before tape-out, when they are still cheap to fix, do not require permanent performance trade-offs, and do not expose security risks in the wild. A major obstacle is evaluation. Hardware security still lacks a widely used public benchmark of vulnerabilities, which makes fair comparisons between techniques difficult. The talk discusses a step toward closing this gap through systematic bug injection. This direction also points to the exciting direction of adversarial validation loops where a verifier is trained against a bug injector, and both improve over time."

# Talk start and end times.
# End time can optionally be hidden by prefixing the line with `#`.
date: "2026-03-04T14:00:00Z"
date_end: "2026-03-04T15:00:00Z"
all_day: false

# Schedule page publish date (NOT talk date).
publishDate: "2026-02-02T00:00:00Z"

authors: [Flavien Solt]
tags: [Weekly Talk]

# Is this a featured talk? (true/false)
featured: false

image:
caption: 'Image credit: [**Unsplash**](https://unsplash.com/photos/bzdhc5b3Bxs)'
focal_point: Right

url_code: ""
url_pdf: ""
url_slides: ""
url_video: ""

# Markdown Slides (optional).
# Associate this talk with Markdown slides.
# Simply enter your slide deck's filename without extension.
# E.g. `slides = "example-slides"` references `content/slides/example-slides.md`.
# Otherwise, set `slides = ""`.
slides:

# Projects (optional).
# Associate this post with one or more of your projects.
# Simply enter your project's folder or file name without extension.
# E.g. `projects = ["internal-project"]` references `content/project/deep-learning/index.md`.
# Otherwise, set `projects = []`.
projects:

# Slides can be added in a few ways:
#
# - **Create** slides using Wowchemy's [*Slides*](https://wowchemy.com/docs/managing-content/#create-slides) feature and link using `slides` parameter in the front matter of the talk file
# - **Upload** an existing slide deck to `static/` and link using `url_slides` parameter in the front matter of the talk file
# - **Embed** your slides (e.g. Google Slides) or presentation video on this page using [shortcodes](https://wowchemy.com/docs/writing-markdown-latex/).
#
# Further event details, including page elements such as image galleries, can be added to the body of this page.

---
Speaker Info:

[Flavien Solt](https://flaviens.github.io/) is an Assistant Professor at the National University of Singapore. Previously, he was a postdoctoral researcher in the SLICE Lab at University of California, Berkeley, working with Chris Fletcher. He received his PhD from ETH Zurich in 2024 under the supervision of Kaveh Razavi, and was awarded the ETH Medal.
4 changes: 2 additions & 2 deletions content/event/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,10 @@ Timetable for upcoming events in AY25/26 (subject to changes):
| March 18 | | --- | ---|
| March 11 | <font color=brown>Qiyu Presentation</font> | --- | ---|
| March 10 | <font color=green>Reading Group</font> | Yibo | ---|
| March 4 | <font color=blue>TBD</font> | TEST-lab | Flavien Solt|
| March 4 | <font color=blue>Software-Style Hardware Testing: Information-Flow Tracking, Fuzzing, and What ML Might Add.</font> | TEST-lab | Flavien Solt|
| Feb 25 | <font color=gray>Recess Week</font> | --- | ---|
| Feb 18 | <font color=gray>Skip: Chinese New Year</font> | --- | ---|
| Feb 11 | <font color=blue>Comfrey: Mitigating Integration Failures in LLM-enabled Software at Run-Time </font> | Yibo Dong | Yuchen Shao |
| Feb 11 | <font color=blue>From Empirical Study to Runtime Mitigation: Addressing Integration Failures in LLM-Enabled Software</font> | Yibo Dong | Yuchen Shao |
| Feb 10 | <font color=green>Reading Group</font> | Junwen | ---|
| Feb 4 | <font color=brown>Suyang Dry Run</font> | --- | ---|
| Jan 28 | <font color=gray>Skip</font> | --- | ---|
Expand Down