Skip to content

TEST_QUESTIONS.md

Latest commit

 

History

History
319 lines (227 loc) · 10 KB

TEST_QUESTIONS.md

File metadata and controls

319 lines (227 loc) · 10 KB

🧪 测试问题集 / Test Questions

使用这些问题来测试AI的推理能力！

Use these questions to test the AI's reasoning capabilities!

💼 职位匹配问题 / Position Fit Questions

英文 / English

  • ✅ "Do you think Oliver can fit an AI agent developer position?"
  • ✅ "Is Oliver qualified for a senior machine learning engineer role?"
  • ✅ "Would Oliver be good for a startup environment?"
  • ✅ "Can Oliver work as a tech lead?"
  • ✅ "Is Oliver suitable for a research scientist position?"

中文 / Chinese

  • ✅ "你觉得Oliver适合AI代理开发者这个职位吗？"
  • ✅ "Oliver有资格做高级机器学习工程师吗？"
  • ✅ "Oliver适合在创业公司工作吗？"
  • ✅ "Oliver能胜任技术领导的角色吗？"
  • ✅ "Oliver适合研究科学家的职位吗？"

预期效果： AI应该分析Oliver的技能、经验和项目，给出理由充分的答案。

Expected: AI should analyze Oliver's skills, experience, and projects, providing well-reasoned answers.

🎯 技能分析问题 / Skills Analysis Questions

英文 / English

  • ✅ "What are Oliver's strongest technical skills?"
  • ✅ "Does Oliver have experience with AI and machine learning?"
  • ✅ "How proficient is Oliver in Python?"
  • ✅ "What programming languages does Oliver know?"
  • ✅ "Does Oliver have cloud computing experience?"

中文 / Chinese

  • ✅ "Oliver最强的技术技能是什么？"
  • ✅ "Oliver有AI和机器学习的经验吗？"
  • ✅ "Oliver的Python水平怎么样？"
  • ✅ "Oliver会哪些编程语言？"
  • ✅ "Oliver有云计算经验吗？"

预期效果： AI应该从文档中提取具体技能，并评估熟练程度。

Expected: AI should extract specific skills from documents and assess proficiency levels.

📊 项目评估问题 / Project Evaluation Questions

英文 / English

  • ✅ "What's Oliver's most impressive project?"
  • ✅ "Has Oliver built any AI applications?"
  • ✅ "Can you summarize Oliver's key projects?"
  • ✅ "What kind of projects has Oliver worked on?"
  • ✅ "Does Oliver have experience with LLM applications?"

中文 / Chinese

  • ✅ "Oliver最令人印象深刻的项目是什么？"
  • ✅ "Oliver做过AI应用吗？"
  • ✅ "能总结一下Oliver的主要项目吗？"
  • ✅ "Oliver做过什么类型的项目？"
  • ✅ "Oliver有大语言模型应用的经验吗？"

预期效果： AI应该描述项目细节，并说明它们展示了什么能力。

Expected: AI should describe project details and explain what capabilities they demonstrate.

🏢 工作经验问题 / Work Experience Questions

英文 / English

  • ✅ "How many years of experience does Oliver have?"
  • ✅ "What companies has Oliver worked for?"
  • ✅ "What was Oliver's most significant role?"
  • ✅ "Does Oliver have leadership experience?"
  • ✅ "What industries has Oliver worked in?"

中文 / Chinese

  • ✅ "Oliver有多少年工作经验？"
  • ✅ "Oliver在哪些公司工作过？"
  • ✅ "Oliver最重要的职位是什么？"
  • ✅ "Oliver有领导经验吗？"
  • ✅ "Oliver在哪些行业工作过？"

预期效果： AI应该总结工作历史，并识别关键成就。

Expected: AI should summarize work history and identify key achievements.

🎓 教育背景问题 / Education Questions

英文 / English

  • ✅ "What is Oliver's educational background?"
  • ✅ "Does Oliver have a relevant degree for tech roles?"
  • ✅ "What university did Oliver attend?"
  • ✅ "Does Oliver have any certifications?"
  • ✅ "What did Oliver study?"

中文 / Chinese

  • ✅ "Oliver的教育背景是什么？"
  • ✅ "Oliver有技术相关的学位吗？"
  • ✅ "Oliver上的是哪所大学？"
  • ✅ "Oliver有什么认证吗？"
  • ✅ "Oliver学的是什么专业？"

预期效果： AI应该提供教育细节，并解释它们与职业目标的相关性。

Expected: AI should provide education details and explain their relevance to career goals.

🔍 对比分析问题 / Comparative Analysis Questions

英文 / English

  • ✅ "Compare Oliver's skills to typical AI developer requirements"
  • ✅ "What makes Oliver stand out as a candidate?"
  • ✅ "What are Oliver's unique strengths?"
  • ✅ "How does Oliver's experience prepare him for AI roles?"
  • ✅ "What skills gap does Oliver need to fill for senior positions?"

中文 / Chinese

  • ✅ "把Oliver的技能和典型AI开发者要求对比一下"
  • ✅ "Oliver作为候选人有什么突出之处？"
  • ✅ "Oliver独特的优势是什么？"
  • ✅ "Oliver的经验如何为AI角色做准备？"
  • ✅ "Oliver在高级职位方面需要填补什么技能差距？"

预期效果： AI应该进行分析性比较，提供深入见解。

Expected: AI should make analytical comparisons and provide insightful analysis.

🎨 创意推理问题 / Creative Reasoning Questions

英文 / English

  • ✅ "If Oliver were to start a tech startup, what would it be?"
  • ✅ "What type of projects would Oliver excel at?"
  • ✅ "How could Oliver contribute to an AI research team?"
  • ✅ "What role would suit Oliver best in a tech company?"
  • ✅ "What emerging tech areas would fit Oliver's skills?"

中文 / Chinese

  • ✅ "如果Oliver创立科技创业公司，会做什么？"
  • ✅ "Oliver擅长什么类型的项目？"
  • ✅ "Oliver能为AI研究团队贡献什么？"
  • ✅ "在科技公司中，什么角色最适合Oliver？"
  • ✅ "哪些新兴技术领域适合Oliver的技能？"

预期效果： AI应该根据文档进行创意推理和预测。

Expected: AI should make creative inferences and predictions based on documents.

⚠️ 边界测试问题 / Edge Case Questions

英文 / English

  • ❓ "Can Oliver work as a brain surgeon?" (Should recognize lack of info)
  • ❓ "Is Oliver good at playing basketball?" (Should say no info, but remain professional)
  • ❓ "What is Oliver's favorite food?" (Should indicate this is not in professional docs)

中文 / Chinese

  • ❓ "Oliver能当脑外科医生吗？" (应该识别缺乏信息)
  • ❓ "Oliver打篮球打得好吗？" (应该说没有信息，但保持专业)
  • ❓ "Oliver最喜欢吃什么？" (应该指出这不在专业文档中)

预期效果： AI应该承认信息不足，但保持专业和有帮助的态度。

Expected: AI should acknowledge lack of information while remaining professional and helpful.

📝 测试步骤 / Testing Steps

1. 启动应用 / Start the App

cd "/Users/pppop/Desktop/Personal AI"
python app.py

2. 逐个测试问题 / Test Questions One by One

从简单到复杂：

  1. 先测试基本事实问题（"What skills does Oliver have?"）
  2. 再测试推理问题（"Is Oliver suitable for X position?"）
  3. 最后测试创意问题（"What startup would Oliver create?"）

From simple to complex:

  1. First test basic factual questions
  2. Then test reasoning questions
  3. Finally test creative questions

3. 评估回答质量 / Evaluate Answer Quality

好的回答应该： ✅ 引用具体文档内容 ✅ 提供逻辑推理 ✅ 给出具体例子 ✅ 承认不确定性（如果适用） ✅ 保持专业和有帮助

Good answers should: ✅ Cite specific document content ✅ Provide logical reasoning ✅ Give concrete examples ✅ Acknowledge uncertainty (if applicable) ✅ Remain professional and helpful

4. 记录结果 / Record Results

问题 回答质量 是否推理 是否引用 备注
Do you think Oliver can fit AI developer? ⭐⭐⭐⭐⭐ ✅ ✅ Very good!
What are Oliver's skills? ⭐⭐⭐⭐ ✅ ✅ Good
... ... ... ... ...

🔧 如果回答不理想 / If Answers Are Not Good

检查清单 / Checklist

  1. 文档质量 📝

    • oliver-knowledge-base/ 中有足够的 .md 文件吗？
    • 文档内容详细吗？
    • 包含具体的项目、技能、经验吗？

  2. 向量数据库 💾

    • 删除旧的 vector_db/ 文件夹
    • 重新运行 python app.py 来重建数据库

  3. 参数调整 ⚙️

    • 尝试增加 k 值（改成 7 或 10）
    • 调整 temperature（0.5 到 0.9 之间）

  4. 问题表达 ❓

    • 问题是否清晰和具体？
    • 尝试换个方式表达问题

💡 最佳实践 / Best Practices

✅ DO - 好的做法

  1. 提供详细文档

    • 在简历中包含数字和成果
    • 描述项目的具体贡献
    • 列出技能的熟练程度

  2. 问清晰的问题

    • "Is Oliver good at Python?" ✅
    • "Tell me about Oliver" ❌（太宽泛）

  3. 测试不同类型的问题

    • 事实类：What, Where, When
    • 分析类：Why, How
    • 评估类：Can, Should, Would

❌ DON'T - 避免的做法

  1. 模糊的文档

    • "Knows programming" ❌
    • "Proficient in Python (5 years), built 10+ ML models" ✅

  2. 太宽泛的问题

    • "Tell me everything" ❌
    • "What are Oliver's top 3 skills?" ✅

  3. 期望AI知道文档外的信息

    • AI只能基于你提供的文档进行推理

📊 性能基准 / Performance Benchmarks

目标指标 / Target Metrics

问题类型 目标准确率 目标响应时间
事实查询 95%+ < 3秒
推理分析 85%+ < 5秒
创意推理 75%+ < 7秒

回答质量评分 / Answer Quality Scoring

  • ⭐⭐⭐⭐⭐ (5/5): 完美 - 引用、推理、具体
  • ⭐⭐⭐⭐ (4/5): 很好 - 有推理，缺少部分引用
  • ⭐⭐⭐ (3/5): 一般 - 基本回答，推理不足
  • ⭐⭐ (2/5): 较差 - 答非所问
  • ⭐ (1/5): 很差 - "I don't know"

🎯 期望改进 / Expected Improvements

之前 vs 之后 / Before vs After

指标 之前 之后 改进
推理问题成功率 20% 85%+ +65%
回答详细度 1-2句 5-10句 +400%
引用具体例子 很少 经常 ✅
用户满意度 ⭐⭐ ⭐⭐⭐⭐⭐ +150%

开始测试吧！祝你玩得开心！ 🚀

Start testing! Have fun! 🚀