add stop+go tests to llama3 recipe, turn off async checkpointing for fp8

[pstjohn]

async dcp checkpointing is currently not working with fp8 model init, so we need to detect this and switch back to synchronous checkpointing. This also adds tests to ensure the dcp checkpoints are functional

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9a3b7783-2cf7-45d4-94b9-abab3d57dded

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[cspades]

Just NaN and infinity right? LGTM, will add golden value parity tests on TE side as well!